It's similar to gentrification, if you're more familiar with that. The natives on these Pacific Islands don't want their culture to be diluted or eventually lost due to an outside group being able to vote and make decisions.
I'm not familiar with the law which prevents immigrants from voting there, but I understand why the native population would be against outsiders impacting their culture and politics.