For me, I think I consistently say “on” with islands if I’m thinking of them as islands, but “in” if I’m thinking of a (possibly coextensive) political unit—e.g.,
“in Manhattan” = ‘in the borough of Manhattan’
“in Cuba” = ‘in the country of Cuba’
“in Manhattan” = ‘in the borough of Manhattan’
“in Cuba” = ‘in the country of Cuba’
Comments
(Then again, island is right there in the name.)
“on PEI” = the island vs. “in PEI” = the province