This is just objectively correct from a usability perspective and from "how socializing works"
If you're building a social space, the people in that space will want to exclude people they don't like.
You either become more likeable or stop trying to talk to them and talk to your friends.