Welcome to PC!

It's hard when people don't understand how hurt we get when we're excluded. You have my sympathies, since it's happened to me a lot of time too. Have you tried talking to him and telling him how much it hurts you? Or tried arranging time to hang out, and invite him and other people to hang out - on your terms, and not theirs?