As far as the board game, if you offended person A but the other players agreed with you then you probably just called them out on something legitimate. Sounds like you were in the right on that and had the balls to call them out when other people didn't - if other people jump on board when someone else says it then usually that person is in the right. Maybe person A is slightly bitter about it but if other people backed you up they probably appreciated your having the guts to say it.

Also I don't think person B could blame you for not driving. I mean if you don't drive and they know that it would be stupid and bitter to make a snarky comment about how they have to drive and are such a victim because you don't... maybe they just feel like they always have to drive and were saying that to you because they are annoyed nobody else would drive (knowing that you don't so can't offer to).

I guess I can't speak to the specifics so much but I always jump to conclusions that other people are rejecting me or that I don't belong or whatever so I know what you mean at least... I just don't become part of social groups anymore because it's too hard for me to navigate the kind of situation you are describing. In this situation I would be liable to just stop talking to the whole lot of them. But the better idea might be to go out with them next time you feel up to it. Maybe you will see it was really nothing and you were just overthinking it (something I know all too well).