Yes it would bother me, and I don't think it's fair if him to call you resistant in front of the group. I don't think his reply means he doesn't want you in the group, more likely he doesn't know how to respond to being called out on his error. He really should learn from this, and in terms of his relationship with you and the group dynamics, I hope you can have an honest conversation when you see him.