Make other plans and in all honesty it's so dangerous right now to hang out in groups and expose yourself to Covid. It is so not worth it for you to even care about what this other guy thinks, it sounds like he prefers to control this friend of yours and sees you as some kind of threat to that need for control. This has nothing to do with how nice you are either, it's the control he is after that you pose a threat to.