I have no trouble at all with your response to James, except that you let it go on too long, expecting him to "care". He doesn't so I would not either. The problem I have is with your boyfriend and his reaction. If he has a friend like this, he can't be enough "different" that he is likely to treat me with respect? I don't see any respect for you from your boyfriend that when you told James to shut his face your boyfriend did not back that; it's your home, you should not have to go in the other room, he should have been asked to leave by your boyfriend and/or the two of them should have left, etc.