You feel how you feel, there is no need to think you are wrong for feeling uncomfortable. It sounds like he is short-tempered and likes to be a one-upper instead of sympathize with you. Personally, I would tell him how you feel and see if he is willing to be more sympathetic to your feelings.