you should feel uncomfortable and I feel you did the right thing by calling him out on it. if this relationships is important to him, he will be mature about and stop being a "sheep" and follow his friends actions.

good luck