I think telling you that you should limit your contact with others because of a confusion/misunderstanding is concerning to me. It sounds like he’s asking you to isolate yourself. That could be harmful to your well-being,