"You looked angry to me" would cover what you were actually aware of without adding anything. After that you and the other person could choose either to move on to other topics, or to sort out what you experienced at that moment from what they did.
I see that not so much as the kind of rule where there's someone in charge of enforcing it, as a rule of thumb: if you comment on certain things in certain situations it's likely (not guaranteed) that the conversation will go in directions you won't be comfortable with. You then have the choice of steering clear of the subject and not starting that conversation, or starting it anyway and being prepared to deal with whatever it might bring up for you.