I'm concerned about how he reacted to how you felt about the situation of going out. If he didn't want the answer he shouldn't have asked. His anger is not generally a good sign.

If you disagree with something - be assertive like you were...Don't at the end give in "I then said if you want my wishes to go then go". Stand your ground.