Well a lot of people have trouble with the holiday, especially with dysfunction in the family.
So you had a decision to make about where you wanted to spend the day, and did have another choice. So to your guy, you had already decided and when you wanted to change at the last minute, not sure he got why. And he was with his family, and if you were a little drunk or whatever probably did not want a lot of drama when he was eating.
Yes you should be able to go to your partner when you are in distress, but he was enjoying his holiday, and you weren't in any life threatening trauma so I don't really see that he did anything wrong.