golden and Tisha said pretty much what I was going to say, but in 2 different posts.

No, you are not being selfish. If it was a one time thing and he very rarely got to go out then sure, I could see him deserving a little time away, but still not an expensive 4-day vacation in which he is clearly not considering the needs of his family if he wants to spend like that. However, it was absolutely inappropriate for him to push and pin you onto the bed in anger. It might seem minor, but that is physical abuse. On top of that, he sounds quite controlling from what you've said here. I don't want to scare you, but most of the time abuse doesn't get better, it only gets worse, and frankly he seems too selfish to recognize he would be in the wrong and get help.