Think about it this way - how good will you be to your husband, kids, etc. if you're miserable? Does he really want you to feel that way? If I were him I'd rather know that you were home feeling good and taking care of yourself.

Sometimes being selfish is the healthy thing to do