An important truth that you have to learn is that you cannot control anyone's actions but your own. You are the only person you are responsible for, because you can't control other people. It's possible that you've caused your husband pain, but he has no right, NONE WHATSOEVER, to make his troubles out to be more than yours. He should be ashamed for trivializing your feelings. All he's doing is putting HIS needs and wants before yours, and that is not love, that is selfishness. It's perfectly alright to stand up for yourself.

If he's so "depressed" (and I use quotes because I think that he's just playing a guilt game) maybe he should see a therapist. You can't carry the burden of his actions. He's a big boy.

mckell13 makes a good point. This relationship doesn't sound healthy for either of you. Maybe some time apart will be good for you.