I have been in your shoes, and I have also beaten myself up over it. I think you need to hear what I needed to hear.

It. is. not. your. fault.

You are not unattractive because he has issues. Having more sex won't help. Getting a boob job won't help.

Get both of you in couples counseling, and individual if you don't already!!