It seems like if you really loved him, you could be intimate in other ways than just having sex with him. There are still ways of being together & being satisfied that aren't strictly limited to having intercourse. Seems as though he's craving intimacy with you in those ways or he wouldn't be getting mad because you choose to not be intimate at all.

You chose to be his wife for better or worse.....that means that you find other ways to have the intimacy WITH HIM since this seems to be a problems that he has had for a long time. If you truly loved him, you wouldn't be just "nice enough to stay with him despite of it". You would be WORKING WITH HIM to find ways for both of you to find satisfaction in the intimacy side of your marriage instead of making it into a battle, or worse, into a war which it will become if you aren't willing to work together on this.