Last night I had a "thing" with my boyfriend. I don't think you could call it a fight; I don't think he even realizes I am upset. I don't even know if I have the right to be upset about it.

First, some history: I have a seizure disorder. Between the two seizures and the drugs they put me on, my sex drive was knocked out completely. My boyfriend has put up with this, even though we haven't had sex for months now. Trouble with the doctors meant that it took a long time for them to switch the drugs out, so he has had to deal with a lot. I give him credit for that.
The trouble comes when he makes continual little comments about sex. Last night I jokingly took his soda can from him and asked what he would do for a Sierra Mist. He said he would (to paraphrase) [perform oral sex on me] or rub my [breasts]. I got upset and muttered that he never fails to remind me that I am failing him.

I hate feeling like I somehow "owe" him sex in the first place. I have been more than reasonable: I have even offered to allow a discreet affair, because I do think he deserves to have a sex life. But I already feel horrible about the fact that I don't want sex anymore (more than that even, I am repulsed by sex) and I miss sex terribly.And I feel bad that he and I are not enjoying the sex life we used to. Am I wrong to be upset when he suggests, jokingly or not, that I just jump right into a sex act? Or does the fact that he is nicely giving up his sex life give him the right to remind me of that whenever he wants?

I don't even feel like I can bring it up to him. To us a thoroughly non-sexual analogy: if you moved into your aunt's house rent-free it would be so nice of them to let you live there that you wouldn't really be able to complain if she reminded you every day that you were living there rent-free.

Am I being overly sensitive to get upset at this? Am I right to let it bother me? If I am, how do you talk to someone about something like this?