Have you ever actually experienced sexual attraction to someone else? (Note that sexual attraction isn't the same as having a libido or the same as wanting a relationship or even finding someone nice to look at). If not, you could be asexual (rather than heterosexual, I assume) with a libido and that could be why you've never really been interested in partnered activities and subsequently didn't really enjoy it.

For clarification, I am asexual myself, which is why it occurred to me to volunteer the possibility.