Hey Genesis,
Hope today is a much better day for you. I can't say too much on whether or not you were abused because, like you said, you have no memory of anything like that.
But something just struck a cord in me when I was reading your post. Are you sure it isn't OCD? I was reading into some of the symptoms of OCD and I saw this story of a guy who when he has sex with his girlfriend, can't get the thought of his naked mother out of his head. He apparantely got incredibly freaked out by this and a lack of interest in sex ensued. That doesn't mean he was "turned on" by his mother, it just meant a disturbing thought got stuck in his head and lo and behold he obsessed about it. I can't say for sure but maybe you have some disturbing obsessions with sex!?!
From my own experience, I can relate 100% to what you are saying, except I am very much aware of the fact that I have been sexually assualted. At first I was sure it hadn't impacted on me, but now I see where things have started to go pear shaped.
Lately my own OCD has turned away from thinking disturbing violent thoughts and back towards thinking VERY disturbing and disgusting thoughts of child molestation. I too worry that I "enjoy" these fantasies and am just a sick pervert, but in the end I know it's not what I want to be and most definitely NOT what I am attracted to. But it has still had a pretty devastating effect on my own sexuality as I think all sex, regardless of age is paedophilia, even the relationship I'm in right now with my girlfriend who is 18 months older than me.
Winding up, probably research sexual OCD for yourself and see if anything relates there. Don't worry you ain't a freak, there are a lot of people out there with disturbing sexual thoughts (including me) so you are definitely not alone.
Hope this helped but more importantly I hope you're doing much better.
Cheers