I had a lot of repressed sexual energy, and I dealt with it in all kinds of ways, many in which I was ashamed of. Dealing with the cause of this repression is the only thing that helped me understand my urges, urges that did not seem to fit with the person I thought I was.
And I was right, it doesn't fit, it's a dysfunction due to the repression, only therapy allowed me to see it, so mostly, the guilt is gone and the urges are changing.

I'm not saying that fantasies are disfunctional, but in my case, they were because they were influenced by childhood trauma.

Hang in there, and do your best to keep fantasies within your comfort zone.