Emre:

At least you recognise that you're not ready for one! Maybe that's part of the reason it's hard to fantasise? Like I said, I have a hard time doing anything alone. I either get board half way through, or am just not that into it. Masterbating became a chore for me for a while unless someone else was there at the time. Even now, anything solo is just annoying lol.

I'd say that unless it's causing you physical discomphort, or anxiety, don't worry too much about it. If it is causing you anxiety, or has gotten to the point it's physically uncomfortable or unhealthy then talk to your GP or T. It's not really an issue unless it's causing issues.