The reason I brought up self-esteem, is because these issues are usually related to some lack of it. I could be wrong though. I think fear of intimacy, as well as the need to guard your emotions, definitely has something to do with that. Especially since it seems you greatly fear getting hurt. I also feel that your issues with family, as well as alienation from friends, is also a contributing factor. Sex is heavy stuff, especially for the first time. The fear of rejection and being used is a very real fear. I probably phrased the fact that you need a lot of work poorly. I just meant regularly therapy (at least what you can afford) will be beneficial for sure.There are a lot of other things that you can do on your own time in the meantime, in order to start feeling more secure with yourself and other people. There are tons of books about fear of intimacy. I would even try googling it. There Is also a lot of info on the net, which can help give you a better understanding of what may going on with you. It's difficult learning to trust people, especially when you have been let down so many times.
The good thing is that you are able to please yourself and have no problems with arousal. If you did have a pretty problem, I think this would be a more complex issue.
Anyway, forget about your age and take time to become truly comfortable with yourself and at peace with past situations that may be haunting you.
Btw, I was a virgin until I was 20. I didn't have the exact problem as you, but I did feel guarded and a had a huge fear of intimacy. Mainly due to the fact that I went through a very awkward phase in High School and was teased terribly. It took time to build myself up, before I felt comfortable with that level of intimacy. It was indeed a scary thought! However, when I did actually have sex for the first time, it was with the right person, who I felt totally comfortable with!

Sent from my iPhone using Tapatalk