Sounds like maybe you haven't had any therapy for the abuse? It's most probably anxiety that is causing this fear in you or maybe some depression, or... a combination of both. They usually come hand in hand.

If you don't want to see a therapist, which I would strongly suggest, you might want to talk to your MD about anxiety. There are anxiety meds you can take to help with that.