I'm not sure if you're asking "why" you haven't been allowed to feel uncomfortable going downtown or "why" the abuse happened.
First, no one has the right to tell you how to feel. Your feelings are your own. Feelings are neither good nor bad; they just are. It's what you do with your feelings that matters.
As for why the abuse happened: It was told to me by a truly Christian woman that everyone was given "free will." Our abusers were exercising that free will that God gave them. It's between them and God what happens for using it in a bad way. They will have to account for their actions, not us. We were innocent victims.
The secret to
overcoming (not just surviving) is to go through therapy, prayer, etc., whatever it is that
you need to do for yourself so that you don't remain in that victim role. It's a long hard road, but you don't have to walk it alone. There are many that can walk it with you.
Have you considered a male therapist? There are also support groups that you can attend. It may be possible to find a group of men or at least some men in the group. You are
not alone.
<font color=blue>"Our doubts are traitors and make us lose the good we oft might win by fearing to attempt" --Shakespeare</font color=blue>
__________________
Psalm 119:105 Thy word is a lamp unto my feet, and a light unto my path.