Stupidme,
I have to agree with your pastor crossed every ethical line in the book. It is against the law for counselors to become sexually involved with their clients. You put a level of trust into him and he used that trust. You are not to blame, but I know that you deserve better in a partner and in a therapist then this guy.
Please get in contact with your local rape crisis center. Many are non profit and could help you for free. Please you deserve to help yourself.
By the way...I do not think you are stupid.

Stay strong,
Jessica