Obviously this has to be your decision, but here are some things that I would consider if it were me: In the last few months has seeing him made you better or worse? One bad session isn't enough for me to give up...but a pattern over months? Have you directly and specifically told him what bothers you and asked if he can change something and he refuses?

If you do decide to quit, will you feel better if you have some closure? Writing a letter is fine if that is all you can manage, but I would want him to respond to me, even if he were defensive. The one time I fired a therapist for cause I called him up and told him. It kind of felt good too...like I was dealing with him as an adult.

And yes it is hard to start over with a new therapist, but I promise he's not the only good one out there. And I've noticed that whenever I switch therapists I start making more rapid progress for awhile.