You could always call and leave a message on his machine, saying why you are discontinuing with him, if you want closure. Something along the lines of "What you said about X at our last session made me feel uncomfortable and unsafe, therefore, I've decided to seek therapy elsewhere."

That was my addictions Dr's advice to one of the women in my recovery group when a therapist had crossed the line with her.

Good luck in whatever you decide to do.

--splitimage