Wow, that is pretty shocking. I'm sorry he said that to you.

He did seem to realize that what he said was bad, took it back, and acknowledged the effect it's had on you and your relationship.

If he's been good for 18 months, if it were me then I'd just be as angry as I wanted to be at him and let him have it for as long as I wanted. And see if we could repair things. But not until I had gotten out all my fury which might take a long time, really.

People screw up and, honestly, in therapy do weird things that sometimes not even they understand. That includes therapists who can find themselves acting out of character. It's possible that your therapist doesn't even understand why he said what he said and is equally horrified by it. If he takes responsibility and is nondefensive and is generally a good therapist otherwise, I personally would try to see if the relationship could survive. But you're not over-reacting at all, that's a horrible thing to say.