He has crossed the line many many times and seems to think that he can carry on being your therapist, pretending nothing happened. Even if he is no longer behaving inappropriately, the fact that he refuses to acknowledge what he did is very problematic. Therapists do F up but the good ones should be able to acknowledge it and apologise... and most likely refer you to a new therapist, to mitigate any further damage.
What do you think he will say if you confront him? What do you want him to say? Going by his previous behaviour, you might not get the response you hope for. Could you find someone else, along side seeing him, just while you are able to detach from him?