well done!! I think you need to file a complaint and get this sorted out otherwise it will be on your mind forever.
I have no opinion about whether or not he was inappropriate but my advice is if this goes anywhere (although you probably already know) you will be questioned and asked whether it was your imagination, delusions etc etc especially with a history of hospitalisation and other mental health disorders - so with that in mind, it would be good to have everything straight in your head to be able to say what happened. you may have already done this but Maybe you should write everything you can remember down with dates etc and everytime you remember something else record it because you are likely to go blank when asked anything. Also it is possible he chooses victims that he knows may not be believed if they complain - but I guess him having a history of prior complaints is in your favor and also having other specialists thinking you are totally credible.
And I think it is good people here question you because you will get a lot worse questions if this is persued.

Good Luck, Stay strong and keep us updated.