Sorry that was so difficult. I really hope that things work out for you. If you can handle telling him how he made you feel, either in writing or verbally I think you should. But if you can't handle that...it's ok to just move on.