I fail to see what your past has to do with now, you have not cheated on him with anyone. He sounds deeply insecure. I am very sceptical of his therapist's claim that people like you don't change - that does not sound like the kind of thing a therapist would say (therapists know more than anyone we can and do change!) I am wondering if he is making that up to add weight to his own opinion.

Really he sounds deeply insecure about his own worth. Not sure what you can do about that, if he recognises it and wants to change it might be possible to find a way forward but dumping all his insecurities on your past is not going to help.