I'm so sorry. He sounds cruel in the way people can be from a bad, depleted place in their own lives. His not holding boundaries and then blaming you for that is not on you. This seems like a place of acute failure in his life, from which he has little to give. This happens in real life love too, ego blows can make people out of emotional reach for a while.

Therapy is so emotionally dangerous. I am so sorry.