Bless your heart....
No, I don't think you did anything wrong, and I can't imagine why he would get upset...perhaps he didn't understand what happened? It seems very odd to me he would be upset about this relatively innocent incident. :/
I am sorry you are in the situation you are.

I hope things improve for you, and I will keep you in my prayers.