I would just let this go. He's your cousin. You got along well enough before this to invite him to live with you. That alone should warrant a second chance.

Also, he might have been saying this because he doesn't want to see your WIFE again, and it might have nothing to do with you personally.

I know it sucks when someone appears unappreciative of the kindnesses that you give. Maybe you two could have a chat about it sometime when your wife is not around and get his side of the story before just cutting him off completely.