written correspondence wins over in such cases
I once was a TA for a professor who was from China and who obtained a PhD from Princeton.
He could not pronounce English words in such a way as to make his point understood. The issue was not mine, but his - I have since then worked with dozens and dozens of Chinese people and would always understand them fine. So it was his issue.
The dissertation that he wrote was on some very boring, convoluted, and highly quantitative subject, so the writing of his dissertation did not require really good English proficiency. I still thought that Princeton should not have bestowed a doctorate title upon somebody who could not have passed the very basic speaking fluency test, but, unfortunately, Princeton did not (and still does not

) ask for my opinion on how to handle its affairs.
So - it was ugly. It was unpleasant. He would get defensive, upset, and all of that would make him speak with even less clarity.
Then I started emailing him. Everything by email. I never talked to him again. But at least I knew his expectations of me.
I do realize that a husband is not somebody you simply work for part-time, so switching to 100% written exchanges is not an option, but switching to more emails/texts and less talking would be prudent. Writing creates a stable record that persists over time. You cannot claim not having received a message, right? But you can - and your husband does! - claim not having heard or understood a spoken utterance.
That said, he clearly needs to see a hearing specialist, a neurologist, and a psychiatrist - all three.