Sounds like maybe he is unhappy with himself. It's always impossible to know only having one side of the story. But you need to hold your end of the bargain, too. You are letting him out of these appointments. Maybe he needs a kick in the @ss. I don't like ultimatums but sometimes that's the only way to get results. Either we go to counseling and you try or I'm gone. But don't make any "threats" you can't keep. Because then you will fall back on your word and he will only get the message that he can continue this behavior.