It sounds like he’s clinging on and/or trying to trap you into staying, to me - him making moves to get a larger house/mortgage against your own desires is a big concern here, regardless of his attitude to materialism. Ditto the physically forcing you to be affectionate towards him, when you’ve already told him you’re trying to work on it. He can’t dictate the speed you make progress.
I get you don’t want to hurt him, that’s understandable. But, does he care that his actions are hurting you?