Thanks everyone for your replies.
Mike_J He is not interested in sex at all and no masturbation going on (as far as I know)
I don't want a divorce, just want to sort this out.
Webgoji This all definitely started when he found out he was infertile. Must be something to do with that.
I don't know how to get to the bottom of it on my own and he refuses any kind of therapy..
Any other infertile men out there, with ideas?