Have you considered going to an IVF clinic for sperm donation rather if you want a baby because you're nearing 40?
I say that because I think you need to cut ties with this guy. Your relationship doesn't seem to be serving your mental health very well