It doesn't look like this is a functional relationship for either of you.

He was already married, is now divorced, and doesn't want to go through that again with you or any woman. You, however, want those things with him, and are looking for ways to convince him to give them to you: marriage and children.

He also wants to move to his small hometown, but you don't want to because that would be inconvenient for you. You aren't willing to compromise because the move would take you away from everything and everyone you are familiar and comfortable with, and it would require you to quit your job and try to find a new one.

To me this doesn't seem like a good relationship for either of you. Neither of you want the same things and aren't willing to accommodate each other's needs.

So, from an outsider's perspective, I'd say you're probably better off ending the relationship and finding a man who will give you marriage and children, won't force you to move away so that you leave your friends, family, and your job.

When my mother married my father, he swore to her he'd never force her to leave her job or friends or city; but when I turned 11 years old my father went behind her back and interviewed for another job that he was offered and bought a house without her permission near his family.

He did this all without consulting her. So, she had a choice: go with her husband and children despite knowing she'll lose everything that gives her happiness and joy, or stay behind, divorced, away from her children still employed, close to her family and friends. Well, she went with my father anyway, and has spent the next 36 years on antidepressants and being miserable and received a mental illness diagnosis and is now a widow and still antisocial and in the early stages of dementia.

As her oldest child, I vowed I would never let a man dictate my choices to me about where I live, where I work, who my friends are, etc,. so that is one of the reasons I remain single (there are other reasons but not letting a man control my life, is the main reason I'd rather be single than settle being with the wrong man b/c I am lonely).

Obviously, do what you think is best for yourself. But my perspective is that you are making a huge mistake staying with this man, esp. if you know he expects you to stay with him, unmarried without children from him, and that you will agree to move 3 hours away with him that will disrupt your life as you know it right now. If that is what you want, then stay with him. But those choices don't seem very fair to you because that's not what you want. He doesn't want to give you what you want. So, you need to decide if he's worth staying together with.