At least at the surface, that sounds to me as if he believes holding you hostage in another state (a dramatization on my part) is necessary for protecting his relationship with you from possible damage by your family. But if your family can/could actually do that, then I would say the two of you need deeper commitment and trust between each other anyway. My own in-laws detested me for a time many years ago, and actually for good reason from the earliest days of my marriage with their daughter, but where my wife and I lived made no difference whatsoever because of the no-compromise commitment and trust my wife and I still share to this very day. She never argued with them about me, I stayed off their property as I had been told and without complaint while giving them all the time they needed or wanted to see I had changed and that their daughter was well-loved and being treated properly. I have no idea what your family's issues with your potential spouse might be, but those really do not matter. What matters is what the two of you have together even if nobody else ever sees, believes, acknowledges or accepts it.