Thanks for bringing your concern here to PC, nesaya. I see this is your first post. So... welcome to Psych Central.

I don't know as there is a lot I could add to what has already been said in the other replies you have received. I guess the one thing I would say is that, regardless of anything else, this is a complicated set of circumstances you don't need, in my opinion. Here are links to 4 articles, from Psych Central's archives, that talk about what healthy romantic relationships look like. Perhaps reading through them can be helpful to you in putting what you are experiencing with this man into perspective:

Are You In A Healthy Relationship?

What Makes Relationships Happy and Healthy? Couples Therapists Weigh In

What Healthy, Happy Couples Do and Don't Do

How Healthy Couples Manage Conflict

I hope you find PC to be of benefit.