A relationship is between 2 people who are equally committed in it. Irrespective of whether you are facing issues or he's facing issues, it doesn't sound like your relationship is in the right place.

I won't say break up, but I would say, don't agree for a long term commitment now.

It would be a good idea to meet a couple's therapist, who would probably be in a better position to see things from a different perspective.