I'm sorry this has happened to you. You must be really hurt. I would set-up some boundaries if you want to stay with him. I would tell him flat-out: if I can't be the only woman in your life, then I don't want to be in yours.

I hope everything works-out for you. I wish you the best of luck!