I wouldn't be able to trust someone who was sleeping with me and someone else. It seems like he wants all the fun, but none of the responsibility. I think stopping the sex will help you a lot.