You are not in the wrong. They are crossing the line and it’s extremely disrespectful to you. if It were me, I’d probably walk out the door. It’s practically cheating. They are definitely doing something wrong. Tell him why don’t you two just have sex and get it over with. I’m leaving.