Looking from the other side of your situation, please don't divorce your husband over this. Your kids need you and him. I've met men who want affairs and tell them to leave me alone when they tell me about their wives and kids. Most women don't want baggage either. You will get over this and be ok.