Don't just assume your husband is having an active affair with this woman and is actually leaving you. It sounds to me from what you have shared that he is lost and confused. Some people who struggle and feel lost tend to run away and want to sit and think and have someone to talk to, someone that eases their sense of powerlessness. Some people do stupid things out of a personal sense of loss and desperation. Could be what your husband likes is simply being heard and getting attention.

Often couples drift apart from being busy and handling things that they don't spend time with each other sitting and talking like maybe they did a long time ago. In the movie "The Note Book" when she was away Noah was lonely and had a woman as company, but that's all she was to him, he did not "love" this other woman. If you were together but not really together a partner can begin to wonder if love is still there. It could be that your husband was lonely and you were so busy with your children you missed it. Men often have a hard time reaching out and articulating their feelings.