Find your anger (instead of anxiety/worry that keeps you from acting on your own and your children's behalf) that he is behaving this way. Even if he has a problem or an "addiction", he is a grown man and responsible for his actions. Go to a lawyer and discuss the situation with her. Two can play the daytime tryst game, only you are playing for yourself and healthy children.