You've got too much going on to keep your thinking straight. Getting involved with this co-worker was an unfortunate direction to go in. You need to be in one relationship at a time. Doing otherwise is double dealing and a legitimate reason to feel guilty.

This co-worker's expression of "having feelings" for you was really not very flattering of you. His wife is dumping him (or he's dumping her) and he would rather not be without access to a woman. So he glances around, sees you, knows you are not in a satisfying relationship and thinks, "Hey, why not? I could give her a shot." Feelings? Sure. A young guy whose relationship has gone down the tubes is easily capable of "feelings" for probably any likeable, halfways attractive young woman who happens to be around. This term "feelings" is so custom made to mean anything and everything one might like it to mean. It's supposed to be so innocent and pure, as in: "There I was, minding my own business, having no designs on this person that circumstances put me near, and without me intending anything to happen, I discovered "feelings" growing inside of me. Don't I have a duty to acknowledge them? Wouldn't it be phoney not to?" So the individual confides these "feelings," so shyly, as in "I hope you won't laugh at me. I'm not someone who goes around hitting on women, but it's just that you're so special." It's never about mere "lust." These "feelings" are something so tender and all about caring with the deepest respect and admiration. So, of course, you have to be touched in your heart. Wonder how this guy's marriage ever fell apart?

At this point, you and the co-worker are probably too much of an item for you to back up from the involvement. So you've gotten sort of committed, while you have a marital commitment that you never concluded that affects the welfare of two small children. So, yeah, I'ld be feeling guilty too.

I'm not trying to moralize or get down on you. I just think you've got too much going on. It's unmanageable, to my mind.