I too would suggest thinking about what kind of need is going unmet by being with a man... The more specific you can get about what is "missing" the easier it is to fill that need.
Then... talk with your husband. I have heard many different "solutions" to this kind of need but they are so specific to the two people that are in the commited relationship.