You mentioned: what else was I going to do, go live with my parents and be socially shunned? Not to mention that everything that I do with our daughter is seemingly insignificant because he's typically the one that gets up with her in the morning, even though I have her for a majority of the day because of his work schedule. Oh, and because I have attachment/abandonment issues, I'm an insufficent mother because I'm not properly attached to her.
I'm not sure on all the details, but I had a similar exp while I was married. I became emotionally attached to my painting teacher, and did kiss him. That was all. Actually the painting teacher detached from me, abruptly, because I could not go thru with the sexual thing. I was deeply hurt by this experience, and told my husband what had transpired. I wanted a divorce, and he told me, flat out, that if I tried to divorce him, I'd be "out on the street with nothing," and he'd keep our daughter. I wasn't working at the time, though I tried to find employment. My art background wasn't in demand, LOL. Anyway...I stayed in the marriage for 20 years, till my daughter left for college. He actually held the issue of custody over my head for the entire marriage, since she was a toddler when it began. I believed him.

It sounds as if your husband has major issues with trust. I'd have a hard time dealing with this. If you love your husband and want to work on it with him (short of taking a truth serum), I say give it more time. If, like me, you are being intimidated and bullied (as I was), start thinking about alternative choices. Keep us posted on how you are doing.
Patty