My dear...there is nothing wrong with you or your reactions. In fact, you're taking this lying down and not even giving him the reaction(s) that he has earned from you. You have no reason to trust him and every reason to doubt him, so it's perfectly normal for you to panic and be suspicious about what he's up to when you're gone. His track record gives you no reason to think anything else.
If your husband is not willing to work for you and own what he has done, then there's nothing at all that you can do to make it work. It would be very unhealthy for you to be trying to court or win him in any way after what you have been through. It needs to be the other way around.
Go here:
SurvivingInfidelity.com - Support Forums for Infidelity Recovery and Healing