I read your post and admire you for fighting for your family. That is very admirable.
It will take time for you to trust your husband again, and it is up to him to provide you with reasons that show you can trust him again. And you have noticed that he is trying.
As far as you not being able to get that thought out of your head, he most likely said that in anger to hurt your feelings, and it did. But if he is proving that he does love you, and is sorry for all this, try to look at the whole instead of one small piece. Judge him by actions, not some words he said when you weren't getting along.