I think you an your ex seem to have a good connection, and your husband seems like an ***, plus you did not marry him for the right reasosn and it will only get worse.

leave your husband if that is what you want, but i know how hard it will be especially considering your son will still have to see him, but he seems like he is really good with him. Because your daughter is not his then she wont have to have nothing to do with him, which will do wodners for her self esteem. Remember you are a mother first and for most you protect your kids!