My humble suggestion:
You and your husband need couples therapy as soon as possible.
You and your husband need to talk this out with a trained, objective mediator.
I wish you and your family the best.