I’d not go back alone. Have police wit you. File for divorce. Block him. He is either a cheater himself (those often throw accusations around) or suffers from paranoia or is on drugs and it triggers paranoia. Or he simply wants to keep you scared.

These poor kids. You’ll be ok. Not sure about them, they might be better off elsewhere, not with him