Hi, beaneroconnor, welcome to PsychCentral (PC). I think if you do not feel comfortable, are worried, then it could be a problem. Are you three living with him or can you get help with telling him he must see someone or he can't be alone with the kids, or what is your situation? If you don't feel safe, then he should be trying to reassure you, if he cares for you and the twins?