if you do decide to take your partner back perhaps do it in small steps, don't move back in together until your sure it's a long standing change. And maybe some form of couples councilling or mediation especially if things don't work out between you setting boundaries will be a great advantage, after all he is the father of your youngest and could be in your life for a long time. Good luck and keep safe