I would leave him even though its hard if he's stressing you out causing you to feel like a jailor and that's something you don't want to do then you should quit him because with all that stress and everything going on with the situation you might develop a serious health problem
Hope evverything gets better