He's already controlling you, nobody wants to be controlled and it will just escalate if you stay much longer. Would he be willing to go to some therapy, if you don't want to leave him? it only gets worse, especailly if he's messing with your self esteem.