Is there any way you can get a friend -- a REAL friend -- to be there when you kick him out? I'm not sure what your mental health situation is, but if a verbal argument is triggering for you -- and it might be, even if it's just through the door when he finds his stuff outside -- it might be better for you to have a friendly face there to make sure you're okay. Plus, there is strength in numbers, right?
Good luck. You're doing the right thing by kicking him out. He needs to go!! Good luck. You've got lots of support here.