Gosh Poppy,

You are a renter, right? Does your landlord know about this other person on the premises? I assume "yes", but if not, are you going to tell them that you took in an undesirable onto their property?

Are you saying that you plan to move and just leave him to figure it out that he is not moving with you, but will have to move somewhere? If he is not on the lease and has no right to be there, the owner will be the one to go the the drama of eviction, not you.

I have had a similar experience and you are right, the laws in CA favor the criminals, (the squatter in this case), not the rightful tenant/owner. Yes, it is a screwy deal to be sure.
I actually did the eviction process and offered to help our squatter on his way, but he refused. And So ... the sheriff came and carted him out bodily. He didn't have time to grab a bag let alone anything else.

In your case, this guy needs a heads up about what is happening and your fear will be helped with numbers. Call the owner, call everyone you know who would be willing to be around, bring lunch, whatever it takes to make you feel safe. I like the previous ideas about church people if you know any, college kids, etc.
If the owner could get him moved out first, that would be the best, but saving that, the guy will likely be there, so get busy arranging for reinforcements.
All the best to you and I hope the new place gives you the peace you seek.