I agree with excelsior- never stay in a relationship just because you don't want to be alone. It definitely sounds like your privacy is not a big deal to this guy (which is a pretty good indicator of how much he respects you)... If you really are worried about privacy, you could ask for your key back in public, and tell your apartment complex you want your lock changed for personal reasons AND because you want one that can't be broken with a credit card!! That's sketchy on multiple levels. If you're worried about a bad reaction from him, you could stay with a friend until your lock is changed, if he refuses to give back your key for some awful reason.