Well, you can correct what you stated earlier about come anytime and my door is always open with him. When you see him you can tell him that given he has a reputation of being a nice guy and you value that, that you want him to know that you did not literally mean what you said to him. You can let him know that what you actually prefer is privacy and given he is a nice person you feel he will respect that. You can tell him that you are actually more of a private person and did not want to give him the wrong impression of yourself.