It's not your job to protect his privacy. Clients could follow him home if they wanted, do you need to warn him about that too? I would think that the deeper issue for you to consider is where your need to protect and guard him is coming from and how that resonates in your life outside therapy.