Next time it happens, try backing up just a little and see if they relax.

I remember once talking with a man who needed a big personal space bubble. I couldn't hear him well and when I tried to get closer (like 3 feet) he would back up and act uncomfortable. So I spent the conversation straining to hear him and kept my distance.