How close are you when they start doing this? My first thought is you're getting too close and invading their personal space.