As somebody who is into energy work, I am very hesistant of touch. Did not grow in huggie environment (and I am totally fine with that. Mom did energy work on me though. She still does, which is awesome. But we never overly cuddled when I was litt). I just feel touching somebody whom I don't have close bond with violates my aura and my personal space. I shield myself even from energetic intrusion...

maybe this may be the factor you like? The energetic/soul connection?