I can identify with not liking to be patted on the back, hugged, etc.

In my case, I didn't notice this problem until I was in my 30s. It seemed like family members were always hugging me, coworkers and friends were always patting me on the shoulder. I wonder if living alone mostly where the contact was less common caused me to develop this aversion.