I have the exact same thing. I get so annoyed when people on the bus sit so wide (especially men do that!) and I make myself as small as possible so they don't touch me. It took me years to allow my friends to kiss/hug me for greeting. I would be the only idiot doing some akward handwave when saying bye. LOL.
Anyhow, I think for me it's because I never learned it and what it's about (affection mostly). I was neglected and abused (not so much physically though). I just feel like my space is invaded when people do it. I hope to someday learn it. I can't imagine T hugging me. If she'd ask I would probably say no. Hehe.
Anyhow, I'm sorry you have a hard time with this. Sounds like a perfect therapy subject and something to work on