I think it can be harder if we are not use to being touched affectionately. It can feel like an invasion of space or like we are supposed to reciprocate in some way and we can't think how. I feel a little better if I can appreciate the lotion being put on my sunburn (or the hair stylist washing/massaging my hair/head) and look for a fond memory of the person touching me.