is there anything from your childhood that could be related to this difficulty? (if so, working through it in therapy may be a great help).

have you tried desensitizing techniques? sensory issues is usually a big issue with those on the spectrum.