Am I correct in assuming that you're comfortable with seeing him for therapy?
Also, when it comes to your past trauma, is it harder to discuss with him, or in general?
It might be what bluebicycle said, a coping mechanism. Therapy is helpful, but certain topics aren't always easy to discuss. When my T has me talk about more difficult situations, I can't look at her because I'm getting emotional. Depending on what it is I'll even cry.
Is the lack of eye contact something you two have discussed?

On a lighter note....I can't really make eye contact with my sister's fiancé even though they've been together almost nine years. I didn't like him at first so it was more out of disgust. As I'm getting used to him (I mean he isn't going anywhere, he is the father to my nephews), it's more of the fact that he's not too easy on the eyes. My sister and I obviously have different tastes when it comes to our significant others.