Lack of eye contact is a coping mechanism for me, and it sounds like it is for you too.
When I don't make eye contact with someone, it usually means I'm very nervous or upset. I just can't bear to look at someone in the eyes when I'm in an abnormal state of mind. Eye contact just makes everything 100 times worse. So, I very much understand the eye contact thing.
I can't offer any advice, but I hope you're able to work out the eye contact problem with him.