Not sure what advice to give you. Maybe tell him what he can do so he doesn't seem to stand offish or tell him specifically what makes you feel that way so he can improve on it.