Does he have a busy social life outside of your visits? It sounds like he has been detaching from the family for a long time. Maybe he's just a loner?

It's nice of you all to care so much about him despite his lack of presence when you were younger.