I think you need to have an open and honest chat with him about how much his lack of concern upsets you.