perhaps you need to tell him exactly how you feel and explain that this attitude will not do. It's hard though I know, and we often prevent ourselves from saying what needs to be said as we worry about the outcome - what if he really doesn't care? Then you have a decision to make. What if he does care and is stuck in a rut? Then he has an opening to discuss this. I suggest making a time for you both to sit down and chat - ask him when he is willing to do this but insist that you do have a chat. Ask him to think about what is good in your relationship and what needs improving and you do the same.
Good luck