I'd definitely encourage you to talk about it with him. Avoidance works for awhile (I should know, as the queen of avoidance!), but it has a nasty habit of sneaking back up and walloping you later.

My first therapist handled our ending really badly. I couldn't bring myself to tell her how difficult it was for me to leave (even though I wanted to in some ways, because I knew we weren't a good fit). She ended our last session 20 minutes early. I was too anxious to say anything, and just sat there, kind of mute. She finally said, "Well, if there's nothing else to talk about, maybe we should just end." It seems like your therapist is offering you a gift, in a roundabout way, of honoring the work you've done together by wanting to take time to say goodbye.