He causes the buzzkill with his comments.

How is your mood after he makes one of those comments?

For you to bring it up isn't causing the buzzkill.

Rather, it is acknowledging and drawing attention to the fact that he has caused one.

It has already happened, and saying that you want to talk about that buzzkill is completely reasonable.