I'm imagining these conversations are happening in the context of your aunt or someone asking 'so what have you all been up to?'

If you have hobbies and your brother doesn't, then your family has more to talk about with regard to you would be one way of looking at it.

I would try to think of what interesting things your brother DOES have in his life and bring that up the next time. It would probably make him feel better if you normally get all the attention.