Don't feel stupid for having something to enjoy in life like this! Move on from those who don't understand, maybe they can't find their own simple interests.
Being made fun of isn't fun for anyone really... those who do this are just trying to take the focus away from themselves, imo.
It's a shame that some people seem to live for causing others distress, eh?
Now have fun!