Perhaps you can try not to judge something as frivolous or not. Ultimately, if it helps you progress - whatever the activity is - it warrants not being labeled.

And finding something that you genuinely enjoy would likely include some trial and error. Sometimes we need to get involved in things that we think we might like before we can decide if it's really something we want to pursue in greater depth...and you never know what could come from the experience.