If anyone acts annoyed or criticizes these sentiments, it's because they are jealous that you can admit it. They get even more jealous if you actually act on such a thing, and go to Brazil and have adventures instead of stay at your tedious job.

What I did in my spare time when I was unemployed (aside from looking for a job):

-Helped my mother start her own business from her passion (she hates that it's a business but needs money)
-Made a lot of art
-Started getting involved with a co-op to open a local store
-Cleanup up beaches with a group of volunteers
-Redecorated the house
-Baked lots of treats; learned some advanced baking techniques
-Made home-made meals every night
-Paid attention to family and friend's birthdays/events
-Planted a garden/cleared invasive species

What I do in my spare time after I got a job:

-watch TV
-eat junk food/take out
-play dumb internet games
-garden died

Yes, clearly working offers the most fulfillment. I don't mind my job though. It's a good job. I do get the pang that I'm wasting my abilities, or trying to force abilities that I don't have to "get ahead" though.