Sounds to me like your coworkers maybe overreacting to your foibles (quirky habits) everyone has or does something a little different. I'm a singer I usually have odd little songs stuck in my head; currently it's cheer up Charlie from the original Willy Wonka film

if you were full on " goo goo ga ga I wuv yooouu" I think it would be right of them to mention it but it honestly doesn't sound like a big deal.
Some people are a bit over sensitive as to what you should act like when you're a "grown up" I am happy to say I disappoint them I am a big kid and proud of it !!!!