Do your coworkers ask you this every single day? If so, I can understand how it could be annoying. I do agree with Mr/Miss Trippin2.0, in that you may be misinterpreting what is meant. To be sure, you could simply ask them if your routine bothers them-that seems like the most logical solution.