I would take it to mean that my outfit looks fancy but I can see how others take it too. I would definitely ask them to clarify what they meant and let the t know it made you uncomfortable.