As far as I'm aware, a panic attack can be triggered by absolutely anything. Its possible you have a bad memory involving this smell/sight/feeling.

**EDIT: whoops, Is it possible you have a bad memory involving this smell/sight/feeling?