I'm sorry that this has been so difficult. I have been there, or been in that neighborhood. Apparently in the research on traumatic memories, olfactory memories are often the strongest links/triggers to traumatic experience and smells can trigger PTSD symptoms, which sounds like you are experiencing something like that.

It is good that you understand that your fear is linked to something specific, and that you understand cognitively that this smell does not mean that you are going to see the person that you associate with that smell. I knew a survivor from my group long ago who could remember what her perpetrator smelled like, but could not remember what he looked like. Smells were really tough on her and triggered memories all the time, and they weren't all that specific either.

Is this the first time you feel that you've been triggered as a response to this smell, or have their been other times? If it's something that has happened to you before, then you know that you have the skills to get through it. For me, anytime I have had a strong dissociative reaction to a trigger and I've understood what that trigger is about, what has helped me is to just dig into the traumatic content that it's associated with and try to unhook the trauma from the trigger. Bringing logic to bear really helps, and telling myself that the fact that this person reminds me of my perpetrator in some way does not make him into one, and that doing X is really an acceptable thing to do.

It does get better.