Smoke makes me feel sick for sure and I have asthma so it can also make me find it hard to breath which causes panic - in your case maybe the anxiety could be more about fearing a confrontation. I have had that dilemma myself of feeling uncomfortable about it but not wanting a confrontation - some smokers I have tried to reason with nicely have got aggressive in the past. It is very anxiety provoking.