You had problems with those around you growing up smoking pot. You felt something which attached itself to "anyone" close to you smoking pot. Probably why you smoke cigs instead of pot.

I have a friend who got a DUI but hasn't drunk for several years but her teenage daughter is still "hyper" and against all drinking at any time by anyone, etc. However, you can hear the anxiety in her "speeches" and realize it is because of her past fears when her mother was drinking.

If you want to get over it, I think the only way to do that is to be present when your boyfriend smokes once in a blue moon and work with yourself/anxieties/anger that arise at that time. But why do you care that it bugs you? Intellectually you know it "doesn't matter" so I would go with that. We're always true to ourselves, even when we deny and/or suppress stuff :-) so I would just put up with the feeling bugged and drop any sense of wanting things symmetrical with any pride that you're not a hypocrite or something.