I use to have the opposite, a smell would be significant and it took a long time before I realized it must have been a scent my mother wore (she died when I was 3 and I don't consciously remember her). Is it a particular memory? I'd think about who else was involved and/or try to find a candle that smells like that (to get the name of the scent), etc. I'd think back on all my summer memories and factual stuff I knew to try to imagine where/location it could be coming from.