Well... I can't say if you should be worried. My thinking on this, however, would be you should have it looked at by a physician or physician's assistant. Perhaps it's simply ear wax build-up. Or perhaps there's something else going on. If what you've been experiencing has been going on for years, it doesn't seem as though it would likely be too serious. But the only way to know for sure is to get it checked out. I myself have Meniere's Disease & tinnitus. So I've had some personal experience with ear problems. Best wishes...