I bf all 5 of my kids. And yes, this same thing would happen to me. I had "leakage" for a few years even after I stopped. My breasts would hurt, and one worse than the other sometimes. My left one was always the sore one.
I went to the dr, and was told it was normal and I was fine. Personally, though, I would go get checked, and just let them see what's going on. You don't want to take any chances with your health.
You may find out as I did that it's normal for you to do this, and it will stop in time. But at least you've been checked and you are sure.
Let us know what happens!!