No, it was correct of you to go. It's impossible for you to know if it is a benign thing like a hemorrhoid or a skin tag from a fissure or something that needs more extensive treatment (which is not all just cancer).

When in doubt it is NOT wrong to see the doctor. I have a friend whom I can't for my life convince to see a doctor for a growth on his body, which could be a cancer but he shrugs it off and says it is "nothing". It might of course be nothing but isn't it better to know?

So I think you did the right thing. I hope it didn't cost you a fortune to go.