No.

Is it out of fear of conflict or something else?

I'd day it's mostly fear of conflict, or maybe it'd be more accurate to say that it's mostly the potential negative consequences of creating conflict. Are you afraid it'll escalate into something worse? Or of losing the relationship? Or, as you mentioned later, of being seen as needy?

The reason is because either that friend may get mad and may even call me clingy or they may hang out with me because they feel obligated to. I've had stuff like that happen in the past where a friend would bail on me consistently to the point of being disrespectful. In some extreme cases, I may have said something a little bit but in most cases I would keep quiet because the times I did say something, the person would get mad and even call me clingy even though it was proven that they were just using me as a back up friend all along.

"Friend." You keep using that word. I do not think it means what you think it means.

Seriously, though. These people aren't your friends, and don't feel afraid to call them out on their ****** behavior. And if they're hanging out with you only because you've made them feel obligated to, you can call them out on that too. Just calmly sit them down at the bar, and say, "Look, Billy, you're a great guy and all, but I think we're on different wavelengths, you and I. Our interests simply don't align. You have your polka dances, and I have my hardcore gambling ring. I think we need to start seeing other peer groups."

I had a friend who was really into smoking pot, and I wasn't. Well, she kept wanting to hang out, so I suggested we play this new video game at my place because I was the only one with decent enough internet to play online. She didn't seem all that thrilled when I initially asked her, which should've been a sign, but later she agreed. Anyway, her and another friend came over, and we're playing games. While I was in the bathroom, I overheard them whispering, and he told her something along the lines of, "See? I told you it wouldn't be boring."

And it was weird to me, because I'm the sort of person who has no problem saying "no" if I don't want to do something. I'll even flat out tell you that whatever hobby you're trying to get me to do is boring. You do you, I'll do me. Any questions? So it was strange to me that she was so demure about the whole thing, that she never even gave me any verbal indication that she would find this boring. Sure, I probably should've picked up on her initial hesitations, but I'm sometimes dense when it comes to these sort of things.

I just don't like having plans constantly be cancelled just because a better offer came along. Doing that so many times can make someone feel used.

That's completely understandable. I'd be annoyed simply due to the fact that they wasted my time.

The problem with this is that they would still get extremely mad and defensive.

Why are they getting mad? They're the ones in the wrong.

Have you ever pulled away from someone rather than telling them what they're doing was bothering you? Do you think telling someone their behavior bothers you could accidentally make you look clingy?

Yes, and it depends on how you say it.

A question, though, what exactly is the "hang out" activity? Going to bars? D&D? Sky diving? Organized crime?