Depends on what his role is with the band, then. If he's just a "cling-on" and you know that the band members have a sense of decency, maybe the leader needs to know.

It's easier said than done, and I don't know, really, what I would do if I was in your situation, but I don't see why you have to stay away from the band's performances as long as you're never alone with this guy. But that will come in time. Do what makes you feel safe.