What makes you think there was an invitation to go back stage from him, when he did not verbally express this to you? Did he gesture for you to go back stage? Did he look directly at you while smiling and gesturing for you to go back stage? And what makes you think that he did what he did simply because in your mind, you were rejecting him?

To me, this is a made up story in your head. We can only speculate on what he did, without truly ever knowing what he was thinking.

It's very logical to believe, however, that perhaps he assumed you were a groupie or made certain assumptions about you given how you were up front, that you were alone, and that you had approached the band before they started playing.