I don't know the specifics of course but is there some way to still do a little of the job you want to do? When I was in school I loved music class and whenever there was an opportunity to sign up for more of it I did. We didn't have band in elementary school but my friend and I were allowed to leave class for trumpet lessons. We had fun and because it was just the two of us and the music teacher basically giving us private lessons we were the best in our middle school band.

In middle school we had a separate jazz band practice one day a week during recess. The music teacher who taught me how to play the trumpet had a huge impact on my life. It sounds like you are pretty close to being in a place where you could do the same thing?