Could you maybe make a schedule together where he can play on certain days or certain times? (when you're not around)

like say--
Monday from 6pm-8pm he can play as you will be at the library
and then Tuesday from 5pm-6:30pm as you will go and do your
weekly shopping at that time, and then perhaps there is a friend's house he could go to once or twice a week to practice and they could play there. And then say for example-- Saturday from 11:00am to 2pm he can drum at home as you will be out with family or friends or giving yourself special time to go to the zoo, museums, the park or whatever you fancy.

maybe something like this could work? you think?

fins