you may find you don't remember playing indoors much because something you have burried deeply inside you happened when you were playing inside and angry one is the only one who remembers/still fears it. it scares him so much or he doesn't want to scare the resy of you that he can not explain why playing is a definite no, it just is. maybe let him come out and talk to your T where she can ask him about it, he may or may not feel ok telling her but it's worth a try.
it could also be as simple as you never were allowed to play much indoors or simply that angry one is a grown up and believes playing is for kids not grown ups, he may be scared it will take him back to childhood and bad memories or even that by playing he is giving permission to be dominated like a parent does with a child... who knows, but a few things to think about.