Well, even an "in house" date or a date where you meet at a bar is still a type of date. But it is good to do other things as a couple as well, and not get in a rut.
Okay, so it is good he interacts with you and wants you to play with him. That's good. If he was just sitting there playing while you float in the background, that's a bad sign. It's also good he is concerned and wants you to get better.
Things you say bring up red flags. When you talk to the T make sure you tell them you sometimes feel like everyone hates you and you push them away.
Also suggest to him to do other activities with you, and you alone. Going in a group is fun but you need alone time to build intimacy (emotionally). Doing things together, actual things even like seeing a movie or playing miniature golf or whatever, that builds memories and bonding time. It is important.
Update when you've talked to him.
Even if you break up everything is going to be okay. But I think he's overwhelmed at the moment and that can cause a shutdown.