I think this is the kind of thing that you have to talk to him about. I can tell you what I think is reasonable or what my therapist offers to me, but at the end of the day, this is something that only the two of you can work out together. I think that's part of what heals relational trauma, asking for what you need and negotiating getting your needs met with important others in respectful ways. That's true regardless of whether he says he can or can't do what you're asking.

For what it's worth, I would have been uncomfortable if my T had been mulling the question aloud in front of me too. Usually in that case she will say, "Let me think about that and get back to you." So hopefully if you decide to ask about weekend contact, your T will have had time since the initial conversation to decide what he is able to offer you.