Sure... you say, "Actually I don't want to spend my time today talking about that." If you want to you can expand upon that and say something like "It's making me nervous that my time is ticking away while we are discussing something that isn't really relevant to here. I would much rather talk about _____."

You can use non-direct communication if you want to (the 'I don't know' stuff, and looking away etc) or you can simply say what you need and be direct. I guess one inherent danger in using non-direct communication is that it can be interpreted incorrectly. For instance, your T may have wondered if your body language etc meant there was something about your friend's visit that distressed you or you were avoiding.
On the other hand, if you laugh and say something like "let's move on, there are much more important things to talk about!" it is much easier for a T to figure out what you feel, want and need.