I don't think any support person can force you to do anything, they can only make suggestions. My knowledge is based more on what would be done with a kid, and in that case the "agenda" of the worker is not specified. But that's because kids don't often get the reason behind it.

It all depends on what you want to get out of it. Do you want to be more socially aware and more comfortable? Do you stay silent because you don't know what to say, or because you don't have the desire to converse? You have to be interested in other people to truly have a relationship. Everyone is different so if you have to identify what you hope to get from the arrangement. Once you know you should communicate that to them.

You mention you ride. Do you ever consider asking this person to go visit the horse with you. That's something that could open up a lot of conversation.