I say that you should go. What do you have to lose? It is a social event that happens to be at a church. He's asking for YOUR company because he has taken a fancy to you. It is just incidental that the event happens to be at a church. So why not go? I don't see this as being the same as before....that is, you are not just being accommodating to this man. Think of it as stepping outside of your comfort zone in order to enjoy the company of another person. If you go and hate it, you can then tell him that it wasn't your cup of tea, but if you like him, then suggest a coffee date or something similar; very casual.

I see you as putting up walls to this guy when you could go out and have a fun time with him. I think it may be overly rigid of you to expect every single guy who is interested in you to know exactly what you want to do on a first date. Remember, he's not a mind reader.