I agree with you. Travelling with someone else always involves compromises. You seem to know 100% how you would like to spend the day. That does not leave time for what a friend would want to do. (Unless, the friend is very accommodating or they share the same interests as you.) Though, it is a sign, perhaps, that you need to make friends who share your interests.

Have a great day! I hope it turns out exactly as you want it to!!