Certainly NOT!! PLEASE tell him you do NOT want to switch!! It's your right!!
I have had to move clients and I know it's painful. When I'm told I have a new responsibility, like a student, who only has limited availability, it goes against my choices as well.

But even where I work and in situations like this, if the client can NOT switch, then THEY have to figure it out! If it's a better day/time for you, maybe it's worth it??? Explore what they want you to switch to! If it's not, please refuse!