You have to decide what is more important going to the clinic or your job. From the sound of things, you value the time at the clinic to be more important - which sounds entirely sensible from what you have said.
You can't control what your boss does when you tell them. It's out of your hands - you can only change what you do and say. Maybe you could suggest a replacement to help....
If you do lose your job you will find a way to cope, and you will have all the benefits of the clinic behind you.