Your pdoc will probably advise against it, but you and he can have an intelligent conversation. You can present your reasons and he can give you info on how these meds work as you go down on them. Ultimately, you're right, it is your body and you should have a say. If the both of you agree to work together, then your pdoc can give instructions for going off meds.

Right now it sounds like you have your mind made up, so it would be best to talk to pdoc to see what the best course of action is.