As long as you wean from them under a doctor's supervision, then nobody can really argue.

What does your family think of the idea? As you might have read in the "off my meds" thread, your family members might have a perspective on whether it's a good idea for you or not.