First: What your pdoc says is ONLY his/her opinion. It's not an order, and it doesn't need to be thought of as such. I have yet to let a single pdoc actually manage what I take, how much, or when. After experience with so many of them, there's just no trust. I only go to them to get a med I want and their personal opinion on something I'm planning on trying.
What I would recommend for you is to come off of Abilify, but at a slow rate so that you can measure your reaction to the lack of it. If you start feeling really down, or some previous conditions/symptoms reappear, then you know it's not time yet.
Also, please don't worry about the possibility of death via Abilify. It's not realistic enough to even waste time thinking about.
Best of luck!