It does sound like you're addicted to Adderall, and not just physically dependent, but emotionally addicted as well. It also sounds like you're well aware of this and know what you need to do.
You need to stop taking this med. How's your relationship with your psychiatrist? Is he/she someone you can come clean with? That would probably be best, then they could taper you off and treat any other issues that pop up as you come off of it.
Some people may disagree with this, but if you feel like you can't be honest with your psychiatrist, I would slowly taper myself off. You can safely come off of stimulants pretty quickly. The only side effects are transient sleepiness and mild depression. And they are transient. You just have to give it time. I would decrease from whatever dose you're on, it sounds like in 30 mg increments, by 30 mg every 3 days and then stop once you've been on 30 mg for 3 days. At least give yourself a couple days not taking it to see if you rebound after a couple days. I know you say you can't do anything if you don't take it, but you can, it's just going to be really hard at first, but it will get easier. You once lived without these meds. You can live without them once again. Good luck.