Gotta say, I experienced that same negative effect from adderall.
Switched to Concerta (which is also amphetamine)and for some reason problem is gone.
Can't say for sure if you will get similar results but worth a try...plus the Concerta seems to agree with me more anyway.