From what I have learned of the sexual brain we train our brain how to function sexually and to what, therefore, I think you may have unknowingly trained your mind/body to orgasm via the use of a vibrator "at your hand not his"..... which may mean that you will have to give up (for the moment) the use of your toy in order to retrain your brain to function sexually with a person.

What do you think about that? - it worked for me, and I can now use both.