Well, first I would suggest backing off the pot. Pot is not conducive to an erection. Here's an article from a rather biased site

Weed Wiener: Smoking Pot May Cause ED | High Times

There are plenty of other resources that support this idea, I just wanted to link you to a site that you would think would deny that connection. Drinking is another one. There's a reason they call it "drunk ****".

Second I think you should talk to a counselor. Found out where this fear is coming from and then develop some coping mechanisms.

So backing off those two substances and getting some coping mechanisms may help.