Seeing your dad and what the booze and pot have done to him, you're afraid that your boyfriend might end up the same way. It doesn't matter that he's stopped smoking pot. It touches a raw nerve in you.
There's also the fact that maybe deep down, you don't trust your boyfriend to stay off the pot. It's understandable, to my, anyway, because of your experiences with your dad.
Try to work through your fear/concern with your boyfriend. It already sounds as if you have good communication between you. This might be more about you and your feelings that it is the possibility of your boyfriend going back to the pot.
Good luck!
__________________
Psalm 119:105 Thy word is a lamp unto my feet, and a light unto my path.