IDK, there is an old saying "the best offense is a defense" Is there any possibility that he is the one with the drug problem? What kind of drug does he think you are abusing? Seems like there is much to work out if this relationship is going to continue. Maybe the help of a counselor? Good luck and big hug!