Starting a realtionship (even platonic) with a (at least at this point, though it would probably grow) facet engaging in substance usage (yes even booze) is a somewhat dysfunctional start IMO.

Not healthy especially if you have personal issues AND are already in a commited relationship. Unfortunately in some respects you have opened a door to this person and hopefully whatever foothold he has will be very minimal.

Be pleasent but firm if he contacts you that you are uncomfortable with spending time together because of your current relationship and hope that will suffice.

On another note the weed smoking and lack of effect is common but if you do smoke again prepare for that neural pathway to open up and then you WILL ghet the full effect and you may or may not be happy with the result.

Be very careful on both counts. Good luck