Yes, you were violated.

If I were you, I would do a couple of things.
Go to your doctor and get a drug test for known sedating drugs, or for so-called "date rape" drugs.
Also, if you are reluctant to have the police investigate it, you can phone and state you want to report your experience with this man so they have a record of it.

If he's done it once, there is a high chance he's done it before, or will do it again in the future. That way, if someone else reports him for drugging and sexual assault, they will see he has been reported before. The next person may take more than a few sips, and be unconscious while he sexually assaults them.

Third, next time you see him, ask him what he put in your drink. He may deny putting anything in there, but he'll know you're onto him.