If it wakes you up, I would make an issue of it right then? If my husband so much as farts I wake him up too so he can suffer along with me

If you don't like the touch at the time, don't wait until the AM, make sure he is awake and aware and tell him "not now" or "not that".