If he is a superior of yours (and not someone in a different department or something), definitely give it a pass. That breeds resentment among co-workers and can go south if things break up.