I do this too. My partner will say something to me, and apparently for a flicker of a moment I will look annoyed.