And may I add; he wanted sleep and it seemed clear he wasn't gonna get any with you, so he went away as quickly as possible to go do that. Being very tired does make you act differently. IMHO.
Don't beat yourself up, just remember to be respectful next time. No means No, no matter who says it, man or woman.