I would not take it so seriously if it was one time incident. I would actually find it a bit funny. I have sometimes lost the track of time myself - also at work - so I understand how it is possible.