I guess it depends on how you worded it, inflection, etc. My husband hates being "ordered" to do something. I remember when he got angry when my arms were full of grocery bags and I was frantic because they were heavy and slipping and I called to him ahead of me, "Open the door!" That was a command so he opened the door but not any faster and started lecturing me about how he doesn't like being ordered to do something :-) It was so odd-seeming under the circumstances that I was mostly curious and we had a really good discussion about it and now I'm more careful how I word things. I was accustomed to my stepmother and her commands and I suspect learned that way of speaking. With her she was supposed to be able to call "Jump!" and our response was supposed to be "How high?" :-) Instant obedience in other words.

So, now you know that "why didn't you. . ." may sound challenging to him (sounds accusatory to me as it was another of my stepmother's favorite traps). Might want to keep with the "I" statements, "I noticed that you didn't say anything and I was curious why not; did you think it was the correct time like I did?"