This is not low self-esteem. It is his playing a game with you. The game is skewed towards him, in that he gets to win either way. If the restaurant is good, he gets to enjoy it, and if it is bad, he gets to enjoy the gratification that comes from saying what he is saying (the bold part in your quote). Essentially, he is saying that he has prophetic powers (Oracle's powers) but is so modest that he does not offer his prophetic wisdom. Who needs such people?

Instead of downloading books on self-help, you should make a rule that either he participates in decision-making, or, he shuts up forever and does not comment on disappointments the way he has been doing, because it must be extremely annoying to you. Also, if he wants to criticize physiques, he should criticize his, but not yours. He can criticize his physique until cows come home, as long as he does not criticize yours.