Good suggestions...

One thing that did occur to me was perhaps your husband's actions/feelings are not from your situation...
There might be something else he is worried about..work, money, well just about anything.
He may not be saying anyway so he does not upset you.

If it is something else, plus his worry about you, then he is indeed going to have some kind of reaction.
Maybe a good, honest talk would be beneficial for both of you...

Best wishes,

Catherine