Agree too! When I was young and spending time with my divorced father, it drove me crazy that I never got to spend any time with him alone. These kind of moments happen naturally when you are together all the time, but when you only have weekends or visits, it is harder to find them. Your son needs to feel sometimes that he is the most important one, and that you are willing to make an effort to make sure he is happy, not just that he has to slot into the life that you and your husband live normally. This is not normal for him. It is not fair to expect him to do all the adjusting.
To say he is "jealous" of your husband I think is misunderstanding the problem. He just misses his mom, and the good times you used to have. A daughter would feel the same "jealousy" so it has nothing to do with "another man" and everything to do with the loneliness of a child. Please, please don't start down this train of thought, pitting a child against an adult in a "competition" for love. It is destructive to all involved.
It also concerns me that you want to protect your husband's feelings by not openly discussing the situation here - you and your husband are both adults, you son is just a child. It doesn't matter what your husband thinks of your son, your husband needs to help him feel comfortable and happy - that is part of the adult responsibility of being a step-parent. If he can't do that you really should make sure you spend time with your son alone. Your son needs to know his mom loves him at least.
I am really sorry if this sounds a bit harsh, but I have personally lived through many situations similar to this in various roles, and it seems it always the children who come out worse. Make time for your boy, talk to him. And listen. He doesn't want to lose you. Good luck Zellbeelee!