You know what? It's ok to be happiest at home.

It's ok to be happy spending your time alone together.
I'm sorry that work situations are difficult though.

That makes for stress! I'm sure that together, you and your H (and maybe some therapy!) could find a way to help make things more bareable for you both.
You're totally not crazy!
It's really hard work. Why not pick out something together that you and your H would both really like to do - but are too shy/nervous to go and do it. Then work your way backwards and break things down into smaller things that would help build towards that one thing becoming more comfortable for you both?