I hope it's okay to add my input although you seem to have your mind straight on what is right for you.

My husband has been divorced from his ex many years and we've been married 23 years, from the get go I met her and her h and honestly I had no problem. However, I liked and trusted her - you don't appear to. Even to this day I think of her as a dear friend.

I just wanted to throw my experience into this because anything is possible and what works for some will not work for others but even when no children are involved (We have a child but they didn't) exes can have perfectly good friendships which do not threaten other relationships. It doesn't sound like it's the case for you and your husband's ex as you do feel threatened however.