If your diagnosis was affecting the children, there would have been evidence of that already. You guys have been together for two years and they are developing fine. His ex sounds really spiteful for no reason?!

Even if you guys break up, she will probably find a reason to prolong this so he cannot see his children. What she did was not right and if you go to court you should bring up that point. There is no reason for her to go out of her way to find your roommate who may/may not have good feelings for you. Two years is a lot of time and there is the potential to change, which you seemed to have done.