Your concern for your son is understandable, and your personal issue with the situation is too. Alienating B is going to cause pain, for B, your son and even J, it is an option sure, and so is telling B the truth about his “father” but who knows what madness that would bring at this already tough time.
Sometimes when things are uncomfortable, and “miserable” as you described it one just has to deal with it, to alleviate the negative feelings isn’t always the solution, sometimes you just have to exist there for the sake of the people you love.
I don’t blame you for trying to ‘improve’ the situation, I’m a parent too, I know that desire well, i hope they can all focus on the love they have for each other and get through it as a family.