Do not use bad language speaking of the other woman.

Do explain to him that you need to talk about the affair, and that you won't use bad language, but just need to talk. Tell him not to shut you up. He does not get to decide that talking about it takes you guys backward - they are two of you and when he says "us" for the two of you while not being congruent with your personal take on this (you personally do need to talk about it), he is overstepping personal boundaries. It is not ok to say "it takes us back" when in essence he is just not in the mood to talk about difficult subjects.

You won't be able to forgive him until he stops saying "this takes us backwards" when he really just means "I would rather not be bothered."

I agree with you - he must have had some feelings for her given the length of the affair, but still it does not justify your calling her names. So please explain that you just want to talk about it in neutral terms.