Until that bit, everything was in good humor.

Your faux pas was your remaining neutral (I do not blame you as you were in a difficult situation, and I am not saying that I would not have done the same thing - I am just offering an objective opinion of an unemotional outsider).

You should have either:

- stop communicating because the twist of being in this kind of a situation that is riddled with possibilities of a conflict, gossip, "he said she said", etc. is never worth it

- communicated (assertively) that you do not entertain complaints about your dad.

Sometimes, remaining neutral is the right thing to do, but it was not that kind of a situation. You should have called him on what he was doing.
Having been unable to do handle that difficult challenge (again, I do not blame you at all, as it was objectively difficult even for a much older person) appropriately, you reached a point of no return. Now there is nothing to salvage, so you might as well forget about this guy.

In general, I agree with Rose in that you should not date guys who talk bad about your family; wholeheartedly agree. However, if someone talks bad about your family but you respond back instantly with an assertive message that calls the person on his talking bad about your family, and the person who talked bad about your family "gets it" and backs off with a "sorry, point taken", you might be able to go on. But you were not able to do that - you were neutral, which demonstrated your weakness. So - done deal and nothing can be done to fix it.