Hello and welcome to PC! I'm glad to see you have found your way around PC.
I don't think that this relationship with his other daughters grandmother is all that awful, though definitely outside of the norm.
Despite his relationship with that woman, he is still the father of your child and has done nothing wrong to her or you so I believe you should let him see her if he chooses to.