Ok first if your bf is showing the same patterns of behavior that alone should b enough to say this is a deal breaker but on top of that wtf isn't your bf standing up for you. if somone, whether family or friends are offensive in my home, they no longer would be welcome and it seems to me your bf has no backbone to stand up for you or agrees secretly with how his father sees black people.

silence speaks just as loud as him being racist himself. without hesitation I say you should end this relationship. NOT worth it. you deserve to be treated with respect. period.