I say let him go. If he criticizes everything you do, this has nothing to do with you and it's all about him. You do deserve better. You deserve someone worthy of your attention, and someone who has nothing to hide.