I don't think you did anything wrong. This is on him, and he should have owned up to it without attempting to insinuate that you are partially to blame.