It’s difficult for sure, you have your reasons for keeping quiet which you’ve explained here, but he feels like you’re stonewalling him. I’m assuming that you’re still working together.

Would it be appropriate to write a short note, explaining why you have been quiet? Not apportioning blame but explaining how you feel?