Honestly I think most people would be apprehensive about vouching for people’s characters if they only’ve met few times and didn’t have variety of experiences. I think his reaction was fairly normal. I don’t know if he was using you for free meals though, that could be, but not wanting to sign a letter is very understandable.

Good luck with a new date. Don’t cook for him or have him over at your house. Nothing wrong with cooking for men but I’d give it few dates at least.