I've always taken this to be a sign of a poor manager, who shies away from dealing with conflict. Their subsequent lack of action, and a tendency to bury their head in the sand and hope it will resolve itself, results in all parties being left unhappy. Sometimes managers have to take unpopular decisions, and someone has to wind up being unhappy. It's the job.

That said, at least he has spoken to you, and listened (hopefully).