You are not being unreasonable but I think you are giving too much "blame" to his previous partner. All you have is what he says, who he blames but it sounds to me like his problems are his own (which may have made her appear to be the problem; think of how he could think of you wanting to be intimate with him now, you could seem "pushy" to him which would make him a dildo with legs?).

It does not sound likely to me that if he was enjoying you before that suddenly he would not and that it would be a result of a previous relationship. He does not sound like someone I would want to be in an intimate relationship with; you did not mention anything he is doing to work on this problem. . . which is all his at this point! He needs to fix the problem, not the blame.

I don't like that he told you so much about his other partner and that relationship; it is none of your business and his bad mouthing another partner he chose before you; he could do the same about you if your relationship with him fails!