From what you are saying, it sounds to me like you are still wanting a relationship with him. Is he saying he is willing to resume the relationship if you ‘apologize and make things right’? But, from what you describe about him, he sounds really bad for you (and for anyone).

The things he gave you are just things. If you like them, keep them. If they are upsetting you because they remind you of him, get rid of them.

If you are calling him, but he won’t take your calls, it sounds like you are pursuing him and still wanting a relationship. So, it’s confusing to me that it isn’t necessarily over, but you are debating getting rid of reminders.