I don't think you're being unforgiving, but you're a better judge of the situation than me since you're in it.

How you feel is important too. There are a lot of demands on you in a situation like this. It forces you to try to navigate some sort of emotional minefield another person laid out before you. An apology doesn't just fix everything or take away that this can be emotionally draining. It's a nice first step, but I'd still feel a bit weird after this for a while. I've had a similar experience in the past where people just continue to try to take.

You seem to be pretty hard on yourself. You don't need to try to please everyone in this scenario. Be true to how you're feeling. That's really tough for me to do so easier said than done. However, it's an honest approach and I think makes it easier on everyone involved maybe? So if you're feeling overwhelmed and you need some time away from the situation that's fine. Or if it made you feel uncomfortable and you didn't want to hurt his feelings and now aren't really sure how to react that's fine too. Take your time. His apology doesn't demand an immediate switch in your feelings/behavior.

It may not directly apply to your situation but it sounds somewhat similar to areas where I have trouble. I get pretty caught up in trying to be or feel a certain way for other people at times, and I'm trying to work on this. For people to be my friend they need to respect what I'm going through as well and they can't do that if I'm constantly trying to manipulate those feelings rather than expressing them as they are.

best of luck