I know it won't help your anxiety, but the truth is that if your friend dumps you for forgetting his birthday, he's not a good friend to you and I hope you won't feel too bad. It seems to me that he has serious problems with how he communicates and treats people and it is no surprise that he has no friends other than you. He thinks people failed him by not coming to his birthday, but he failed his friends by not inviting them and by demanding too much.

People forget my birthday all the time and I can't remember anyone's birthdays except for 2 of my oldest friends and a few relatives. Normal people don't expect most others to memorize and plan for their birthdays because it's childish and self-centered.

If you really like your friend you might just have to accept that he has some unusual personality flaws and shrug off his unreasonable anger so long as the friendship is making you happy in some way. If your friend makes you feel bad often or uncertain about whether he is still your friend, it may not be worth having someone that selfish taking up your time and emotions. It's possible that you are the only person with enough patience to be his friend and to forgive him for his bad social skills. Whatever happens, remind yourself that it is not your fault that he is angry. Maybe you should be the one questioning whether he is good enough to be your friend. Is he nice to you most of the time? Does he help you when you need it? Does he remember your birthday without being reminded? Does he do or say things to hurt your feelings? Why do you want to stay friends?

Good luck! I hope your ok whichever way it goes.