Hey,
I think for me it's mainly sadness and upset that I leave with because I didn't want it to be over. There's many reasons I leave Therapy feeling bad - Maybe something I sad, something I didn;t say, not using the time productively, something my Therapist has said, feeling like my therapist doesn't care or understand, annoyed at the whole "professional" relationship and maybe just annoyed at Therapy in general.
Like you the panic often comes later when I just want to have my session again and I dont want to wait a week for the next session!
Maybe after a while you will figure out why you feel angry. Sometimes t can just be hormones or something I think.