Same thing happens to me. I struggle to fall asleep and when I do I have panic attacks in my sleep and wake up in the middle of them.

In the mornings I feel good, around noonish I feel OK, and about 5 or 6 my anxiety starts kicking in and by bedtime I'm a wreck. Don't know why