Can you talk to prescriber re: way to ease the stim med drop? (a low dose afternoon before end of day...) Just to see if there is a connection.
Typically, panic attacks do occur without warning or notable precipitant. We may make connections over time ("it is when i drive") to make sense of it---and some people limit their activities to avoid the feeling---I am not aware of this being very effective. It is all about flight or fight kicking in out of place.