You're still unstable and he's talking about going again and soon. You need to talk to him and say these are the reasons I can't go, and these are the reasons you need to think about going (both positive and negative). Is there a possibility to jump on a plane for a week or two if you stay home?