I have the same problem in yoga, on the rare moments when I am able to let go I immediately startle because I suddenly feel lost, exposed and panicky. But I think it is something I want to try and get through so I am keeping it up. Ultimately yoga and meditation should be beneficial but there are just additional barriers to work on which require particular techniques. For example when I am doing yoga relaxation on my back which is a particularly triggering position I keep a bolster pillow close by and when I feel exposed place it over my belly to ground myself, and sometimes curl up to it on my side.