I'd say it depends on how the place makes you feel. Surprisingly (or not so suprisingly after all?) there are people for whom hospitals are destabilizing. Sure, you get a safe place........ but it is sorta like when you take painkiller for broken bone - it feels tenfold worse when it wears off.

So depends on your previous experience and how safe you are. Your pdoc is professional but he does not live in your head (and he is protecting his *** and is on safe side). If you feel you cannot make it go. However, if you feel "just" depressed, maybe you could do it on your own - if you feel better and safer at home.