Does the term "cognitive distortions" ring a bell for you? You are making assumptions (or the depression is doing that for you) that have no basis. (There's a list of them at the top of the psychotherapy forum, and how to counter them.)

Do you have medication the doctor thinks you're taking? You have a right not to, but you need to let the doctor know that you aren't if so.

Why not try the medication if you have it, and see if that helps limit the worrying you are doing.