I'll jump in even though I'm not sure what to say.

You said this is your family doctor and not a licensed psychiatrist? I would be wary of telling him. Frankly, he doesn't have the training to deal with your feelings.