You speak as if there's a grand medical chart of all your medical records. In short, there's not. Every Doctor you've ever been to has a "chart" for you, and they don't need your permission to put down a diagnosis. However, that diagnosis goes no further than THAT doctor and your insurance company.

HIPPA......be wary! It only covers Doctor to Doctor interaction. If someone impersonates you to get your records, it falls outside of HIPPA privacy laws. (I've seen it happen before.)