Having a flat affect or loss of emotional expression are DEFINTELY part of depression.

If you are not comfortable with your current p-doc, try looking for another one. The diagnosis may or may not change, but your comfort level might. However, since you have just started seeing him, you might give it some time as it takes awhile to form a working relationship.