How well do you know your therapist? Are you attracted to him at all? My male therapist is conservative (religiously), so as a courtesy to him, I think about what I wear when I see him, but I don't worry nearly as much when I visit my p.doc.