Hard to believe, but I have agreed with a BF post.

I read most of the article--until I found it too tedious and flawed to continue (when it started talking about thermometers). Feedback is great and can take many forms and should be part of the give and take of a relationship. But what I'm seeing described is not what I would define as feedback, but rather as evaluation. Just as I resent vendors asking me to evaluate customer service (evaluation of a company's employees isn't my job--it should be the job of someone being paid by the company), a metrics driven approach to evaluate the effectiveness of a complex human endeavor is deceptive. The ability to break down a runner's stride into infinitely smaller parts is helpful because running is a discrete, repetitive, physical action. Therapy is an entirely different animal.

I liken it to "teacher assessments": the metrics will always be incomplete at best and completely wrong headed at worst. The system employing the evaluations will always abuse them either through a lack of comprehension of data interpretation and/or the purposes they are inappropriately used for (interestingly, there is more and more research coming out showing the failure of models of performance evaluation in education and the detriment of their influence on educational systems).

Feedback, in my case from students, is immensely valuable because it gives me insight into how my interventions are perceived. But such feedback tells me little about whether my interventions have been understood by an individual student (tests and other student performance tasks tell me far more), nor does it speak in any way to whether I am a "good" teacher or not. En masse, discrete results of discrete tasks can be helpful--for instance, if most students interpret a test question in a way I hadn't intended, it indicates a problem with the question--it doesn't indicate poor student performance. But an individual student evaluation is meaningless because it reflects the student far more than the teacher.

So what if an entire class answers the question, "The teacher begins class on-time" in the negative--surely that means the teacher is not punctual? Well, maybe yes, maybe no. Perhaps the teacher arrives on-time, but for pedagogical reasons, begins the class by circulating among the students addressing individual students briefly about non-class related matters? Just because the students perceive this activity as indicative of non-punctuality doesn't mean it isn't valuable (or that it is about punctuality). OTOH, a teacher who enters the room 10 minutes past the start time every meeting, clearly is not starting the class on time. But the metric doesn't distinguish intentions/motivations behind what appears to be a discrete behavior.

OK, so let's take the flaw of metrics out of it and look to human observation for assessment. Less algorithmic, but maybe a better match in complexity. Sounds good, but the problem with a teacher observation is that it can only assess what is visible. An observer can see what I do and hear what I say, but cannot see what I don't do, nor hear what I don't say. And in pedagogy, what is unseen and unsaid and the reasons for each can be of utmost importance. I would say the same is true of therapy. A report of a session cannot reflect such variables. Supervision, consisting of in-depth discussion of a session can come closer--but then, its value will depend upon the competence of the supervisor, so back to the same issue.

An example in the article is a client who regularly answers that the skills she's been taught are helpful, yet the metrics indicate she's suicidal. When the T presses further--and makes inquiries at a deeper level--the client reveals she feels worse. It was reported earlier in the account that the client repeatedly looked away when reporting that the skills were helpful. My question would be: why didn't the T notice this before being alerted to a problem by the metrics? While it may be valuable that the metrics alerted this T to delve more deeply, it seems to me the problem here is that the T's performance shows poor training/competence/professionalism to begin with. Missing such an obvious cue to emotional state indicates a T who's lacking. Will the metrics simply "bail out" sub-standard Ts, or be used as evaluative tools to get them further training? I see no reason to believe that metrics can ever evolve into practice to the level of complexity that would be necessary to really remedy inadequate practice. And it seems to shift the supervisory burden to the client in a deceptively facile way.