Teaching Evaluations Again

I have written before about teaching evaluations but have not done an in depth study of what they tell us. As a economist many years ago all of us teaching the basic course did use a common exam and compared student evaluations with how the students did and found no correlation. Some studies, however, do show some correlation between evaluations and learning. Lately an article by Deborah Merritt on the topic is close to reading a great book in terms being a page turner. "BIAS, THE BRAIN, AND STUDENT EVALUATIONS OF TEACHING," 82 St, John's L. Rev. 235. It's 2008 but I think it has been out for a few months. Here is an excerpt.

Yet, the research on student evaluations is troubling. It confirms not some connection between a professor's style and student evaluations, but an overwhelming link between those two factors. Nonverbal behaviors appear to matter much more than anything else in student ratings. Enthusiastic gestures and vocal tones can mask gobbledygook, smiles count more than sample exam questions, and impressions formed in thirty seconds accurately foretell end-of-semester evaluations. The strong connection between mere nonverbal behaviors and student evaluations creates a very narrow definition of good teaching. By relying on the current student evaluation system, law schools implicitly endorse an inflexible, largely stylistic, and homogeneous description of good teaching. Rather than encouraging faculty to use nonverbal behaviors to complement excellent classroom content, organization, and explanations, the present evaluation system largely eliminates the "dog" of substance, leaving only the "tail" of style to designate good teaching. Neither law students nor faculty benefit from such a narrow definition of good teaching." (notes deleted).

This article is chuck full of summaries of experiments relating to teaching evals. My favorites are those that show a few seconds of a teacher on tape with the sound off. A group of students is then asked to evaluate the teacher on a number of measures. These evaluations -- again based on sound off seconds -- turn out to be remarkable close to the evaluations the same teachers receive at the end of a semester from their regular classes. In short, looks, movement, expressions, etc, may trump everything else. Later on in the article Professor Merritt reports on a study that seem to indicate that whatever the students are responding to has virtually nothing to do with objective measures of learning.

This leads to two questions. How would a Moneylaw school evaluate teaching? If tenure, promotions and salaries are based on student evaluations would be fair to view the process as arbitrary and capricious?