Plus/minus and the problem of measuring Arschlochkeit
I misspoke when I asserted recently that MoneyLaw strongly prefers baseball and football to hockey. It turns out that hockey maintains a statistic that may hold the key to an ongoing exchange between this forum and Concurring Opinions over the impact of tenure on the intellectual and social behavior of professors.
After Dave Hoffman's initial post asserting the absence of a "deadwood" effect, my response and Dave's rejoinder appear to have swung the focus (at least for now) from the impact of tenure on productivity to the problem of patrolling against Arschlochkeit. In commentary on my post, Ani Onomous rightly observes that personnel decisions based on perceived Arschlochkeit have the potential to negate MoneyLaw principles. It therefore behooves this project — or "movement," as Dave so generously calls us — to find some sort of empirical way of measuring Arschlochkeit.
As it happens, organized hockey has been recording a statistic called the plus/minus. Plus/minus is quite simple: it measures the team goal differential when a specific player is on the ice. According to LCS Hockey's thoughtful analysis of plus/minus, this metric awards "a plus . . . to all skaters on the ice when their team scores at even-strength or short-handed" and assesses "[a] minus . . . to all skaters on the ice when their team yields an even-strength or short-handed goal." (Those familiar with hockey will notice that power-play goals in favor of the team with superior manpower do not advance plus/minus.) The upshot is that if your team scores while you are skating, you gain plus/minus points. If the other team scores on you, your plus/minus regresses.
The problem is that plus/minus is a team-dependent statistic and requires further analysis before it sheds light on individual talent (or its absence). As LCS Hockey observes: "One player can only do so much. He could be the best defensive player in the history of the game, but if he's out there with four stiffs and a sieve, chances are he's going to finish on the minus side of the ledger." Hockey's challenge reminds me of the problem that Paul Edelman and I faced over the course of our Most Dangerous Justice series of articles (parts 1, 2, 3), which attempted to discern voting power on the Supreme Court on the basis of each Justice's participation in winning coalitions.
Again, from LCS Hockey:
Lousy teams are going to have lousy plus-minus ratings. The expansion 1974-75 Washington Capitals, perhaps the worst team in NHL history, owned nine of the 13 worst plus-minuses that season, including Bill Mikkelson's -82, which still stands as the record for futility. In fact, not one Capital player that season finished as a plus, a natural result of being outscored 446 to 181.But filtering plus/minus figures by overall team performance does shed light on individual players' value. For example, a player who is +15 on a powerful offensive team may well be less valuable than a player who posts a valiant +10 on a poor defensive team. A player who posts a negative plus/minus figure for a Stanley Cup-winning squad is an affirmative liability.
Plus/minus is robust enough to capture value from different positions in hockey. Both Bobby Orr (lifetime +597, pictured above in one of his most triumphant moments) and Wayne "The Great One" Gretzky (+518) are among professional hockey's all-time plus/minus greats, even though they played different positions in radically different styles. The statistic recognizes lifelong consistency: Ray Bourque (+528) holds the third-highest plus/minus in NHL history without ever winning a single-season plus/minus title.
All that remains is to find an academic equivalent of team goals, for and against, that would power an analogue of plus/minus in our profession. Alas, like obscenity, Arschlochkeit defies definition, let alone quantification, even though we know it when we see it. People who threaten junior colleagues with negative tenure votes on nonacademic grounds are Arschlöcher. So are those who invite other colleagues to think of the school's top administrators according to invidious racial stereotypes (going so far as to describe one administrator as "lazy and shiftless" and another as "crafty and inscrutable"). [Yes, Mr. Worst-Professor-in-America, I'm talking about you.] But these are hardly things that get tracked on Westlaw and SSRN.
Perhaps it's time to revisit a post I wrote long ago, suggesting things faculty governance might learn from Wikipedia. Wikipedia's list of policies identifies a wide range of antisocial behaviors that can scarcely be tolerated in that collaborative environment. These range from the obvious — personal attacks, legal threats, general incivility — to forms of conduct that are as colorful as they are crass — sock puppetry, edit warring. Courtesy of plus/minus, hockey teaches us to count team-wide outcomes and to notice who's skating when good and bad things happen. And courtesy of Wikipedia, we may yet learn what things to count, in an enterprise where goals elude definition, let alone trigger celebratory lights and sirens. The grand problem of knowing what to do with this information remains. In the meanwhile, though, we'd do well simply to take some preliminary steps: deciding what the quantifiable constituents of Arschlochkeit are and undertaking to measure them.
11 Comments:
I always get nervous when that worst professor in america issue comes up. I think it is a high time for a new version of "three deans" in which the Arschloch and destructive qualities of deans is discussed.
My comment isn't about tenure directly. But I think it may, in some odd way, touch on it...
A hypothesis: Every law school provides BASICALLY the same quality. I mean, all the teachers at the ABA-approved schools went to Harvard/Yale/Stanford, plus a few other schools. There is tremendous competition for all academic jobs, and the same pool of people teach at all law schools. Therefore, the teaching is of the same quality everywhere.
The teaching methods and classes are generally fairly consistent at most law schools. This is especially true of the first year courses, which is what employers scrutinize most heavily.
Stanford isn't supposed to teach you better than Louisville. It's just that better students go there to begin with. Law schools simply serve as an expensive sorting mechanism.
So why bash USNWR? It seems that law schools would have little reason for existence without it.
This is a fabulous post. Three things to add:
1. Ray Bourque did win a Stanley Cup - in 2001 with the Avalanche. I think your point stands, though.
2. One point about the +/- that is undeniably present and important but sometimes hard to measure is the ability of one player to raise the play of others. Players on the same line as Peter Forsberg tend to have higher +/- ratios than if those same players were on a third or fourth line. Measuring the effect of this is difficult, though, do these players score more, do they play better, or are they along for the free ride?
3. I am tempted to agree with the anonymous poster's hypothesis but for some differences between schools:
a. The types of student who attend a school can affect the classroom environment and thus affect quality of education.
b. Some schools have more resources to offer additional programs - more classes, more clinic, more ... that can affect quality of education.
c. Even if professors are the "same" wherever you go, some schools have much more of an emphasis on teaching, and thus both develop better teaching methods among the faculty and create environmental pressure for improved teaching (availability norms, quality evaluations, etc.).
I should add to point 2 of my last comment in keeping with the Arschloch theme - some players can bring down the play of others as well!
The +/- rating is a great way to compare players on a given team... or in your hypothesis, a school. So while the quality of facilities, students, funding, etc. would make it difficult to compare Ivys to community colleges, a professorial plus/minus rating could be very useful for comparing teacher quality within a given school.
In hockey, there are no free rides. With only 5 skaters, a player without talent/effort won't have a good +/- total since likely his poor play will drag down his linemates. The original post's comment about a +10 on a bad team being more impressive than a +15 on a good team is absolutely correct; the same idea could apply to schools.
Each hockey team, and school, basically has a bell curve of +/- ratings. Being above average for your team/school is good, regardless of your team/school's quality.
Apologies for babbling. Excellent article!
Thanks for continuing the dialogue.
1. Let's assume, for sake of discussion, that there were some way of measuring law school performance that's akin to the W/L/tie available in the NHL, and that schools competed in the same way. The trick is to make more fine-grained analysis of team/school results. Hockey allows that through frequent substitutions; the Bruins can isolate the performance of a center from the performance of the team as a whole because it puts different configurations on the ice. (Line substitutions impair, but do not eliminate, this.) The result is that they can assess players individually and more accurately than by saying, for example, that the team has had a plus/minus score of X since so-and-so joined the team in 1999.
Can law schools do this? The big problem is that they lack anything comparable on the output side -- nothing like goals or wins that are measurable in the short term. Otherwise, you may have stumbled upon the hidden rationale not only for sabbaticals, but also for the increasing popularity of visits: the opportunity to make discrete, per-person measurements!
2. Personally, I think football and baseball are more fruitful for true MoneyLaw, since hockey is relatively bereft of statistical measures. But if you want to explore sports metaphors more generally, I would think the hockey answer for Arschlocher was suggested by your photo: simply hire your own goon.
This is your cue to insert a video for Warren Zevon, Hit Somebody.
Dear Jim:
Thoughtful post as usual. But there appears to be a flaw with your analysis. The reason why the Arschloch is a problem for a law school is not because that person is necessarily less talented and thus brings down the team average (measured in term of publication, teaching ability, grants, etc.). The Arschloch is a problem in law school's because that person is an internal goon. Unlike the hockey goon who goes after the other team, the Arschloch goes after his/her own teammates. The problem is exacerbated when the Arschloch is not talented (and thus is likely to have a negative +/- rating). But the fundamental problem is having someone on your team or faculty who will intentionally harm his/her colleagues to benefit themselves. In short, you seem to have equaled a "Deadwood" with an "Arschloch" and I don't think they're comparable in academic institutions where tenure is a given. You can make up for the deadwood by hiring talented people to raise the institution's average. The only way to deal with the Arschloch is to leave the institution.
I don't entirely agree with this last criticism. The idea of plus/minus is that it allows one to measure whether someone brings down overall performance, not just the cumulative average (as revealed by, say, batting average). This is relevant if one cares about overall performance in the very short term. So one assumes that an Arschloch hurts team performance, not just average -- even if one can also imagine an Arschloch who seizes all available perks but teaches her chosen classes superbly, is beloved by students and donors, etc., so as to help overall performance.
What the comment may be getting at is that there are distributional and morale effects. This is certainly ignored by a plus/minus measure in the very short term, since all the goals can be scored by the player in question, etc. Taking measures over longer periods help, but probably cannot take into account effects like the departure of peeved colleagues, impaired recruiting, and so forth.
To keep this on the straight and narrow, it might be useful to try to think about how Arschlochkeit might be objectively measured on an individual basis -- assuming no good team/school measure -- to prevent the kind of biases and clubbiness that allegedly infect personnel decisionmaking.
Jim: My head is spinning because the underlying assumption of the discussion seems to be that a faculty is affected by (or is that afflicted with) one or two Arschlochs. If we are talking about actual faculty productivity and not social comfort (I am not saying the are completely separate) the the problem can be more pervasive than weeding out one Arschloch. In fact, Arschlochkiet can characterize a critical mass of people, albeit a minority, who set the personality of an institution and affect others. And, when the Arschloch's have enough power they are the ones who define what it means to be an Arschloch.
"And, when the Arschloch's have enough power they are the ones who define what it means to be an Arschloch."
Ooh! To paraphrase, "We have met the Arschloch, and he is us."
The Arschloch problem is also one which shows up at Irish music sessions--regular informal gatherings of Irish musicians. Here, one arschloch can kill a session of some years' standing. The most effective solution is usually for one person to assume the role of policeman and to drive the offender away or at least to effect a mending of the ways. Whether someone is an arschloch is determined by consensus in this setting.
In a teaching environment, though, direct interactions between teachers may be less important than at Irish sessions or on hockey teams. Interactions between professor and student, however, are important, and are measured by grades, standardized tests, and other quantifiable standards. It might be possible to devise a plus/minus rating which measures a professor's influence on students. How many of that professor's students ace or fail later courses?
Post a Comment
<< Home