Mythical law school rankings
 |
Pallas Athena and the Pigskin
As the college football season winds down, it bears remembering that the NCAA recognizes no official champion for the Football Bowl Subdivision (which real fans will forever call Division I-A). Even if this season's BCS National Championship Game manages somehow to reprise the 2006 Cotton Bowl with a rematch of Alabama versus Texas Tech (this time with zero rather than two losses apiece), major college football will stage a mythical national championship:
A mythical national championship . . . is a colloquial term used to describe a champion in a sport in which a championship is determined without the use of a playoff or tournament of some kind. It is most commonly used when referring to NCAA Division-I college football, since there is no playoff in that sport.Bo Schembechler, longtime coach of the once-great Michigan Wolverines, provided the definitive statement on mythical national championships:
"If there are any Big Ten teams that shoot for a national championship, they're damn fools," Schembechler said . . . . "You play to win the Big Ten championship, and if you win it and go to the Rose Bowl and win it, then you've had a great season. If they choose to vote you number one, then you're the national champion. But a national champion is a mythical national champion, and I think you guys ought to know that. It's mythical."Yeah, it's mythical. And it's stupid. Ask the President-Elect:
And even if this isn't change college football traditionalists and university presidents can believe in, never fear. Legal academia seems committed, if solely by reason of inertia, to mythical law school rankings.
Championships, athletic or academic, are bogus if they depend on subjective voting. College football deserves better, and if the NCAA heeds Barack Obama's advice, we might at last see a Division I-A football playoff. By contrast, legal academia is still struggling to meet a challenge that MoneyLaw laid down in its infancy:
No academic ratings system is valid if it depends in whole or in part on a subjective survey of academic reputation.
4 Comments:
1. Do you think Bo would regard a championship as any less mythical if it were based on an objective measure like net yards, or net points, or graduation rate? Probably not. So where is the head to head competition he'd demand? In the stats of entering students? Bar passage rates? Hiring?
2. If subjectivity, rather than the indirectness of competition, was the underlying problem, I would be interested in knowing what method might be suggested for assessment (or stitching together different means of assessment) wasn't subjective.
Again, I have sympathy with the approach here, but it's really hard to extractive normative lessons from these pronouncements in wildly disparate settings. Hard to see Billy or Bo trying to succeed by changing the rules of their respective games (e.g., adopting a fan-sensitive, or student-sensitive, metric for success). And hard to see Bo saying, "Consider the humble nautilus . . "
Does this mean that there is no champion in gymnastics, or diving, or ice skating, or, often enough, boxing? You can say yes, but to my mind that ends up being pretty much a reductio of your position. Certainly it's not one based on anything more than a bald assertion of personal preference.
To Ani:
1. Part of the problems is that legal education hasn't decided what "winning" means. To me, it's return on student investment. U.S. News and other self-appointed arbiters of quality in legal education have exploited the lack of consensus and converted the game into one of reputation. In all fairness, the big law firms have given the rankings complex a big boost by using them as a filter for hiring.
2. I've proposed the concept of reputation-independent performance statistics (RIPS), and had I time and world enough, I'd outline them here. You'll see them, or at least early echoes of what I hope to accomplish, in this forum.
And Matt:
Of course there is a subjective component in the evaluation of sports involving an artistic component (diving, gymnastics, skating, ice dance). Boxing, too. And even football and baseball, since offensive holding and the balk both depend on the eye of the beholder.
The difference is that these athletic realms offer and uphold reasonably discernible guidelines as to what constitutes mastery of uneven bars, legitimate pass defense, etc. In law we can't agree on what counts. And so we engage in exercises that hinge on the flimsiest of impressions about "reputation."
Thanks to both of you for your commentary.
I think that maybe there's more subjective element in judging in sports than we think, but that it washes out over a series of events. One way to notice this was to watch when the way that scoring in Olympic boxing was changed several years ago. (I don't know if they still use this system or not- it was very controversial after its first major use.) In pro boxing, and in the old system of Olympic boxing, the scoring of each judge is individually done- each judge gives a point for a hit that he sees. The scores are usually quite close together- not the same but usually quite close, and usually in the 80s or 90s. In the modified system used in at least one Olympics the boxer only scored a point if all three judges recorded a point within a short time of each other, in the belief that this would make scoring more accurate. What it did was give scores in the mid teens to low twenties. But no one believed that only that few punches were landed. So, this system made the scoring less accurate by demanding more "objectivity". (Intersubjectivity is usually all the objectivity we can hope for, after all.) How does this fit with reputation ranking? Well, one thing that seems bad about ranking by reputation is that obviously many rankers will know little about most of the members of any given faculty and little about areas of the law other than their own. But, if you get a big number of rankers in a wide degree of areas this is likely, it seems to me, to wash out, giving us a fairly useful ranking. Obviously it's not perfect. But unless you think there's no such thing as a better or worse here- something that's really hard to believe- then a big enough group of rankers with a wide enough body of experience areas should give us a reasonable scale.
That said, this will work much better for ranking like Brian Leiter's that purport to only rank on one axis- if you try to do what US News claims to do, and mix in a bunch of factors to get some ultimate ranking, the ranking is much more likely to be an artifact of the weights given to various factors, and those weights would need more of a justification than they seem to have. But, this doesn't change the fact that judging can give us quite good measures of skill or worth, especially when we have many judges judging over a large body of work.
Post a Comment
<< Home