MoneyBall MoneyLaw

When I heard they were making a movie based on MoneyBall I could not sleep just thinking about who would play the key roles in the obvious sequel, MoneyLaw. Would some of us have bit parts? Alas, the MoneyBall movie has been deeply back burnered and, I think this means the MoneyLaw movie is similarly delayed but probably only for a millennium or two.

With respect to the actual Oakland A's there is good news and bad. They are in last place. But among last place teams they have or are tied for the best record. Somehow, if this were all switched to law schools and those who play the USN&WR game (and who doesn't?), I think we could show just how commendable this is. We would just create a category -- really awful law schools. Then rank within that category and in the decanal glossy it would read "A's Law School Ranked Tops in Field." Another possibility is just to be better but that is so much trouble.

Z-Scores in Model of 2010 USN&WR Law School Rankings

If you want to know how U.S. News & World Report's law school rankings work, you'll want to know about z-scores. In very brief, z-scores measure how well each school performed relative to its peers, thereby establishing its rank. (See here for a fuller explanation.) My model of the rankings aims to recreate those z-scores, and thus the rankings themselves, by duplicating both the data and the methodology that USN&WR uses. Here are the results for the law schools most recently ranked in the top 100:



For cross-year comparisons, please see the similar reports I offered in 2005, 2006, 2007, and 2008. This year, in response to a reader's request, I've added various diagnostic measures, such as the mean, median, and standard deviation of each itemized category of data. As I did last year, I again provided weighted z-scores, meaning simply that I've multiplied the z-scores in each category of data by the percentage that category influences a school's overall score. That method of presenting z-scores has the virtue of highlighting which scores matter the most.

Unsurprisingly, you'll generally find the largest numbers in the upper, left-hand corner of the chart. There lie the most heavily-weighted z-scores of the law schools that scored the highest in USN&WR's rankings. Consider, for instance, the .71 weighted z-scores enjoyed by Yale and Harvard under the "PeerRep" category; those numbers nearly swamp the effect of other measures of those schools' performances, and have twice the impact of the peer reputation scores of schools ranked as close as 20th from the top.

This presentation of the data also shows how very little influence many of the things that USN&WR measures have on its rankings. The weighted z-scores for Bar pass rates, for instance, vary between only .07 and -.02, with a whole lot of zeros filling that span. Bar passage rates evidently do not matter much to any school's USN&WR score.

Rankings geeks will doubtless find close study of this table rewarding. I'm especially interested in the surprising impact of the top schools' overhead expenditures/student—a phenomenon that I discussed in some detail last year. Perhaps I'll return to that topic, and raise some new ones, in later posts. In the meantime, I welcome your own observations.

[Crossposted at Agoraphilia, MoneyLaw.]

Accuracy of the Model of the 2010 USN&WR Law School Rankings

I earlier offered a snapshot comparison of the scores generated by my model of the 2010 U.S. News & World Report law school rankings and the original. After Robert Morse, director of data research for USN&WR, asked me if I could quantify the fit between the two data sets, I realized that others might share his curiosity. Here, then, are the r-squared measures (more precisely, the squares of the Pearson product moment correlation coefficients) for each of the models I've done over the past few years:

2010 rankings: 0.999
2009 rankings: 0.999
2008 rankings: 0.999
2007 rankings: 0.997
2006 rankings: 0.995

What do those numbers mean? In brief, an r-squared closer to 1 (or –1) shows a closer fit between the two data sets. It might seem a bit absurd to report these results out to three decimals, but I wanted to make clear that the model has yet to obtain results absolutely identical to those reported by USN&WR. I daresay, though, that any r-squared above .99 shows a pretty strong correlation.

[Crossposted at Agoraphilia and MoneyLaw.]

A Model of the 2010 USN&WR Law School Rankings

As in every year since 2005, I this year again built a model of the law school rankings published by the U.S. News & World Report ("USN&WR"). Figuring out the rankings—the "2010" rankings, as USN&WR's calls them—proved especially trying this time around. USN&WR changed several parts of its methodology this year and the ABA, which distributes statistical data on which my model depends, fell far behind its usual publication schedule. Finally, though, the model ended up generating scores gratifyingly close to those that USN&WR assigned law schools. Here's a snap-shot comparison of the results:



For details about how and why I modeled USN&WR's law school rankings, as well as for similar snap-shots, see these posts from 2005, 2006, 2007, and 2008.

Perhaps in later posts I'll offer some reflections on what this year's model of the USN&WR rankings teaches. For now, I'll just offer this happy observation: The close fit between USN&WR's scores and the model's scores suggests that law schools did not try game the rankings by telling USN&WR one thing and the ABA (the source of much of the data used in my model) another. Even a skeptic of law school rankings can find something to like in that.

[Crossposted at Agoraphilia and MoneyLaw.]

A fresh take on rankings, redux

I posted my latest thoughts on possible alternative rankings systems here. Would love your comments. Thanks, and happy 4th!