Tuesday, May 29, 2007

Whom Would Billy Beane Admit?

In keeping with my back to first principles theme, I have been wondering for some time what the Billy Beane approach to law school admissions would be. We know that Billy Beane's principal contribution to running a baseball team was to discover underutilized statistics that were better at predicting big league performance than those being used by other clubs. (For example, Beane was one of the first general managers to make regular use of OPS.) In this article, Beane describes his project as essentially arbitrage for baseball players:
"Arbitrage. We don't use that word too much in baseball, but that's what it is. In a market where people are competing for scarce assets -- for us, it's players or, really, the things that players can do -- there's always going to be some inefficiency. We're always going to have to find that dark corner, the stone that hasn't been turned over."
By looking under different rocks than the other GMs in the league, Beane was able to gain a competitive advantage and to sign promising players overlooked by others.

Are there similar, underused criteria for admission to law school? We know that most schools use some combination of LSAT score and undergraduate GPA (UGPA) as the primary factors in their admissions, largely because these factors are agreed to be less bad at predicting law school performance than others. If a school could find other criteria that worked even marginally better than LSAT and UGPA, they would be at a huge competitive advantage with regard to admissions.

My colleague Joyce Sterling and I spent some time looking for these factors a few years ago. We ran a multivariate regression on a number of possible admissions criteria -- undergraduate major, presence of an advanced degree, work experience since undergraduate graduation -- to determine whether we could find the OPS of law school admissions. We found nothing that correlate but if we had I sure wouldn't share it here. If everyone picks their players based on OPS, the advantage is lost.


Blogger Jeffrey Harrison said...

Sam: I am curious, what did you use as the dependent variable in "law school performance." And just to make things a bit more complex, does "winning" mean getting the best possible performance or does it mean getting the best performance from a given set of students. Wouldn't this second measure be a better gauge of law school performance?

5/30/2007 1:10 PM  
Blogger sam kamin said...

Jeff --

Ok, I lied just a little. We were actually doing our study in the context of bar passage. We had previously found, as has nearly every other study in the field, that the best predictor of bar exam performance is law school performance. So, we were hoping to bring in students who would be successful both in law school and on the bar.

As to how you win the game, I think that in general you win by getting the best students you can, the best faculty you can, and by getting the most out of both.

Other schools set more specialized goals. For example, there are certainly schools out there that specialize in taking students with low predictors and making them into successful lawyers. If they have chosen to set that as their goal, I think that they're winning.

Billy Beane knew he wasn't going to get the best players that money could buy, so he tried to put together the best team he could. At our school we know we're not going to get the students with the 178 and the 3.9, but if there are great "bargains" to be had among the 160s, that's where our school can really improve.

5/30/2007 2:10 PM  
Blogger Jeffrey Harrison said...

Sam: Thanks. UF is in the same boat. Probably everyone knows this but I am wondering if you found a relationship between bar passage and LSAT score.

I also wonder if you looked at courses taken. Here we are attempted to determine what can be done to raise bar passage. One theory -- completely untested -- is that students who do not do well grade-wise in the first year then take various externships, independent students, and seminar that are unlikely to prepare them for the bar exam.

5/30/2007 3:14 PM  
Blogger Patrick Lyons said...

It seems to me that the real goal of all this should be to make the best lawyers. What good reason is there to believe that bar passage is at all a better indicator of being a good lawyer than say law school grades? Maybe your best bet would be to test things like conviction percentage for ADAs or odds at entry of making partner at a certain class of firm, both tested over long periods.

6/20/2007 3:30 PM  

Post a Comment

<< Home