Tuesday, September 26, 2006

Why Abandon Average LSAT Scores for High Ones?

Standardized testingImagine yourself as the American Bar Association Section of Legal Education and Admission to the Bar. You have been asked to approve a policy recommendation, originating from the Questionnaire Committee, that looks likely to: 1) Make LSAT scores less accurate at predicting law school success; 2) Increase the LSAT scores of Caucasian and Asian American law school applicants relative to the LSAT scores of applicants in other racial or ethnic groups; and 3) Double the number of law school applicants who expend time, effort, and money retaking the LSAT. What do you do?

Regardless of what you would have done, the real ABA adopted the policy recommendation in question. In other words, the ABA Section of Legal Education and Admission to the Bar decided last June to change the definition of the LSAT data it collects from its member law schools each year. Whereas formerly it had asked law schools to use each student's average LSAT score in calculating the 25th, median, and 75th percentile LSAT of entering classes, the ABA will now ask law schools to use each student's highest score in calculating those figures. The 2006 version of the Annual Questionnaire that the ABA requires each member law school to complete already reflects the policy change. U.S. News & World Report's questionnaire will doubtless follow suit.

How could that seemingly minor change to the ABA's definition of "LSAT score" have the effects described above? Sam Stonefield did an admirable job, in his May 24, 2006 letter to the Section of Legal Education, of explaining why. To his able analysis, I would add only a reference to a research study by the very organization that administers the LSAT—the Law School Admission Council: The Performance of Repeat Test Takers On the Law School Admission Test: 1997–1998 Through 2003–2004 Testing Years (TR-04-02). It reports that among those who took the LSAT more than once, "the mean score gains in descending order have been for Caucasian and Asian American (2.9 points), Mexican American (2.7 points), Puerto Rican and Hispanic (2.4 points), and African American (2.0 points) test takers." Retaking the LSAT thus seems to help most the people who need help least.

If you successfully imagined you were the ABA Section of Legal Education and Admission to the Bar, perhaps you can tell me why that body decided to start ignoring repeat test-takers' average LSAT scores in favor of their high LSAT scores. I'm having trouble with that mental exercise. My natural skepticism keeps getting in the way. It causes me to dwell on the likelihood that LSAC stands to make a lot of money thanks to the policy change, which will lead many more students to retake the LSAT, and on the fact that the ABA's member law schools own and operate LSAC. Curious about the details of that organizational relationship, I've requested the LSAC's federal tax returns for the past three years. Perhaps the details in those dry documents will stimulate my imagination, making it easier for me to understand the reasons for the ABA's policy change.

[Crossposted to Agoraphilia.]

1 Comments:

Blogger William Henderson said...

Tom,

I read the same item in the Section of Legal Education newsletter. It was very strange indeed. It provides a huge incentive for students to keep taking the LSAT in the hope of getting one magic score. Further, schools are likely to use those high scores instrumentally to boost their own median scores and, in the process, duck the clear empirical evidence that average scores of multiple LSAT tests predict the best.

By the way, does anyone really believe that many law schools still rely upon the LSAT/UGPA admissions index (which is weighted on a school by school basis using three years worth of data) to make admissions decisions? My colleague Jeff Stake has documented that the midrange between the 25th and 75th LSAT percentile shrunk 1.5 points between 1998 and 2004. If the usual compensatory model were being used, compression of this sort is quite unlikely.

The system is completely broken, and this puzzling decision only makes matters worse. bh.

9/26/2006 3:02 PM  

Post a Comment

<< Home