Validity of SSRN's "# of New Papers" Measure
I earlier related the kerfuffle that followed Chapman Law School's acknowledgement of its recent success in the SSRN's "# of New Papers" measure. A skeptic might conclude that Chapman drew criticism primarily because it "acted above its station," daring to trumpet that it had bested the champions of USN&WR's law school rankings. Perhaps so. I find more interesting, however, the claim that Chapman erred in describing the SSRN's "# of New Papers" measure as a "key scholarly output ranking." Outlandish puffery or accurate reporting?
Bernard Black and Paul Caron consider the utility of SSRN's measures in their paper, "Ranking Law Schools: Using SSRN to Measure Scholarly Performance," 81 Ind. L. J. 83 (2006). They argue that both the number of posted papers and the number of downloads “play a valuable role in the rankings tapestry," offering new and potentially more accurate alternatives to the USN&WR rankings.
Black and Caron observe, moreover, that measures of the number of papers posted to SSRN arguably offer a more accurate measure of genuine scholarship than measures of the number of papers downloaded from the network. Why so? Because the latter offers more opportunities for gaming the system. Though I again admit that I'm not purely disinterested in the matter, I thus conclude that the SSRN's "# of New Papers" does plausibly qualify as a key measure of scholarly output.
Granted, none of the SSRN's rankings work perfectly. In particular, as I observed in my earlier post, the "# of New Papers" measure could stand some improvement. Black and Caron put it this way: “The SSRN measures have important field and other biases. Still, they offer up-and-coming schools a way to ‘show their stuff,’ long before the US News rankings respond to the school’s improvement. That alone is an important contribution.”
[Crossposted to Agoraphilia.]
Bernard Black and Paul Caron consider the utility of SSRN's measures in their paper, "Ranking Law Schools: Using SSRN to Measure Scholarly Performance," 81 Ind. L. J. 83 (2006). They argue that both the number of posted papers and the number of downloads “play a valuable role in the rankings tapestry," offering new and potentially more accurate alternatives to the USN&WR rankings.
Black and Caron observe, moreover, that measures of the number of papers posted to SSRN arguably offer a more accurate measure of genuine scholarship than measures of the number of papers downloaded from the network. Why so? Because the latter offers more opportunities for gaming the system. Though I again admit that I'm not purely disinterested in the matter, I thus conclude that the SSRN's "# of New Papers" does plausibly qualify as a key measure of scholarly output.
Granted, none of the SSRN's rankings work perfectly. In particular, as I observed in my earlier post, the "# of New Papers" measure could stand some improvement. Black and Caron put it this way: “The SSRN measures have important field and other biases. Still, they offer up-and-coming schools a way to ‘show their stuff,’ long before the US News rankings respond to the school’s improvement. That alone is an important contribution.”
[Crossposted to Agoraphilia.]
posted by Tom W. Bell at 2:58 PM
1 Comments:
I couldn't agree more with the view that the Chapman criticism is tinged with snobbery. There was something very mean-spirited about the glee and vigor with which Chapman was called out for daring to suggest that perhaps it is more "scholarly" than its presumed "betters" (and the USNWR folks) consider it to be. I believe the original analysis of Chapman's "mistaken" claims even refers to Chapman as being among the law school "have nots."
Imagine, by the way, if Duke or Chicago or NWern had made a similar claim based on postings dating back to the early 80s. Sure, someone might've pointed out the methodological problems in their approach, but I doubt it would have been conducted as the condescending dressing-down that Chapman received.
Post a Comment
<< Home