Rankings redux
I just posted, over at my own blog, a question about whether touting SSRN "top 10 download" status would be one useful way to help people become aware of what the faculty of a school has been doing in terms of research. On the one hand, it's some indication of productivity and interest; on the other, SSRN deals with articles and not all of the other forms of scholarship, and it might be possible to game this system, too. I don't know if emphasizing such things would become so much "noise" (a la car alarms, fancy brochures about law schools that tend to come out in early September, the rankings themselves) or whether law professors would find this type of information interesting or useful. If you want to discuss this, you could either do that here at MoneyLaw or over at my blog. If SSRN downloads wouldn't be useful, would something else be useful? And--this is the more important question--would monitoring this sort of thing create behavior that we don't want to encourage?
6 Comments:
The fact that a paper has been uploaded seems much more significant than the number of times others have downloaded it. In fact, I am not sure what that adds to the mix. Plus, I wonder if there is a vanity element element in the SSNR plan. Like you I was interested to find that an article I uploaded is on two top ten lists. But, it's almost as though there are so many categories that everyone is on one or the other. Next thing you know making a top ten will listed on resumes. I don't know if this a product of my cynicism or of a feeling that nothing I have written really deserves to be on a top ten anything.
A list of what a law school's faculty has uploaded each week or month would help me get a better feel for productivity than downloads.
Cool idea, Jeff--how would someone gather that kind of info?
What about faculty workshops (times, subjects, speakers) or works in progress? Would that be useful?
On second thought, what do we do about gathering info on faculty members who write more books than articles? Would they be shortchanged by "uploads on SSRN"? Is there a way to gather that AND info about books?
Paper downloads do not seem to be a very useful indicator. In the course of researching a topic, I have had to download several articles because they contain my search terms but the abstracts did not give me enough information so that I could rule them out before increasing the author's download counter.
Downloads are probably easy to game. For example, a download counter could be increased by assigning a paper as assigned reading. Having students log-in and download papers may inflate the perceived value of the paper.
As far as the number of uploaded papers goes, it doesn't say anything about the quality of the work. At least with citation counts, there is an indicator of how useful an article has been for some purpose (whether as a prime example of incorrect analysis, or good analysis/unique thinking). The description of industrious drudge on Leiter Rankings may be helpful in evaluating the use of download and upload statistics.
Not only does SSRN undervalue books -- it also overvalues essays. My download stats look quite impressive until you notice that a large proportion of the downloads come from a series of legal writing essays I've posted.
Because of the gaming possibilities, the over- and under-valuing of certain types of scholarship, and the occasional idiosyncratic "home run" (e.g. Fairman), I think SSRN data is only likely to be useful in the aggregate. Even then, I'd take it with somewhat of a grain of salt. For example, my institution has been extremely successful on SSRN relative to our USNWR ranking. That's probably reflects in part the fact that we do more quality scholarship than most folks might expect, but it also reflects the fact that we've gotten almost all of our productive faculty to participate in SSRN -- something that may not be true at all schools.
I really like the study that Rick Bales mentions in his comment. Is there any way to take the research farther down the list to see if there are certain bands of schools that naturally group together?
Post a Comment
<< Home