What to Make of SSRN Downloads?
I am not sure. I like reading about various ways of ranking law schools, faculties, and reviews because there is no one way that captures what it means to be productive. Thus, it is a wonderful subject for debate (unlike many scholarly issues in legal education where debate is not permitted.)When I read the most recent posting about tax profs, I did not know what to make of it. Although I personally oppose educating tax LLM students on the taxpayers' tab, the tax profs at my law school have an exemplary work ethic and I was surprised that none of them ranked in the top 25 as far as downloads.
My curiosity led me to select three of the profs from my School and tally their citations by scholars and courts using Westlaw. I then selected three professors from roughly the middle of the top 25 SSNR downloads. I tallied their scholar and court citations also using Westlaw. On average, my three colleagues outperformed the middle of the top 25.
So, I ask this quite seriously, what do SSRN downloads tell us? And, is it a better measure of productivity, impact, or the importance of an individual's work than a Westlaw citation search?
posted by Jeffrey Harrison at 3:34 PM
6 Comments:
Isn't this an apples and oranges comparison? Westlaw citation counts are biased in favor of papers written years ago, while SSRN downloads are biased in favor of recent scholarship (even all-time downloads, since it is still a relatively young service). Furthermore, citation counts include treatises and casebooks cited by courts and articles, while SSRN downloads do not (unless someone posts an excerpt from one of those, which is relatively rare). Thus, all you have established is that UF's tax professors are probably older and more heavily invested in treatises and casebooks than the comparison group you selected. That would be consistent with extrinsic evidence on the issue, which isn't to say that SSRN is necessarily preferable to citation counts, although a faculty committee would probably look there first to find currently active scholars.
One big problem with taking SSRN citation counts to mean anything important is how easily they're manipulated. Some people actively strategize about getting their SSRN papers mentioned on certain blogs. I know someone who has a roster of blogs to which s/he regularly e-mails his/her papers. And s/he once got a student to post something on a student blog encouraging everyone who knew and liked that prof. to download the paper to increase the citation counts.
I think what it tells us is that the best (and perhaps only) way to assess quality is for people with good knowledge of a field to read a writer's work and provide a written evaluation of it.
If such a method proves incapable of "ranking" people, then so be it.
Hopefully "post-publication peer review" processes such as those proposed by Mike Madison in his article on OA will help make such indications of quality as accessible as download counts.
by the way--fantastic picture!
SSRN downloads tell you the number of downloads of items listed. Not all scholarship is listed. Ranking professors by SSRN count is like ranking amusement parks on the basis of the number pamplets picked up in a hotel lobby. Before SSRN people did the equivalent of downloading by photocoping articles. And, before they did that, they examined them in the library. Those that were useful were cited. I think you should stay with citations.
Ranking is problematic. Consider the too-common law review and journal process of evaluating submissions more on the basis of the CV than the writing. Without devices to make the evaluation blind, subjective reputational factors will come in. And there are tales in peer-reviewed fields of territorialism, where an established authority in the field will dispargage good work that might support the career of a competitor or someone who would supplant the reviewer's line on the field.
I agree with commentator 4. SSRN downloading is like browsing in the library was 30 years go. Plus the library's holding are limited.
Post a Comment
<< Home