I am not so sure the same level of scrutiny is uniformly applied in law. I say this not because I know how decision-makers and deans evaluation resumes but because of the persistent use of "stats" that can be deceptive. Since I believe that markets sometimes work, I believe that some decision-makers must be buying what some law professors are selling.
1. Lists of books. The best book is a university press original offering. It is not a recycling of old articles which most books by law professors are. Also listed in the books category are collections of works by others. This takes the form of a book but is hardly a "book" in the sense that it signifies a traditional book-like effort. In fact, the professor's role here is more as an administrator. Casebooks are harder to classify. I'd rank them behind the real university press book and way ahead of the edited book of readings. Still, there they all are listed as "books."
2. Coauthors. It is very difficult to find any book that is not coauthored. This is especially true when it comes to casebooks. There is nothing wrong with this. In fact, it probably makes for better work. But, I have seen way too many claims to be an "author" as in "I have published 3 books" when the person was one of two or three This problem is magnified when the author is actually a "coauthor" of a selection of readings by others. The same issue comes up with articles. Professor X boasts of 15 articles and Professor Y has measly 10. X's are all authored, none of Y's are and the difference goes unnoticed.
3. Articles. Don't even get me started. I have now seen listed and reported as "articles" everything from 2 page introductions to sweeping 100 page treatments of complex issues. Yet from the mouths of many of those with the 2 pages I hear "I have written X articles." Worse I have seen decision-makers look no further than numbers.
4. Placement. Privately professors will concede that placement is a crap shoot Actually, not quite. Your chances of a high placement soar if you can list some elitist credentials. Placement for my purposes here means was the article send into the marketplace for articles or was it a solicited symposium article. Not only are the latter generally shorter but placement is based, nearly exclusively, on what the author has already said somewhere else. Still, there they are listed along with other articles and often counted as equal to the the other articles. Don't get me wrong. Some of these pieces are better than standard articles but, for the most part, they are not selected by someone looking at two or 2000 completed articles and deciding which is best. To me ten articles send out to fight it out with the other hundreds of articles law professors produce is, without more, way ahead of ten invited offerings in which the author is often asked to preach to the choir.
5. Speeches and presentations. This part of a resume has always puzzled me. Is it scholarship? So far, I do not think I have heard a talk that was not part of something already published or part of something to be published. So, what is the contribution here? Orally presenting something that can be read? More importantly, if these talks are viewed as scholarship, isn't the risk of double counting huge?
6. Double counting. There are many ways to do this. I have noted lunch time talks and recycling articles into books. I have also seen this. Under articles there is an entry. The same entry is then found in "Book Chapters" because the article also appears in another book of readings.
All of this does go on. A few months ago I surveyed 4 schools at the level of my own to determine if elite credentials were good predictors of future scholarship. They were not. In the course of that, I discovered a number of outliers -- people with resume items way in excess of the average. In every case, all or some of the above were at work. Of course, it also goes in the context of shorter resumes.
I'd like to think law schools are as good as sports general managers at looking past the numbers but I am not sure they are. Obviously, a great number of law professors are counting on the possibility that they are not.