Monday, November 26, 2007
Skepticism About Leiter's Citation Rankings
Like Mary Dudziak, I’m skeptical about Brian Leiter’s latest citation rankings of law professors. In response to Dudziak’s concerns about the study, Leiter claims that “The study is a ‘true measure’ of what it purports to measure, namely, impact in legal scholarship.”
Sounds like an example of the classic maxim:
There are lies; there are damned lies; and then, there are statistics.
As a practicing criminal defense attorney, my problem with the rankings in criminal law and procedure is the practical experience level of the professors ranked. Any of them voir dire a jury? Cross-examine a hostile witness? File, brief, and argue a motion to suppress? Writ up a judge? It's a bit hard to take these folks seriously from here in the trenches.
WANTED: LEAGLE EAGLE
Hey, all of you law professor types: one of you ought to join in the discussion over at Glenn Greenwald and Atrios and the Anonymous Liberal and Firedoglake concerning Joe Klein's idiocy over at the Time Magazine Swampland blog regarding the FISA legislation before Congress.
In a column to be published in Time Magazine, he utterly mischaracterized the FISA legislation and then slammed the Democrats for being weak on national security based on his mischaracterization. He actually accuses them of giving foreign terrorists the same rights as Americans. As Greenwald said:
"The most obvious and harmful inaccuracy was his claim that that bill "would require the surveillance of every foreign-terrorist target's calls to be approved by the FISA court" and that it therefore "would give terrorists the same legal protections as Americans." Based on those outright falsehoods, Klein called the House Democrats' bill "well beyond stupid.""
Why doesn't one of you join in the criticism of Joe Klein for his idiocy? His piece is slated to be published in the next issue of Time Magazine, helping reinforce among millions of readers the stereotypical critique of the Democrats as being soft on terror.
Klein refuses to make an honest correction and admits that he got his interpretation from a Republican congressional staffer. He appears not to have consulted any Democrats or, say, this web site or any of the others that could have explained the facts to him.
He deserves to be made an example of -- to take a shot across the bow of the media whores to let them know that they are not going to sabotage another Democratic election unchallenged.
A trenchant analysis of exactly how wrong Klein is -- and why and how inexcusable his "error" is -- coming from one of you would be very powerful.
fo wheel? Who cares? Is balkinization really worthy of this discussion of shameless academic-penis-measuring?
As an example of how citation count can mislead as to impact, I wrote a law review article which had a footnote with the outcome of a particular question in every state. This article so far has had 12 citations--11 for the summary in this single footnote and only 1 for the substantive topic addressed in the article. The "impact" of the article should best be measured by just counting that 1 citation, not the other 11.
BT is absolutely right: Leiter's methodology is nonsense. And I'd go farther than BT does in condemning Leiter's fascist tendency to relegate to the trash heap any approaches to law & philosophy that aren't on all fours with the particular kind of training he had in analytic philosophy.
I think the part of Leiter's study that bothers me most is that he thinks students care about these rankings. Why on earth should 99% of students (those without academic aspirations) care if their teachers are much-cited (or impactful) scholars?
And if it's true that students don't care, then who are these rankings for? Are they just a glory list, similar to a law firm sending around a list of who billed the most hours this month? Is there any actual information here for scholars?
(Where Leiter to permit comments on his blog, I could have asked these questions in a more appropriate forum.)
Fine method of telling, and enjoyable article to acquire factual statements.local citation servicePost a Comment