The Most Cited Law Review Articles-- and the Rise of the Nodal Scholar

JB

Fred Shapiro and Michelle Pearse's newest study of the most cited law reviews is here.

In our 1996 essay "How to Win Cites and Influence People," Sandy Levinson and I dubbed Shapiro the founding father of a new field of study, "legal citology." Citation studies, we argued, did not merely measure, they also increasingly constituted how the legal academy constructs ideas of intellectual influence and merit

We had planned to write an entire essay along these lines, dutifully showing how academic conceptions of merit were constructed by the economy of citation and offering helpful suggestions on how to reform our current practices. But, after discussing the issue between ourselves for several weeks, we gradually realized that most readers of this Symposium probably couldn't care less about our views on these weighty theoretical matters. What most readers really want to know is who's made the all-star team in the law professor game and how they can get there themselves.

We therefore organized our article in the form of a tongue-in-cheek self-help manual.

Since Shapiro's original articles in 1985 and 1996, the influence of citation counts on the legal academy has become, if anything, even more pronounced, and new statistics, like numbers of SSRN downloads, have joined the calculus.

One way of looking at these developments is through the lense of fetishism-- through which particular kind of representations (or in this case, measurements) displace the "real" things (influence, merit) that they merely purport to represent. Under this view, citations draw us away from what is real, and increasingly engage in a fetishism of what can be measured and counted. In offering this sort of critique, we might further distinguish between mere "influence," on the one hand, and true "merit" on the other. Influence is a measure of how much one's work has drawn the attention of others, while merit is a question of how good the work is on its own.

Yet another, equally interesting way of looking at the process is that merit and influence were never fully separable from representations of them-- they were always in part what people thought about what other people thought about other people's work. If that is so, then changes in representations of influence and merit produce-- at least in part--changes in their content. Far from a numerical fetish, what we are witnessing are changing conceptions of merit itself.

This is, roughly speaking, the transition from the model of

* the profound scholar (i.e., one whose work is valued because it is "deep," regardless of -- and possibly even in inverse correlation to--its popularity); to

* the productive scholar (i.e. one who is valued for producing a great deal, in the way that a better factory produces more widgets per unit of time); to

* the nodal scholar (i.e., the scholar who is valued because of their presence at the center of a network). I use the term "nodal," because you can imagine each citation to be a one-way link from one node in a network to another. The scholars with the most citations are the dominant nodes in the network.

We know the profound scholar by connoisseurship: experts read the work and judge the quality of mind as a wine expert judges the quality of wines. The quality of the judgment depends on the quality of the experts, and thus it is not necessary that many people judge the work, only those who themselves have the appropriate degree of connoisseurship. Indeed, as noted above, widespread popularity (or even, in some cases, accessibility) may be inversely proportional to merit.

(An earlier version of this model is the model of the "brilliant" scholar that I associate with the mid twentieth century Oxford philosophy department, in which production of actual work--which is only a signifier of merit--is less important than the quality of the mind itself, and its influence on colleagues and students. The work is merely a representation of the mind that produces it, and it may be that the work never fully captures the brilliance of the mind, which is the real source of merit. Therefore, experts judge the quality of the mind--not the work--through repeated conversation and social interaction, and through personal influence on students.)

The movement from the profound scholar to the productive scholar takes us, haltingly and tentatively, from the world of expert judgment into the world of numbers and what can be measured. We judge the productive scholar by counting appropriate units of production. The more articles the scholar writes, all other things being equal, the greater the merit. Obviously, there is a danger is that the scholar will write loads of trash, but the assumption is that if a person writes enough, their mind is engaged, and there will be plenty of good ideas and good work.

The movement from the productive scholar to the nodal scholar is a movement from measuring units of the scholar's own production to measuring units of production of other people that refer to the scholar. We judge the nodal scholar by measurement not of the quality of the mind, or of the quality of the work or even the amount of the work, but by number of links to the work. By links I merely mean measurable choices to point to the work, either measured through hyperlinks, downloads, or citations. Merit is constructed through what network scholars call preferential attachment-- i.e., that some nodes in the network get many more links than others.

The more people choose to link to this scholar's work rather than to the work of other scholars, the more likely it is that others will find the scholar's work worth linking to, which means the more likely it is that it has merit. (The obvious analogy is to Google search results, which use a more complicated algorithm.) This creates what Clay Shirky has called "algorithmic authority."

To be sure, there is a danger that people are linking to work for reasons that have nothing to do with its quality-- because the scholar is clearly wrong, because the scholar has become a symbol of an idea developed better elsewhere, because the scholar is famous, or because the scholar is placed at high ranking institution that in turn produces many scholars. In many networks, the numbers of links form a powerlaw distribution, in which most links are to a comparatively small number of nodes, and the most linked-to nodes receive a disproportionate share of links. This phenomenon may create a "winner take all" effect, giving the impression that the most linked- to scholars are head and shoulders above the rest in quality, when in fact talent (however measured by other means) is far more evenly distributed. Nevertheless, the assumption is that linking will act as a sort of a wisdom of the crowd, separating out the wheat from the chaff.

Each conception of the scholar and scholarly merit is associated with a different conception of what authority is and how it is produced. In the case of the profound scholar, the authority comes from the judgment of a small number of connoisseurs, who know what merit is. In the case of the productive scholar, the authority is produced "objectively" by measuring units of production, which are countable. In the case of the nodal scholar, the authority is produced by measuring the structure of the network and systems of preferential attachment.

In actual practice, these models of merit are not exclusive; they are layered on top of each other. To give only one example, people might look at the amount of work weighted by the status of the journal where the work is published, use citation counts as a threshold measure to decide what or who to read and then carefully read the most cited pieces in order to get a sense of a scholar's quality of mind. Note however, that the last step is the most time-intensive, which explains why people are likely to use measurable qualities as a shortcut or heuristic. This is especially so in a world with ever more demands on scholars' time and attention.

In fact, we shouldn't forget that earlier models of merit based primarily on connoisseurship also used shortcuts and heuristics-- in the form of reputation, word of mouth, and gossip. People have always used shortcuts and heuristics for judging quality; they are simply using newer ones in addition to the older ones.

In short, people's judgments of scholarly merit have moved toward a complicated combination of connoisseurship, attention to production, and attention to citations, downloads, links, etc. People hope that these different forms of judgment will be mutually supporting, and converge in many cases. Whether or not that is the case, the layering of these different models of authority by itself subtly affects scholarly judgments, not only of influence but also of merit.
Posted 1:14 PM by JB [link]