Balkinization: The Interpretive Poverty of Data

The Interpretive Poverty of Data

Guest Blogger

Stanley Fish

This brief essay was delivered as a response to a paper co-written by Justice Thomas R. Lee and James Phillips, Data Driven Originalism, at the Originalism Works-in-Progress Conference held at the University of San Diego on February 16-17, 2018. The desire to generate human meaning by eliminating from the patterns that convey it all traces of the human is at once perennial and doomed to be ever unfulfilled.

Back in the seventies there was a fast-growing sub-discipline that promised to marry linguistics and literary criticism in a way that would provide an objective basis for the interpretation of texts. It was called stylistics and I am pleased to say that I pretty much killed it by writing two essays, “What is Stylistics and Why Are They Saying Such Terrible Things About It?” and “What Is Stylistics and Why Are They Saying Such Terrible Things About it? Part Two”. In those essays I said that Stylistics failed in two respects. More often than not, the mountainous machinery it usually cranks up labors to produce something less even than a mouse; you wade through a whole lot of charts, distribution patterns, selection patterns, contiguity patterns and find waiting for you at the other end something that would have been obvious from the get-go to a ten year old. And, on the other hand, if at the end of the whole business there is an interpretive insight that seems novel and arresting , its relationship to the operations of the analytical machine is entirely arbitrary. Either you do all that counting and sorting and come up with a pompously inflated version of what you had at the beginning, or you do all that counting and sorting and then attach to the resulting data an interpretation it does not generate; to be sure, the interpretation may be one the data can be made to support, but only because the data , just sitting there in all its empty bulk, can be made to support anything.

Consider, for example the case of Louis Milic, a computational stylistician, who studied the relative frequency of initial determiners and initial connectives in the sentences of Swift, Macaulay, Addison, Gibbon and Samuel Johnson. He discovered that Swift, to a much greater degree than the other authors, began his sentences with connectives and he concluded that “The low frequency of initial determiners, taken together with the high frequency of initial connectives, makes [Swift] a writer who likes transitions and makes much of connectives”. That’s the banal fruit of the analysis. The arbitrary fruit comes a bit latter when Milic notes that it was Swift’s habit (as it is in fact mine) to extend the length of his sentences by producing a series of appositional phrases that threatens never to end. His conclusion? Swift’s “use of series argues a fertile and well stocked mind”. Why not say that Swift’s use of series argues an empirical rather than an abstract cast of mind , or that Swift’s use of series argues an anal-retentive personality or that Swift’s use of series argues an unwillingness to let go of a thought. These and a thousand other interpretive conjectures will fit the data Milic assembles, but there is no genuinely motivated route from the data to any one of them. As a statistical output, Swift’s use of series has the status of an ink-blot in a Rorschach test. The researcher/analyst asks, what might that mean , do you think?, and the answers are immediate and infinite.

Behind Milic’s efforts is the assumption, difficult to dislodge no matter how many times it has failed to cash out, that banks of data , especially data compiled disinterestedly, can yield interpretive conclusions; and the further assumption is that the conclusions thus yielded will be more objective, because less impressionistic, than the conclusions reached by a single interpreter who , because he or she is a finite, limited creature, can only survey a finite and limited number of texts and will therefore always be working with an inadequate and distorting sample. This is an important claim, for after all why go to all the bother of getting the machine going if there’s no payoff? The payoff has to be some improvement in interpretive methodology, an improvement that responds to the usual fears of interpretation’s being biased, partial, manipulative, and willful. Stefan Th. Gries , a leader in the corpus linguistics movement, puts the matter forthrightly when he says that the “assumption underlying most corpus-based analysis is that formal differences reflect or correspond to functional differences” and adds that “ functional difference is understood …as anything—be it semantic or discourse pragmatic—that is intended to perform a particular communicative function”. In short, it should be possible—and this is the only reason someone other than a descriptive linguist would be interested in corpus linguistics at all-- to map formal differences onto content differences, to correlate features of a text described independently of any interpretive hypothesis with a particular interpretation.

It isn’t possible and it never will be.

What this means is that while Thomas Lee and James Phillips’s paper might be interesting for several reasons, one of those reasons won’t be that its methodology will help us to determine the meaning of legal texts. But before I explain and defend that large assertion, let me briefly note some of the problems I have with the paper in general. First, Lee and Phillips fail to take the measure of the intentionalism they casually dismiss and their failure takes the form of a misrepresentation of the intentionalist thesis. Lee and Phillips rehearse the now standard story that intentionalism had been discredited in favor of the doctrine of ordinary meaning or Original Public Meaning. They explain that “Original Public Meaning originalists turn the focus away from framers intentions and toward the public’s understanding of the text.” But this statement assumes what intentionalists would dispute: that there is a text to be “focused on” independently of the assignment or assumption of intention. The entire Original Public Meaning project, and therefore the corpus linguistics project, depends on the possibility of identifying the text as a text (rather than some abstract scribbles) before any inquiry into the intention informing it, and that is what intentionalists say cannot be done. That is, for example, what Larry Alexander says in many places and especially in the essay Lee and Philips cite, Is that English You’re Speaking: Why Intention-free Interpretation is An Impossibility. The subtitle of the paper might be rewritten to read, Why Intention-Free Identification of Something as a Text Isn’t Possible; for in this and other essays, Alexander, along with other intentionalists, argues not simply that you can’t interpret the text without recourse to a governing intention, but there is no text until a specification of an intention is in place.

In a subsection entitled, Texts Cannot Declare that they are Texts, Alexander and his co-author, Emily Sherwin, declare that “one cannot look at the marks on a page and understand those marks to be a text (i.e. a meaningful writing) without assuming that an author made those marks intending to convey a meaning by them.” That is, absent an intention, either assumed or specified, what you will have are black marks and white spaces. It is only when one thinks that someone meant something by these marks that one searches them for patterns conveying a message. Just as when smoke appears on a distant horizon, its forms can only become the object of interpretation when it is assumed that the serial sequence of puffs is designed. Once that assumption is in place, then you can ask, what does the sequence of puffs mean (it will only be a sequence rather than something random if that condition is met); but without that assumption in place, the question of meaning does not and cannot arise. The same argument holds for individual words. If I see these markings – HELP – on a sidewalk and wonder what they mean, my wonder and my effort at interpretation will both cease if I discover that the marks were made by dirty rain water dripping from the gutters of an adjacent building (unless you think that buildings have intentions and can signal distress at having been allowed to deteriorate). It is not simply that seeking intention is the best way to interpret a text; rather it is that if you don’t seek intention (or assume intention) you will not have a text. Now, one might find Alexander’s argument persuasive or unpersuasive, but it cannot be characterized as Lee and Phillips characterize it when they say that it questions “our ability to resolve ambiguities in the meaning of a given text without resort to authorial intention”. But “given text” – the text just sitting there detached from authorial intention or institutional history, the text that might or might not be ambiguous -- is what Alexander denies the availability of. It is the intention-free existence of the text, not its being ambiguous or unambiguous, that Alexander calls into question.

Why do Lee and Phillips misrepresent the intentionalist position? Because that position, taken seriously, means that textualism in general and corpus linguistics in particular are nonstarters if the game being played is the game of interpretation. Lee and Phillips report, quite accurately, that the majority of citizens in the originalist’s universe are now Original Public Meaning textualists. This undoubted fact is a statistical measure of what the majority of those people in the field believe; it has nothing to do with whether that majority is correct or whether its methodology is coherent or whether it is the methodology interpreters are actually following. Indeed, intentionalists characteristically say (of Justice Scalia and others) that textualism is a method that cannot be followed and that self-identified anti-intentionalists enact intentionalism even when they rhetorically repudiate it. Lee and Phillips present an example when they describe Justice Thomas’s dissent in Kelo v. Town of New London as an instance of a jurist relying on original communicative content -- that is, a jurist being a textualist. However, they are honest enough to note that Thomas’s cites “early state practices” that he claims “shed light on the ... meaning of the ... words contained in the Public Use Clause.” What kind of light is that? It’s certainly not light shed by an examination of the words; it is the light shed by an understanding of what state legislatures usually had in mind when they exercised their eminent domain power. The sequence of reasoning is as follows: here are some words in the Public Use Clause; in order to determine what they mean let’s look at the way legislatures of the period usually thought about these issues, what they were concerned to promote or prevent. That’s intentionalism, pure, simple, and unavoidable. Thomas is not slipping here from Original Public Meaning textualism to a suspicious intentionalism; he is doing what everyone must do, whether the text is a clause in the Constitution, or a grocery list, or a lyric poem, or a word that appears on a sidewalk – trying to figure out what someone meant by these marks.

This is all to say that except in narrow and carefully defined circumstances (which I will describe later) Original Public Meaning or ordinary meaning doesn’t get you anywhere. I am not denying either that Original Public Meaning is a thing or that specifying it, by whatever means, might be occasionally and limitedly helpful. I am only saying that it is not a magic key to interpretation and indeed is not a key at all.

What then is it? It is in fact a construct, a put together set of mark – meaning correlations that are statistically predominate in a particular population of speakers at a particular time. But, these correlations, in and of themselves, are of no more interpretive help than the raw data of frequencies and distributions. You can know the shape of public meaning, just as you can know that Swift often begins his sentences with connectives and still be unable to proceed to any legitimate interpretive conclusion. Original public meaning is interpretively inert; the mere fact of it doesn’t get you anywhere, just as the mere fact that Swift makes extensive use of appositional series doesn’t get you anywhere. In both contexts what is missing and required is a determination of what an agent intended to signify by those patterns. Let’s say that in the lexicon of Original Public Meaning a certain word is associated with a certain meaning; but having specified that, nothing interpretive follows unless an audience knows that Original Public Meaning is the code the author is deploying. The phrase “Original Public Meaning” is deceptive because the word “meaning” suggests that meaning is what it delivers. But what it delivers is data; what it captures are some observed regularities, but those regularities are of no more interpretive significance than randomly occurring puffs of smoke. It is only if you know that the author has tied himself to the code of Ordinary Public Meaning that knowledge of that code would be interpretively relevant.

But how do you know that? The question is urgent because as Alexander often says, the text won’t tell you what language it is written in. That knowledge, in the absence of which interpretation cannot begin, must come from the outside, from some persuasive indication—non-textual—of what the author had in mind, of what he or she or they intended. Data analysis might be able to tell you whether x is or was the ordinary meaning of y, but you’d still be faced with the question was it ordinary meaning the writer was deploying. The bottom line is that while corpus linguistics can help specify what the Ordinary Public Meaning is or was, it cannot help you take the next step, the step of interpreting; for that you need the intentionalism corpus linguistics dismisses. If it is somehow determined that the speaker/writer intended to deploy the code of Original Public Meaning, Original Public Meaning can help guide interpretation. If that intention is not in place, the fact that there is something called Original Public Meaning -- again, I stipulate to that fact -- is no more interpretively interesting than the fact that the speaker/writer is six-feet tall or the fact that he or she likes coffee ice-cream. Lee and Phillips say that “by tabulating the relative frequencies of different senses of a word or phrase within a corpus, a linguist can . . . discern the more common sense of a given term in a given linguistic context.” But there is nothing a linguist can do with what he has discerned unless the intention of the author to tie himself to the common sense of a word has been established by nonlinguistic means. Original Public Meaning is a resource, not a constraint; it is an option speakers and writers are not required to choose; the constraints come in, if they do, when the intentionalist question—Is that Original Public Meaning you’re speaking?—has been answered. In and of itself corpus linguistics is an interpretive dead-end.

Let’s test this out by considering Lee and Phillips’s examples.

Lee and Phillips’s first example is the phrase “domestic violence” as it appears in Article IV of the Constitution where the government is assigned the task and duty of protecting the republic from foreign invasion and “domestic violence.” The corpus linguistics question is how do you know the “domestic violence” here means violence committed by militia men like Timothy McVeigh rather than the violence committed by one spouse or partner against the other. The obvious answer is that anyone who has gone beyond the sixth grade knows that; but that answer will not be accepted by corpus linguists because it does not have any statistical or numerical backup. So, Lee and Phillips proceed to their data mining operation and, after a lot of work, say triumphantly that their method has “confirmed the intuition” that they and we had in the first place. “Our data show that domestic violence today is almost always used in reference to an assault on a member of a person’s household, but was a reference to an insurrection or rebellion in the late 18^th century.” Shades of saying that Swift’s preference for connective transitions over determiners means that he is an author who likes transitions.

In a second example utilizing eye boggling charts and tables, Lee and Phillips perform the same operation, in which something obvious is put through a machine that huffs and puffs and delivers exactly what it began with. It’s like the transporter device in Star Trek that breaks down your molecules and then reassembles them in the exactly the same form on the other side. (The protagonist of the movie “The Fly” was not so fortunate.) In this second case, the object of analytical attention is the word “commerce” as it appears in article 2 of the constitution, where Congress is given the power to “regulate commerce.” The question is, what does “commerce” mean? You might think that commerce means trade, although, of course, “trade” itself can be understood in both narrow and expansive ways. Lee and Phillips also think that commerce in this context means “trade” but they want to support that intuition by lexical analysis performed by a powerful search engine on a massive database. They proceed by thinking up some of the other things that “trade” might have meant or could possibly mean, and then they search the frequency with which the word “trade” is collocated with some of those other things. So they pair off or face off “trade” and “manufacturing” as two potential meanings of “commerce” and discover that “trade,” “shares six top 30 collocates with commerce” (don’t ask!), while “manufacturing” “shares just two.” Their conclusion: “the fact that commerce and trade have more overlap in their collocate network than “commerce” and “manufacturing” do is evidence that the “trade” sense is likely more common than the “manufacturing” sense.” Louis Milic lives. Lee and Phillips then say, in a truly terrifying statement, that “further research could be done,” for, “the value of a corpus is the ability to slice and dice context to get to the most relevant semantic context.” I would ask relevant to what? As far as I can tell, there is no semantic context in their analysis -- all there is is the tabulation of frequencies -- and no way of getting from their analysis to a semantic context.

It is important to understand what Lee and Phillips’s analyses prove and do not prove. They do prove (or strongly suggest) that in the code of ordinary meaning as it existed in the late eighteenth century “commerce” in the phrase “regulate commerce” meant “trade.” They do not prove that this is the meaning the framers chose, for it remains possible that they chose a meaning for “commerce” that departed from the statistical regularities Lee and Phillips uncover; chose a meaning, that is, which departs from Ordinary Public Meaning. They were certainly free to do so – it is always the speaker who nominates mark-meaning correlations – and evidence that they did or did not will not be provided by the data Lee and Phillips have so laboriously assembled. In short, the fact that there is an Original Public Meaning doesn’t compel a speaker/writer to use it and the determination of whether he did or not, cannot be made by consulting the text.

The third example cited by Lee and Phillips is the phrase “public use” as found in the Constitution’s Taking Clause. The question is, does “public use” mean use related to public government purposes (military, economic, education, etc.) or can it be interpreted, as the majority in Kelo does, to mean private uses of which the public government approves? Lee and Philips are on the side of Justice Thomas, who believes that the state errs in adopting the more expansive definition of public use, and I agree with them. Lee and Phillips dutifully fire up their machine again and find that “the direct sense that Justice Thomas argued for is much more common than the broader, indirect sense that the Kelo majority adopted.” But again, the fact that the direct sense is statistically predominate in the period says absolutely nothing about what sense the founders chose. Information about that must come from elsewhere, perhaps from an argument that foregrounds the founders concern to separate the public and the private, a concern in the light of which you might conclude that what they had in mind -- what they intended -- when they wrote the taking clause was the maintenance of the public/private distinction.

At one point in their analysis of the word “commerce”, Lee and Phillips wonder if perhaps “commerce” was the wrong linguistic unit to begin with. “Maybe we need to search for institutions of the regulation of commerce by the government.” They withdraw from their own suggestion because it “demands too much semantic context.” They realize that in order for their methodology to work, it must begin (as Chomsky tried years ago to begin) with no hostages to semantics at all; it must be purely formal, a matter of assembling the data independently of semantic contexts or suppositions and then proceeding from there to a genuinely motivated interpretation. In short, they know at some level that beginning with semantics will render their countings nugatory because the interpretive conclusion those countings supposedly generate have already been put in at the beginning. Here then, is the corpus linguist’s choice. Either stick to your method and have nowhere legitimately to go after the data has been assembled or compromise the method by grounding it in a semantic hypothesis and end up with a product that is nothing more than your original input tricked up in numbers.

Toward the end of their paper, Lee and Phillips display an attractive modesty. Although, they remain confident that “the use of corpus analysis” can be “a central element of the first stage of any originalist inquiry,” they do acknowledge that it is not quite clear what happens after this first stage. “We are less sure of the precise role [corpus analysis] should play.” In a recent paper, Stefan Th. Gries and Brian G. Slocum (two leaders in the field) display an even more robust modesty: “corpus analysis cannot by itself provide conclusive meanings to legal text” and again “corpus analysis can provide valuable insights about language usage but cannot by itself resolve normative issues.” That is it cannot direct us to what words and phrases mean. But I thought that was why we were going to all that trouble in the first place.

Can corpus analysis do anything? Yes it can. It can reveal, verbal patterns the naked eye, or the merely intuitive reader, would never see. And those patterns, often deeply embedded in an author’s stylistic presentation, can serve as a means of identifying him, of fingerprinting as it were. Larry Alexander and I have been talking about collaborating on an op-ed about current campus protests, a subject on which we have similarly Neanderthal views. Now let’s suppose Larry and I do manage to write this piece, and for some unfathomable reason an analyst wants to figure out which of the sentences in it were written by me and which were written by Larry. No doubt both Larry and I have verbal ticks of which we are largely unaware, ticks that have no particular meaning except the meaning that it is we who perform them. Corpus analysis would reveal those ticks and therefore would be able to distinguish Larry sentences from Stanley sentences. But that’s all it could do; as for contributing to a determination of what Larry or Stanley means by those sentences, forget about it.

To be sure, there are more serious uses to which a fingerprinting analysis might be put. It has been speculated that Shakespeare had a collaborator on some of his later plays. Did he, and if so who was it? There are poems whose authorship is in dispute between John Donne and Ben Jonson. Who really wrote them? This is a serious question in literary history and it is one corpus linguistics could help answer, and if it did there would be an interpretive pay off, for you would now approach the poem and receive its words and phrases within the knowledge that it was Donne or Jonson, each with his distinctive concerns, obsessions, and visions who had designed it. Real interpretation could then begin and corpus linguistics would have brought you to that moment, but it would itself have done no interpretive work; it would have just cleared the ground in a way that allowed interpretive work to proceed. That’s not nothing, but it isn’t a contribution to interpretation, just as a modern tennis racquet will help you to hit a winner -- it puts you in a position to do the job -- but doesn’t itself do the job. So there’s a use, and for some a significant one, of corpus linguistic analysis. And as I have already acknowledged corpus linguistic analysis can also help us to determine what the contours of Original Public Meaning or ordinary meaning were at a particular time. But beyond that, I don’t see its utility and therefore, I don’t see why the enormous expense and Casaubon-like erudition involved in learning how to do it is worth the candle.

Lee and Phillips and Gries and Slocum might respond by saying two things: (1) they might say that although the way we humans produce and receive meanings is quite different from the digitizing and retrieval processes of corpus linguistics, there is no reason why corpus linguistic methods cannot elucidate meanings produced by other means. But that’s like saying one could observe and transcribe the physical patterns of movement enacted by players in a football game without knowing that the moves you are describing are moves within that game, and have something interesting to say. If all you have is the catalog of physical actions performed in a sequence by 22 persons you can never get from that catalog to any statement of what those actions mean. Once you detach patterns from the intentional context in which they have significance, you can’t get the significance back; and (2) they might say as the stylisticians of the 1970s said and the digital humanists of the present day are saying, our discipline is new; don’t expect us to have all the answers at the beginning: but in time, trust us, this methodology will deliver what we seek, a legitimate relationship between the amassing of data and interpretive conclusions. To which I would respond by invoking one of my favorite lines from Shakespeare’s King Lear: “Never, never, never, never, never.”

Stanley Fish is the Davidson-Kahn Distinguished University Professor of Humanities and Law at Florida International University. You can reach him by e-mail at fishs at fiu.edu

Posted 12:11 PM by Guest Blogger [link]