Search Neutrality as Disclosure and Auditing

Frank Pasquale

We had an excellent online symposium on Tim Wu's book The Master Switch at Concurring Opinions last week. I just wanted to respond to some concerns about the role of search engines raised by Jonathan Zittrain and Katherine Strandburg. For example, Strandburg observes:

Google . . . is held up as a paradigm of openness in The Master Switch, yet in many very important ways Google is anything but open. Google’s search technology is open primarily in the sense of access by searchers and “connection” by those who are the subjects of search. It is quite closed in the senses of both dissemination and follow-on innovation.

Google’s algorithms are patented and protected by trade secrecy and its databases of information about users and their searches (which are crowd-sourced entirely from users themselves) are also protected by secrecy. To call Google open is, for better or worse, implicitly to privilege certain kinds of openness. . . . As a general matter – and not just in the information arena – companies prefer their inputs to be open and their outputs to be closed.

Zittrain also believes Wu should take a position on whether search neutrality is as pressing an issue as net neutrality.

What is Search Neutrality?

Search neutrality is on the rise in Europe, and on the ropes in the US (or at least should be, according to James Grimmelmann). We barely have net neutrality here, and the tech press bridles at the thought of a sclerotic DC agency regulating god-like Googlers. I want to question its conventional wisdom, by proving how modest the "search neutrality" agenda is now, and how well it fits with classic ideals of neutrality in law.

There are many reasons to think that Google will continue to dominate the general purpose search field. Sure, searchers and advertisers can access a vibrant field of also-rans. But most users will always want a shot at Google for serious searching and advertising, just as a mobile internet connection is no substitute for a high bandwidth one for many important purposes.

Given these parallels, I've compared principles of broadband non-discrimination and search non-discrimination. But virtually every time the term “search neutrality” comes up in conversation, people tend to want to end the argument by saying “there is no one best way to order search results—editorial discretion is built into the process of ranking sites.” (See, for example, Clay Shirky's response to my position in this documentary.) To critics, a neutral search engine would have to perform the (impossible) task of ranking every site according to some Platonic ideal of merit.

But on my account of neutrality, a neutral search engine must merely avoid certain suspect behaviors, including:

1) Stealth marketing (secretly taking cash or other consideration in exchange for elevating the profile of sites in organic search results)

2) De-indexing without notice and explanation (removing legal, non-spam sites from the index after they have been included in the search engine’s corpus, and failing to give some explanation to the removed site as to why it was removed)**

I think my concept of neutrality is much closer to the way the term is normally used in political philosophy (where, say, a “neutral state” is one that does not favor any particular conception of the good, rather than one that accords exactly the right amount of respect to each conception of the good). Neutrality is a very broad term, and the obvious differences between the technical operation of physical infrastructure and search engines should not stop us from applying certain broad principles to each entity.

Neutral State, Neutral Search Engine

What do we mean when we talk about search neutrality? Opponents of net neutrality have called the term too vague, identifying "31 flavors" of neutrality to support any ideological commitments under the sun. But Michael Powell's "four freedoms" have proven relatively uncontroversial. I think a similar consensus will coalesce around stealth marketing and de-indexing. A brief review of how one insightful political philosopher conceptualized neutrality may help get us there.

In his "The Ideal of a Neutral State" (in this collection), Peter Jones helps us understand key controversies in conceptualizing neutrality.* His central insight is a recognition of the familiar, "ordinary language" sense in which neutrality is used. The "question of neutrality often only arises in a conflict; a commitment to neutrality indicates a willingness to help or hinder parties to an equal degree."

By examining a list of examples of such conflicts, Jones demonstrates that "being neutral" can take different forms in different contexts. For instance, a nation may have to refrain from helping or hindering either side in an international conflict if it is to remain neutral; a judge may have to ensure that there are strict guidelines for the presentation of evidence if he is to be seen as a neutral arbiter of a fair decisionmaking process. Based on examples like these, Jones argues that there are two senses of neutrality, negative and positive. Whereas negative neutrality can be achieved simply through inaction, positive neutrality "is a matter of establishing certain conditions of equality among individuals, conditions which 'neutralise' certain factors that might otherwise enable one individual to fare better than another."

Does a search engine have a duty of "positive neutrality?" No, but much of what it does amounts to a similar effort. If some scheming company starts "link farms" to make its sites more visible, it should be punished. It's gotten an unfair advantage. We can, in general, count on the search engine to promote its users' interests by detecting and deterring that kind of advantage-seeking.

But there are many other situations where a search engine's interests may coincide more with those of its best advertisers, rather than its users'. As Brin & Page stated, "we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers." We've been in this situation repeatedly in the communications context, and the neutral state has come up with a solution: require both conduits and content providers to disclose whether they are raising the profile of those who pay them. Stealth marketing is unfair to consumers and to competitors of the stealth marketer. The state realizes that money already confers an enormous advantage in the battle for mindshare, and requires, at the very least, that such advantage be disclosed when it is bought. It is right to neutralize certain factors (such as sub rosa payments) that might enable one source to fare better than another.

Opaque Search Technology is a Widespread Problem

Search engines are only one of many intermediaries that use opaque search technology. Moreover, to the extent search engines become some magical category of zero-regulation, expect other entities to incorporate search technology to obtain the same advantages that search engines have. The regulatory arbitrage game is just too easy to play.

Given these looming problems, some entity should be able to audit the systems used by any dominant intermediary to find out if stealth marketing and de-indexing without notice is happening. I don't necessarily care if the entity is public or private---we just need some group to be able to "look under the hood." And if anyone says "it's just too complex to explain or understand," consider the kind of black box future that position portends.

Looking ahead, I don’t actually think this disclosure remedy will do much to change consumer or advertiser behavior. Dominant search engines, online retailers, device makers, and social networks have enormous advantages over rivals now, and as Tim Wu has shown, a "Cycle" of early cutthroat competition has repeatedly congealed into oligopoly in the communications and media fields. But even ex post disclosure will at least allow future historians, academics, and policymakers to understand how our information environment was shaped.

In the long term, we’ll probably need to have some publicly funded alternative to dominant internet intermediaries, just as we have a Postal Service to serve those who can’t afford FedEx, or Medicare to serve those who are too old or disabled to get reasonably priced private insurance, or arts subsidies to help forms of expression that the market will never underwrite. Disclosure helps us understand how urgent the need is for an alternative.

*Peter Jones, “The Ideal of the Neutral State” in Robert E. Goodin and Andrew Reeve (eds.), Liberal Neutrality (Routledge, 1989).

**My work in the article "Asterisk Revisited" has also led me to believe that a trademark holder should appear somewhere on the top results page when its trademark, without more, is a search query. But I don't believe current law requires that, and I don't want to clutter this post with what would certainly be a complex discussion of trademark policy.

X-Posted: Concurring Opinions.
Posted 11:16 AM by Frank Pasquale [link]