Balkinization: Crisis in the Archives

Crisis in the Archives

David Pozen

Critics of the executive branch’s information control practices tend to focus on the here and now. They argue that overclassification of national security–related documents undermines democratic self-rule. They inveigh against delays and denials in the implementation of the Freedom of Information Act. They condemn regulations that “incorporate by reference” materials developed by industry groups. They worry about the growing use of black box algorithms, criminal leak investigations, and secret waivers for former lobbyists turned political appointees. All of these critiques raise important issues, even if they sometimes understate the transparency that exists—U.S. administrative agencies “are some of the most extensively monitored government actors in the world”—or overstate the benefits of sunlight.

One of the executive’s most worrisome information control practices has received relatively little attention, perhaps because it requires taking a longer view. Over the last several decades, as Matthew Connelly explains in a new essay on “State Secrecy, Archival Negligence, and the End of History as We Know It,”[*] our national archives have been quietly falling apart. FOIA backlogs look like a Starbucks queue compared to the 700,000 cubic feet of records at the National Archives and Records Administration’s research facility in Maryland that were unprocessed as of 2013. The Public Interest Declassification Board recently estimated that it would take a year’s work by two million declassifiers to review the amount of data that a single intelligence agency now produces in eighteen months.

The U.S. government’s entire system for organizing, conserving, and revealing the record of its activities, Connelly maintains, is on the verge of collapse; a “digital dark age” awaits us on the other side. His is less a story about excessive information control than a story about the absence of information control. Archivists simply have not been able to cope with the flood they face. The negative consequences extend far beyond the professional study of history, as Democrats learned last month when NARA announced that it was incapable of reviewing and releasing all of Brett Kavanaugh’s papers before the Senate votes on his nomination to the Supreme Court.

How did this crisis in the archives develop, and what might be done to mitigate it? Woefully inadequate appropriations and “dubious management decisions” bear some of the blame, according to Connelly. When the ratio of spending on the classification and protection of national security secrets to spending on their declassification exceeds 99 to 1, the historical record is bound to suffer. But the deeper cause of the crisis, Connelly suggests, lies in the exponential growth of government records, particularly electronic records. In a world where the State Department generates two billion emails each year—all of which need to be screened for sensitive personal and policy details prior to disclosure through any official process—the traditional tools of archiving cannot possibly keep up.

Maybe the tools ought to be updated for the age of “big data,” then. Connelly has collaborated extensively with data scientists on the problems he highlights, and he argues that sophisticated use of computational methods, from topic modeling to traffic analysis to predictive coding, could go a long way toward rationalizing records management and accelerating declassification. If these techniques were to be combined with bigger budgets for archivists and greater will to curb classification, NARA might one day make good on its aspiration to ensure “continuing access to the essential documentation of the rights of American citizens and the actions of their Government.” There is something intuitively appealing about this vision: Digital technologies got us into this mess, and now they ought to help get us out of it. Connelly’s diagnosis of information overload and political neglect is so stark, however, that one wonders whether any such reforms will prove adequate to the challenge.

Three response pieces recast this challenge in a somewhat different light. The Archivist of the United States, David Ferriero, emphasizes steps NARA is taking to digitize its holdings, enhance public access to them, and enforce government recordkeeping requirements. Ferriero does not dispute that “the country would be well served” by greater funding for the agency he leads, but he suggests that progress is being made even within severe budgetary constraints.

Elizabeth Goitein largely endorses Connelly’s reform proposals but urges that they be pushed further in the area of national security information. Drawing on extensive research and advocacy she has done as co-director of the Brennan Center for Justice’s Liberty and National Security Program, Goitein offers a suite of specific recommendations, from tightening the substantive criteria for classification to requiring federal agencies to spend certain amounts on declassification to subjecting officials who engage in serious overclassification to mandatory penalties.

Finally, Kirsten Weld raises critical questions about Connelly’s characterization of the problem and urges that his reform proposals be pushed much further. Weld points out that the records maintained by NARA represent just a “slice” of U.S. history, albeit an important one, and that the government’s management of that slice has always been bound up with larger political struggles. The true source of the crisis at NARA, Weld submits, is not the rise of electronic records or the politicization of transparency but “the dismantling of the postwar welfare state and the concomitant ascendance of neoliberal governance.” To address the crisis, accordingly, technical fixes are bound to be insufficient. Nothing short of “a sea change in the federal government’s priorities” and “a massive reinvestment in the public sphere” will do.

A crisis in the national archives, all of the authors agree, is a crisis in American democracy. It is certainly not the only one we face, and it may not be the most acute, but preserving a record of our collective history arguably has a kind of epistemic priority. As we fight for our democratic future, these essays remind us to fight for the institutions that help us understand how we arrived at the perilous present.

[*] Connelly’s paper is being published, along with three response pieces, as the sixth installment in a series I am editing for the Knight First Amendment Institute at Columbia University.

Posted 1:17 PM by David Pozen [link]