Balkinization  

Thursday, March 31, 2016

Accountable Algorithms

Guest Blogger

Joshua A. Kroll, Joanna Huey, Solon Barocas, Edward W. Felten, Joel R. Reidenberg, David G. Robinson, and Harlan Yu, Princeton University


For the “Unlocking the Black Box” Conference, April 2 at Yale Law School

Computers make decisions across a wide and growing spectrum of modern life; from consequential decisions such as counting votes, assigning visas, or approving credit to mundane decisions such as controlling the internal operation of cars, aircraft, and home appliances, automated decision making will only grow in importance.

However, as several recent news stories demonstrate, the governance of these systems is lacking, in large part because the approaches for governing human-mediated decision systems do not translate well to automated ones. This can be seen through many examples that have made news just in the past year: whether it is televisions that might be transmitting what you say in your living room to a foreign company; vehicle control computers programmed to detect emissions testing and reconfigure cars to defeat the test standards; or startups using machine learning to make loans to borrowers with nontraditional credit scoring standards and triggering discrimination concerns, the pattern is the same: software-mediated decisions today do not receive the same level of oversight as human-mediated decisions, nor is it easy to trust that they can be effectively overseen.

One approach to solving these problem is through transparency: if only we knew what software was running for any given decision and what data were fed to it, so the argument goes, we would understand whether the decision was made in a way which is socially, politically, or legally acceptable. However, on its own, transparency cannot solve many of these concerns.
Read more »

Labels:


Tuesday, March 15, 2016

Machine Learning as a Challenge to the Legal Profession (and vice versa)

Frank Pasquale

In an earlier post, I described our upcoming Yale ISP conference on algorithmic accountability in the professions. We've already featured some blog posts from speakers, and have more on the way.In my earlier post, I described the community of scholars engaged in the study of algorithmic processes of ordering and ranking information. Today, I want to give a sense of the intellectual stakes of our work.

By and large, the lawyers pressing for algorithmic accountability (#algacc, for short) want to see basic principles of due process, anti-discrimination, and transparency imposed on processes like search engine data processing, credit scoring, and terrorist threat assessments. This is fundamentally an effort to make technology accountable to legal values.

Meanwhile, a different, much more dominant movement has been focused on the converse: forcing attorneys to speed up their work with new technology. This legal technology (or #legaltech) movement is extraordinarily influential for several reasons. Many consumers are justifiably frustrated by the slow pace of ordinary legal transactions. Many investors believe that software can do the work of several types of attorneys at, say, one-tenth their current costs, that they can still charge one-half that cost, and make outsized returns.  (Far fewer acknowledge the role of this kind of legal process automation and acceleration in troubling denials of benefits or due process.)

At present, there is very little interaction between the #algacc and #legaltech communities. I have tried to begin that conversation with a review of a book co-authored by a leading #legaltech thinker, Richard Susskind, and his son Daniel. As I discuss there:

A persistent mistake undermines the The Future of the Professions. The authors conflate the professional role with the delivery of expertise. Thus they fail to seriously address two issues at the core of professional identity. First, there is some degree of self-governance among professionals. They primarily work with clients or patients, for example, and not for bosses or shareholders. Second, the main reason they enjoy this autonomy is because they must handle intractable conflicts of values that repeatedly require thoughtful discretion and negotiation. In isolation, these factors damage the Susskinds’ case; together, they prove fatal to it.
To start with the question of values: rarely, if ever, is a vocation simply a matter of conveying information. The duties of professionals do not end with an assessment of the relative likelihood of an intervention “working,” where “working” is defined by a clear, quantifiable metric. That’s most obvious in, say, elder care or education. A robot telling a shut-in elderly person, “Your friends and loved ones care for you” is not really a good substitute for visits. As for children in school, they require some guidance as to how to conduct themselves — as persons, not simply as absorbers and generators of messages. To think otherwise is to put society on a slippery Skinnerian slope to behaviorism.
There is no single measurement of success in the professions, and further complexities arise as soon as one lacks a single quantity or thing to be optimized. For example, attorneys, doctors, and teachers often face very difficult conflicts of values — between, say, zealous advocacy for a client, and acting as an officer of the court; or between extending a patient’s life, and assuring quality of life in the time remaining; or between attending to disruptive students, or simply ordering them out of the classroom to ensure others can learn better. We mark the importance of these decisions by insisting that a human be directly responsible for them. Routinized or robotized approaches do not respect the dignity of the client, the patient, and the student.

There is more in the review, particularly with respect to law and medicine as professions. But I hope the excerpt above makes my main point: professional values are just as important to the deployment of technology, as technology is to the further development of professionals' expertise and effectiveness.

Of course, more widespread deployment of technology can make professional work more difficult. But that does not reduce its importance--indeed, it may well increase it. A critical goal now for legal professionals (and skilled workers of all kinds) is to assure that their values, norms, and skills are reflected in deployments of software and predictive analytics. I hope our conference "Unlocking the Black Box" can showcase cutting edge research on how that may be done--and the unfortunate consequences when it is not.

Labels:


Tuesday, March 08, 2016

Automating Authority? Algorithmic Practices, Knowledge, and Journalistic Professionalism

Guest Blogger

Matt Carlson

For the Unlocking the Black Box conference -- April 2, 2016 at Yale Law School

It is either ironic or proving a point that my first step in writing a paper about algorithms was to consult the algorithms of Google Scholar. Nonetheless, serendipitous things happen when you take a deep dive into repositories of academic work. In this instance, my search turned up a computer science paper with a rather instrumental title, “A front-page news-selection algorithm based on topic modelling using raw text.” This was not at all surprising; searching for research on algorithms regularly results in a mishmash of material arrayed across a spectrum marked by sociological studies comprising what Tarleton Gillespie and Nick Seaver call “critical algorithm studies” on one end and hosts of technical studies focused on the construction and operation of algorithms on the other.

This particular article fell in the latter camp, but luckily I stopped to read it. I can’t speak to its technical value or procedures, but it was the opening paragraph that caught my attention:

The front page of a news aggregator, like Google News or Yahoo! News, is the showcase where readers expect to see significant news articles. With human-editor-based news aggregators, the burden of reading several news articles and selecting important ones is a challenging task. Editors may select worthless news unintentionally, or even according to their own points of view. As a result, intelligent algorithms that allow news aggregators to process news and select significant ones, need to be developed.

On the surface, this is a vague statement used to justify the subsequent development of an algorithm taking a unique approach to sorting stories into a finite list for the front-page of a hypothetical news site. It is hardly a full-blown argument, let alone a manifesto. But this paragraph is also pregnant with assumptions about what journalism is, how it works, and how algorithms can be introduced to make it work better. In this sense, it is an ideology, a way of abstracting the world and formulating a particular set of values that in turn drives concordant actions. And to the extent that it is expressed so unproblematically and definitively, it deserves a second look.

Read more »

Labels:

Privacy and Accountability in Black-Box Medicine

Guest Blogger

Roger Ford and Nicholson Price

For the Unlocking the Black Box conference -- April 2, 2016 at Yale Law School

Medicine is a notoriously unpredictable science. A treatment that provides a miraculous recovery for one patient may do nothing for the next. A new chemotherapy drug may extend patient lives by two years on average, but that average is made up of some patients who live many years longer and some patients whose lives are not extended at all, or even are shortened. And with new drugs costing more and more money, personalizing medicine is increasingly important, so that doctors can predict disease risk and choose treatments tailored for individual patients.

This unpredictability has a simple cause. The human body is extraordinarily complex, with endless genetic variations, biological pathways, protein expression patterns, metabolite concentrations, and exercise patterns (to name just a few of the dozens of variables) affecting each person differently. And only a few of these variables are well-understood by scientists. When a drug doesn’t work, then, or a patient develops a rare disease, it could be because of some genetic variation, or a particular metabolite concentration, or several of these things acting together in ways doctors may never understand.

Black-box medicine—the use of big data and sophisticated machine-learning techniques in opaque medical applications—could be the answer.  It takes significant time, money, and luck for scientists to discover the precise combination of variables that makes a drug work or not—if it can be discovered that way at all—but with enough data, a machine-learning algorithm could find a predictive correlation much more rapidly. Using datasets of genetic and health information, then, researchers can uncover previously unknown connections between patient characteristics, symptoms, and medical conditions. And these connections promise to yield new diagnostic tests and treatments and to enable individually tailored medical decisions.

Read more »

Labels:

An FDA For Algorithms

Guest Blogger

Andrew Tutt

For the Unlocking the Black Box conference -- April 2, 2016 at Yale Law School 

The views expressed in this essay are the author’s only and do not necessarily reflect the views of the Department of Justice or the Office of Legal Counsel. This post is adapted from a working paper on the same subject.

On February 14, 2016, a Google self-driving car struck a bus on a California street, apparently the first time one of Google’s self-driving vehicles has caused an accident. The bus failed to yield, the car failed to stop, there was a collision. Google’s engineers were unsure how much responsibility to put at the autonomous vehicle’s feet. The head of Google’s self-driving car project, Chris Urmson, said that although Google’s car bore some responsibility, fault for the accident was “not black and white.” In a written statement, Google called the collision “a classic example of the negotiation that’s a normal part of driving—we’re all trying to predict each other’s movements.” Google also wrote that “[w]e’ve now reviewed this incident (and thousands of variations on it) in our simulator in detail and made refinements to our software. From now on, our cars will more deeply understand that buses and other large vehicles are less likely to yield to us than other types of vehicles, and we hope to handle situations like this more gracefully in the future.”

Google may have tweaked its algorithm, and run thousands of simulations (and variants on the simulation)—but how can we be sure that Google’s vehicles are safe? How many thousands of miles, or tens of thousands of hours, should an autonomous vehicle algorithm log before it’s road-ready? How do we decide that we are confident enough that, when an autonomous vehicle algorithm does fail, it won’t fail catastrophically? The answers to those questions are all still being worked out.

My contribution to the Unlocking the Black Box conference is to suggest that the rise of increasingly complex semi-autonomous algorithms—like those that power Google’s self-driving cars—calls for developing a new specialist regulatory agency to regulate algorithmic safety.  An FDA for algorithms.

That might sound strange at first, but hear me out. The algorithms of the future will be similar to pharmaceutical drugs: The precise mechanisms by which they produce their benefits and harms will not be well understood, easy to predict, or easy to explain. They may work wonders, but exactly how they do it will likely remain opaque. To understand why will require a dive into the future of algorithms.
Read more »

Labels:


Older Posts

Home