Balkinization: The Program's Progress

The Program's Progress

Frank Pasquale

I've done a series of posts at the Health Care Blog on the unexpected consequences of data analysis in clinical settings. Not enough policymakers have recognized how pervasively predictive analytics can utilize data from one setting in another, unexpected one. As Scott Peppet argues, once you've "quantified yourself," it may not be easy to opt out of invasive uses of your digital doppelganger. We all too often have "delusions of control" about technology once it is introduced. But sooner or later, many key technologies end up disciplining us.

Situating these controversies in a broader analysis of social trends, Ira Basen's article in the weekend's Toronto Globe and Mail is an excellent survey of the many ways that we end up "programming our lives away:"*

Increasingly, algorithms are used to determine whether we can get access to credit, insurance and government services. They are posing a challenge to human decision-making in the arts. They are being used by prospective employers to decide if we should be hired. They can determine whether your online business will succeed or fail, and they have revolutionized the world of high finance.

And yet these algorithms remain a mystery to us, their inner workings protected by various intellectual property and trade-secrecy laws. Critics are beginning to wonder if we are surrendering too much human agency to the all-powerful gods of mathematics.

After discussing troubling uses of algorithms in employment, credit, finance, and other fields, Basen quotes some skeptical experts:

[Jaron Lanier's] book, You are Not a Gadget: A Manifesto, . . . calls for “a new digital humanism” to counteract the trend toward “cybernetic totalism.” Mr. Lanier urges readers not to succumb to an ideology being peddled by the gurus of Silicon Valley that seeks to devalue human creativity.

He believes that they are asking us to abandon our faith in ourselves and, instead, to put our trust “in the crowd, in the algorithms that remove the risks of creativity in ways too sophisticated for any mere person to understand.” They want us to believe, he concludes, “that the computer is evolving into a life form that can understand people better than people can understand themselves.”

Having questioned this trend in a few areas, I highly recommend the entire article. As Douglas Rushkoff puts it, we face the choice: "program or be programmed."

*In the interests of full disclosure: I will be on Basen's documentary, Engineering Search, which will air on CBC Radio 1 on Dec. 5.

X-Posted: Concurring Opinions.

Posted 10:59 AM by Frank Pasquale [link]