Can Crunching Help You Prepare for a Colbert Interview?

Ian Ayres

Greetings from London where Super Crunchers has come out under the alternative subtitle: "How Anything Can Be Predicted." There are at least two meanings here (neither which I particularly like). For example, a Balkin-like wag could say that statistical analysis is so easily manipulated than it can predict anything you want it to. Or it can be taken to mean that Super Crunching can usefully produce predictions about every and any process.

Crunching numbers can’t help you predict everything, but it can uncover the hidden causes behind a lot more things than you might imagine.

There’s almost an iron law that people have trouble imagining how number crunching could help them make decisions. People tend to think that whatever they do is too specialized, too unquantifiable for computers to help? It’s easier for us to imagine how it could help someone else than ourselves.

Stepping back, number crunching is most likely to help when there are lots of examples from the past and when the thing that you want to predict is a function of several causal factors. It turns out that humans are pretty bad at putting the proper weights on multiple causes.

For fun, I tried to see if number crunching could help predict what kinds of questions Steven Colbert asks when he interviews people at the end of his show. So my 12-year-old coauthor Henry and I coded up more than 250 questions. Let me be clear. This is not Super Crunching. This dataset is miniscule compared to the terabytes of data that are often mined these days. And I only controlled for a handful of variables. This exercise is more a provocation than scholarship.

Nonetheless, here’s what I found.

On average, Colbert asked hostile questions (e.g., “Isn’t it true that . . .”) 31% of the time. His questions were self-referential 39% of the time. His questions took a premise to a logical extreme a whopping 56% of the time. And his questions were grammatically framed as statements about 39% of the time.

But what’s more interesting is that these percentages varied depending on the type of guest.

When the guest was identifiably liberal, Colbert was . . . .

20.5% more likely to ask a hostile question (p. < .01)
15.7% more likely to ask a self referential question (p. < .05).

[There were no statistically significant effects for identifiable “conservative” guests.]

When the guest was a put forth as an “expert,” Colbert was . . .

14.9% less likely to ask a hostile question (p. < .05); and
15.0% less likely to frame the question as a statement (p. < .05).

It’s easy (after the fact, once you've seen the results) to say to yourself, I could have predicted he would ask liberals more hostile questions. So to keep you honest, you can test your ability to predict before seeing the statistical results.

Post a comment and tell me if you think there are any differences in how he treats men vs. women or in how he treats famous vs. non-famous guests? I’ll post the results later on.
Posted 1:04 AM by Ian Ayres [link]