The BOGSAT anxiety and the Rule of Five

Ian Ayres

In modern military jargon, “to bogsat a problem” means to informally discuss and kick around possible solutions. The term is an acronym for “bunch of guys sitting around a table.”

But many people feel an anxiety when its just guys sitting around any official table – particularly as the number of guys sitting at the table becomes large.

I recently felt this anxiety when I was on a committee that was charged with making a public appearance. The full committee in fact included more than one woman but on the day of our appearance only six males were in attendance.

It somehow felt wrong to me (and I bet to some of the people who saw us) that it was an all male committee.

The anxiety I felt is related to a natural cognitive perception of statistical significance. Statisticians tend to call an estimate statistically different from a null hypothesis if there is less than a 5% chance that the estimate would have occurred by chance if the null hypothesis were in fact true.

Nobody feels anxious if a two-person committee is all male because this could happen by chance. But if a committee is composed of fifty men, we should wonder whether women had a fair chance.

So what is the magic number between two and fifty where our anxiety really kicks in.

The binomial theorem suggests a Rule of 5. If a coin comes up 4 times in a row tails, statisticians can’t reject the hypothesis that it is still a fair coin (because 6.3% chance of that a fair coin would produce this result). But 5 tails in a row is too unlikely – only a 3.1% chance. If 5 tales come up in a row, we reject the hypothesis that it was a fair coin (at the standard 5% level of statistical significance).

The binomial distribution lets us exactly calculate the likelihood of an observed outcome given an assumed underlying probability. (It’s easy to calculate binomial distribution probabilities – just use the =binomdist() function in Excel.) With regard to the sexual composition of committees a natural normative focal point is a 50% probability of females.

Of course, there may be contexts where we implicitly should expect a higher or lower probability. But I predict in many settings that 50% probability has a normative pull.

A testable hypothesis of my theory is that, for example, all male panels with 5 or more participants are more likely to be challenged by the audience than all male panels with 4 or few participants.

Heather Gerken in a recent Harvard Law Review article on 2nd order diversity uses the binomial theorem to show that randomized fair draws of jury panels will by chance produce some that are disproportionately male or female (7.3% of the time there will be 3 or fewer men). She forever changed my thinking by showing that there might be (2nd order diversity) values to have a fraction of juries with disproportionately male and disproportionately female compositions.

But having just lived through a disproportionate composition, the pulls of first-order diversity are still strong. Indeed, I’m was so troubled about my participation that I’m attracted to the idea of dropping out of groups that fail the rule of 5 test. If a fifth man drops out, it stops projecting a statistically significant disparity. 4 men are not statistically significant. But non-random engineering of this type might back fire and obscure the underlying problem. As in other contexts, it is an empirical question whether voice or exit will be more effective. Sometimes noisy exit is the best of both worlds.

The bionomial distribution has other ways of informing our civil rights intuitions. In the 2000 census, white non-Hispanics constituted about 70% of the population. The binomial theorem suggests a racial “Rule of 9.” If a group of 9 or more people is comprised of solely of white non-Hispanics , we can immediately reject at the 5% level the notion that the group was randomly chosen from the general population.

Of course, having a single woman or a single minority member does not end the inquiry. The bionmial distribution can give the exact probability that a group of size N would have 1 or fewer women or 2 or fewer women. Indeed, if we take 50% to be the underlying normative null hypothesis, the potential replacement of O’Connor with Alito is that the Supreme Court would again slip into statistical non-compliance. You see the probability that 2 or fewer women would be drawn out of 9 trials is 9%. While the probability that 1 or fewer women would be drawn out of 9 trials 2%.
Posted 10:02 AM by Ian Ayres [link]