Balkinization  

Tuesday, October 28, 2008

Spreadsheets vs. Mean Streets

Ian Ayres

Crosspost from Freakonomics:

Last Monday, the A.C.L.U. of Southern California released a report that Jonathan Borowsky and I wrote that analyzes more than 700,000 pedestrian and motor-vehicle stops conducted by the Los Angeles Police Department over a 12-month period from July 2003 to June 2004.


As I said in a Los Angeles Times op-ed:



We found persistent and statistically significant racial disparities in policing that raise grave concerns that African-Americans and Latinos in Los Angeles are, as we put it in the report, “over-stopped, over-frisked, over-searched, and over-arrested.” After controlling for violent crime rates and property crime rates in specific neighborhoods, as well as a host of other variables, we found the following:


For every 10,000 residents, about 3,400 more black people are stopped than whites, and 360 more Latinos are stopped than whites. Stopped blacks are 127 percent more likely to be frisked, and stopped Latinos are 43 percent more likely to be frisked than stopped whites.





INSERT DESCRIPTION



Stopped blacks are 76 percent more likely to be searched, and stopped Latinos are 16 percent more likely to be searched than stopped whites.


Stopped blacks are 29 percent more likely to be arrested, and stopped Latinos are 32 percent more likely to be arrested than stopped whites.


The evidence regarding over-stopping, over-frisking, and over-searching is particularly compelling. Although stopped blacks were 127 percent more likely to be frisked than stopped whites, they were 42.3 percent less likely to be found with a weapon after they were frisked, 25 percent less likely to be found with drugs, and 33 percent less likely to be found with other contraband.



Frisks and searches of blacks and Latinos were systematically less productive than those of whites:


INSERT DESCRIPTION


Our report has helped unify labor and management — as both the union president and the Chief of Police quickly rejected the findings of the study. Union President Tim Sands dismissed our report as “an exercise that might work on a spreadsheet at Yale, but doesn’t work on the streets of Los Angeles.”


This is an elegant (anti-elite) turn of phrase; but they’re fighting words to people like Steve Levitt, who have used statistical packages at a distance to uncover unexpected truths about our criminal justice system. I respond to some of their particular criticisms in my op-ed:



Police Chief William J. Bratton quickly rejected these findings, primarily because the study used data that were more than four years old. This is a fair point. But we had no other choice: The department has not released the more recent stop data that it has been collecting, nor has it analyzed the more recent data to test for racial disparities. If Bratton is truly confident that unjustified racial disparities are a thing of the past, he should be able to show the change in the current data. I would be happy to organize a group of respected academics to help analyze it.


Bratton also asserted that the report was flawed because we failed to control for the race of both officers involved in the stop. On this point, Bratton is simply wrong about how to conduct a statistical analysis. When testing for unjustified racial disparities in who is stopped by the police in cars and on the street, it’s inappropriate to control for the race of either of the officers. The likelihood of being stopped, frisked, or arrested shouldn’t turn on whether a black, Latino, or white officer was involved.


As an ancillary test (after we’d calculated the general disparities), we did look at the officers involved, and we found that the racial disparities in the likelihood of arrest were substantially lower when at least one of the stopping officers was the same race as the suspect.


For example, we found that the black arrest disparity was 9 percentage points lower when at least one of the stopping officers was black. Bratton should be troubled that there is less disparity when the officer is the same race as the person stopped, as that result adds credibility to the idea that the disparities in different-race interactions may be because of racial bias.


The president of the Los Angeles Police Protective League, Tim Sands, even more harshly rejected the results of our report. Sands said I appeared to start with my conclusions and then “worked data to fit.” This is a vague charge, but one way to respond to the concern is with transparency. I’ve posted the data I used in the report and the associated statistical files to the internet so that other academics can easily double-check the report’s analysis.



Sands has argued that the results are not valid because officers often don’t know the race of the suspect when they decide to pull over a car. That may or may not be true. But our study looked not just at motor vehicle stops, but at pedestrian stops as well, which also showed racial disparities. We also found that, once people were stopped, officers were more likely to frisk, search, or arrest African-Americans and Latinos than whites. At the point of making these decisions, officers can certainly see the apparent race of the suspects.


It is particularly telling that neither Bratton nor Sands responded to the evidence that the frisks and searches of minorities systematically produced less evidence of crime than the frisks and searches of whites. It is implausible that higher frisk and search rates are justified by higher minority criminality, when these frisks and searches are substantially less likely to uncover weapons, drugs, or other types of contraband. Independent of racial disparity, it is a sign of ineffective policing to have officers engage in such a large number of fruitless searches.


Sands charges that I cannot use data to “prove what 9,700 individual officers are thinking when they make traffic stops.” But if he thinks that is what I tried to do, he seriously misreads the report. I never suggested that the data show what an officer might be thinking, and I was careful not to attribute the disparities to conscious discrimination on the part of individual officers.


What the report finds is that there are statistically significant racial disparities in a variety of police behaviors that are not explained by legitimate police concerns, such as the local crime rate — or, in the cases of frisks and searches, the likelihood of actually uncovering contraband.


My inability to probe the minds of officers does not make my results less important. The report shows that people of color in Los Angeles experience harsher treatment by police that doesn’t appear to be justified by any legitimate law-enforcement concerns. The L.A.P.D. can’t just deny that racism is involved and let the matter rest; it should take steps to address that inequality.



So what steps should be taken? My report makes a series of recommendations. But one of the simplest is my recommendation that:



[T]he department should require that all existing and new officers take the Implicit Association Test (IAT) and retain the results of these tests. The IAT is a free internet test which takes about 10 to 15 minutes to complete. It produces a measure of unconscious bias with regard to the attribute being tested. For example, the black/white IAT produces a measure of whether an individual has unconscious negative associations with photographs of African-Americans relative to photographs of whites.


Collecting information about the IAT [results] of individual officers would provide a powerful validation test of the estimated racial disparities reported in this report. If the individual officer disparities estimated [in the report] are positively correlated with the disparities estimated by the IAT, it would be very hard to accept that the policing disparities were justified by suspect or stop attributes.


The willingness of the department and the union to collect data on the possibility of individual officer bias will itself be a test of how serious they really are about responding to the issue.


By the way, the IAT is not just for the L.A.P.D.; anyone can take the test for free at Harvard’s IAT site. There is something powerful about learning whether you can keep yourself from treating photographs of African-Americans differently than photographs of European-Americans.




Comments:

People have been doing studies of racial profiling for many years. Are the data comparable enough over time to draw conclusions about whether things have gotten better, stayed the same, or even gotten worse?
 

We found persistent and statistically significant racial disparities in policing that raise grave concerns that African-Americans and Latinos in Los Angeles are, as we put it in the report, “over-stopped, over-frisked, over-searched, and over-arrested.” After controlling for violent crime rates and property crime rates in specific neighborhoods, as well as a host of other variables, we found the following:

For every 10,000 residents, about 3,400 more black people are stopped than whites, and 360 more Latinos are stopped than whites. Stopped blacks are 127 percent more likely to be frisked, and stopped Latinos are 43 percent more likely to be frisked than stopped whites.

Stopped blacks are 76 percent more likely to be searched, and stopped Latinos are 16 percent more likely to be searched than stopped whites.
Stopped blacks are 29 percent more likely to be arrested, and stopped Latinos are 32 percent more likely to be arrested than stopped whites.
The evidence regarding over-stopping, over-frisking, and over-searching is particularly compelling.

Although stopped blacks were 127 percent more likely to be frisked than stopped whites, they were 42.3 percent less likely to be found with a weapon after they were frisked, 25 percent less likely to be found with drugs, and 33 percent less likely to be found with other contraband.


Whether blacks and latinos in LA are "over-searched" is a matter of perspective.

Based on these figures, it appears that blacks and latinos as groups are substantially more likely to be carrying weapons, drugs and contraband and thus pose a substantially higher threat to the officer and the community. Thus, it is a rational, albeit potentially unlawful, response for police to frisk black and latinos at a higher rate than whites even if they come up empty t a higher rate in the former searches.. This is the reality to which the Chief was referring when he likened the report to “an exercise that might work on a spreadsheet at Yale, but doesn’t work on the streets of Los Angeles.”

This is the Trooper Vogel approach. Vogel was a Florida Highway Patrol trooper who worked up a rather accurate profile for drivers running drugs up I-95 until the courts told him he was acting unconstitutionally. I practiced as a prosecutor in Vogel's area are a few years back and have seen the raw video of the stops. The police are still using the same effective profiles, but simply disguise the stops far more effectively.

While race is often an element in these profiles, profiling is not per se racist. Black and latino officers use these unofficial profiles out of self preservation and to more effectively do their jobs.

Nor are these profiles "subconscious" as implied by the IAT test. The police know damn well what they are doing and laugh at these academic exercises as a waste of their time.
 

Here's another neat statistic:

In LA county so far this year, there have been 582 murders. 46 white, 189 black, 317 hispanic and 30 asian/other. The percentages are therefore 7.9% white, 32.4% black and 54.4% hispanic, 6.8% asian/other. The population of LA is 46.9% white, 11.2% black, 10.5% asian and 46.5% latino of any race (these statistics are of course always screwy because "hispanic" is not a "race" at all and you can be white black or asian and still be hispanic, but anyways)

So, here we have 11% of the population, black being responsible for 32.4% of the victims, and almost certainly, over 30% of the perpetrators (anyone who has spent any time around murderers knows that most victims are of the same race as their killers, and while I'm willing to be corrected on this, I believe the statistics are a white is more likely to be killed by a black than vice-versa, at least on percentage terms if not absolute terms, due to the difference in numbers). The statistics are likely the same for hispanics, although again, it is harder to tease out given the white/hispanic overlap.

So, using your same statistical methods, wouldn't it be just as valid to say that blacks are more murderous than whites (at least in LA?) just as you now make accusations of racist stops and searches by the police? Of course, I don't think black are more murderous, and think there are all kinds of reasons for the higher black murder rate in LA ranging from poverty, to broken families, to gang culture to take your pick. But the corollary to that is if you are the police, wouldn't you want to focus on those communities which have the highest and deadliest crime rate, in this case the black and latino communities? And wouldn't that focus necessarily involve more stops/searches/frisks in high risk populations (i.e. black and latino) than low-risk one (i.e. white/asian)? So what you see as racial animus I and the police see as focusing resources where the problems (or in this case, the bodies) are.
 

Again, crunching some numbers, I get LA's black murder victim rate as being 17.2 times that of the white victim rate, 4.4/10000 as opposed to .26/10000, and this is just through October 6. I'm sure if you massaged the data to the neighborhood level you'd get some much higher murder rates. So again, as the police, wouldn't that be where you'd want to focus your resources?
 

Scott's posts are tendentious, but they do raise an interesting issue: did the study control for the neighborhood of the stop?

The effect of the neighborhood can be argued either way, no doubt, but it would be useful to know.
 

The data are from LAPD?

Then what makes you think the situation isn't much worse than the data show? LAPD has a clear incentive to sanitize the data to the mere disparities you report.

The analysis isn't the problem, nor is it even very interesting, if the data are suspect.
 

Post a Comment

Older Posts
Newer Posts
Home