Tuesday, April 22, 2008
Why I'd Stick With Yale Clerks-- Some Econometric Ruminations
well ..thanks for all that .. i don't pretend to understand all you've illuminated ..but it is an interesting read nonetheless ..
I agree with almost everything here, but have a few questions:
1. Why does the author refer to "econometric" issues rather than statistical issues? Nothing here is economometric, this is all Stats 101 (or 301, maybe).
2. There may be some differences of opinion as to whether a judge (like the Judge Newman whom the author mentions) who is pushing the law in a way the Supreme Court doesn't want to go is acting appropriately; I am not sure he is. This issue is beyond statistical analysis.
3. It doesn't really matter whether the paper holds up statistically: it's funny and good for annoying one's Yale Law grad colleagues, and surely that is more valuable than mere statistical truth.
Your opening comment about Justice Scalia's alleged misreading of current social science data may well be correct, though as a longtime opponent of the death penalty you are hardly a disinterested observer. But this is hardly unique. When judges, be they liberal or conservative, engage in pop social science, the result is nearly always bad.
One of the implications of this observations is that judges ought not adopt an interpretive methodology that requires them to think about social science at all. Maybe something like "read the Constitution and do what it says." Under such an approach, the question of whether the death penalty deters crime, while undoubtedly interesting, is wholly irrelevant to the legal issue.
You are also almost certainly right that the study in question doesn't demonstrate that judges ought not hire Yale grads as law clerks out of fear of reversal. Indeed, any judge who did so would be tacitly admitting that his or her law clerks had excessive and improper influence. But again, you miss the main point: one ought to refrain from hiring Yale Law School graduates as law clerks because the Yale Law School has been and continues to be a pernicious influence on both the law and legal education. Hiring law clerks from other institutions undermines (to some very small extent) that pernicious influence.
One of your suggested analyses makes me very nervous. You argued that better evidence in favor of a causal relationship could be produced if the change in the negatives (between time periods) was correlated with the change in percent Yale clerks. That is not an appropriate way to do it. Change scores should never be the predicted variable. Instead, the negatives from the first time period should be included as a covariate and the negatives from the second period should be the DV. (Using the change in Yalies as a predictor is fine.)
With that said, and agreeing that anything approximating a time-series analysis will always produce stronger evidence of a causal relationship than a simple cross-section, please not that time series are not immune to spurious relationships. Barring (as it were) an actual experiments, you have to include everything as covariates to establish causation.
I thank you for your detailed observations on my paper, and I am hopeful that I will be able to benefit from your wisdom as I further review your lengthy post, as well as the numerous other comments I have received.
I am somewhat puzzled by part of your comments that I have been able briefly to skim. You discuss why one might include some reference to whatever corresponds to area in your geographic area in the death penalty hypothetical (the judge or something else):
"clogit fits maximum likelihood models with a dichotomous dependent variable coded as 0/1.... Conditional logistic analysis differs from regular logistic regression in that the data are grouped and the likelihood is calculated relative to each group; i.e., a conditional likelihood is used. ...
"Economists and other social scientists fitting fixed-effects logit models have data that look exactly like the data biostatisticians and epidemiologists call k1i:k2i matched case-control data." Stata Base Reference Manual 275-76 (release 10).
That's what's reported in the results. Of course, with this kind of model, any variable that does not vary for the identity of the judge is collinear and cannot be included. So that would be why, for example, those models do not include dummy variables for the jurisdiction in which the judge is located or, of course, the judge identity itself.
Other commenters have requested that I simply report results showing dummy variables, one for each judge (other than one). I did that when I was at the office and posted on volokh.com the following:
There was an inquiry about the results from a more customary logit estimation using dummy variables for each judge (other than, of course, one of them). I can confirm that using a logit estimation with all the independent variables that are in model 1 in Table 4, but with 92 dummy variables added, one for each of 93 judges (2 judges' opinions being dropped because their opinions never have these adverse signals), has a parameter estimate [t-statistic] for the Yale Law School variable of: 1.64 [3.09], which is not materially different from the results reported in the paper.
I don't have the full regression results here at my home, though I would be pleased to post them were they of interest. The reason I did not report that kind of result in the paper is that the authorities I referenced indicated that would be improper. Rabe-Hesketh & Skrondal, Multilevel and Longitudinal Modeling Using Stata at 131 (2005), which has the following discussion concerning estimation of the impact of certain treatments of patients (corresponding to the judges in the paper):
"[I]t would be tempting to use fixed intercepts by including a dummy variable for each patient (and omitting the overall intercept). This would be analogous to the fixed-effects estimator of within-patient effects discussed for linear models in section 2.6.2."...
The authors, after describing a problem with doing that, says, "we can ... construct a likelihood that is conditional on the number of responses that take the value 1 (a sufficient statistic for the patient-specific intercept). ... In logistic regression, conditional maximum likelihood estimation is more involved and is known as conditional logistic regression. Importantly, conditional effects are estimated in conditional logistic regression ...."
So, I am having some difficulty harmonizing this discussion, which describes the estimation technique used, with your assertion, "The judges almost surely drive the error rate and the clerks show up as “significant” in the regression because there is small, but possibly significant relationship between the type of judge that gets reversed and those who will select (and be attractive to) Yale law students."
Research is part of an ongoing discussion among academics. I am happy to benefit from your wisdom in this regard. If you are interested in the results from estimating different models, I will be pleased to report to you the results of a couple of alternative estimations that you would find of interest using the variables used in the models in the paper (of course, in addition to the judge identity, as that is, in fact, part of the syntax of the Stata estimation of the models that are reported), as long as you fully specify them and do so in the syntax of Stata commands (adding variables not specified in the models currently reported could not be done in short order, depending on whether the underlying data, as it has been saved, includes it).
My e-mail address is available from Missou's web page.
This post is 18 paragraphs too long. Everything after "there is small, but possibly significant relationship between the type of judge that gets reversed and those who will select (and be attractive to) Yale law students" is superfluous.
I could not tell whether the closing assertion in the "In sum" paragraph was written with tounge-in-check. If not, I have some concers expressed here:
"Why empirical research is better at raising questions than answers --- some ruminations about ruminations about the Yale clerk study"
I am neither a lawyer nor an accomplished scholar. But as one who is beginning to stretch his empirical wings, is forced to spend hours a day picking apart empirical research for professors and an individual with a particular disdain for lawyer’s lack of statistical/economic knowledge I must objected to Professor Donohue’s numerous straw men and red herrings. While I will not take pointed issue with every point he has made, I feel I must set some records straight in case some poor law student reads this entry and garners his only impressions of empirical research from this blog. I will also throw aside the usual common pleasant tone I normally adopt for academic discussions and instead adopt the brutish, defensive demeanor of Professor Donohue.
First to the general objections Donohue has presented. I am appalled that any person who claims to be an empiricist (and he has many credits to his name) could ever say, “I am confident that a more suitable methodology than the one employed by Barondes would reveal that Yale Law clerks are extraordinarily capable and effective public servants.” That is the epitome of naivety. So he knows that including other controls and using panel regressions would clear his precious graduates? How can he? How can anyone know what the results of modified models and techniques would be?
In the spirit of his volley at Barondes, I would levy an argument that there is a fundamental flaw in Donohue’s own research (Donohue and Levitt, The Impact of Legalized Abortion on Crime, Quarterly Journal of Economics, 2001). Their research claims that legalized abortion altered the demographic makeup of the youth of the 70’s and 80’s which caused a reduction in crime in the 90’s. The problem is that they only showed the negative relationship between abortion and crime. No evidence was presented that the underlying demographics of our society changed, or if changed were related to abortion (and they indeed argued that it did not matter whether or not they could prove a demographic change, the regressions showed the link between abortion and crime and that was all that mattered). Even cursory analysis of demographic trends show increases in poor births and single mother births as percentages of total births (those they argued impact crime).
But no matter how strenuous my belief that their work is incomplete, does that mean I know that inclusion of demographic trends would invalidate their results? No. And no number of real, straw or red herring arguments can change the actual, tested result (especially since that is how Donohue and Levitt responded to critics, including but not limited to, “Further Evidence that Legalized Abortion Lowered Crime: A Reply to Joyce”, Journal of Human Resources, 2004).
Second, it is the sign of a desperate person when they start throwing out everything they can think of, regardless of its actual validity (maybe one will stick?). Flowing with Donohue’s argument, if I see that more people eat ice cream in the summer and there is more crime in the summer, does that mean there is any real relationship between ice cream consumption and crime? Maybe we should consider the temperature or the fact school is out? There is logic there to considering temperature or the school calendar. So maybe we combine them into a model. What logic says we need to include the current administration? Do appellate judges really base their opinions on the current administration? How many appellate positions were filled in 2001 and 2002 by Bush? How many changed party? How would that impact reversal rates? There may be an argument there, but since Donohue does not feel compelled to state his case for such inclusion (nor any evidence) he should not argue for its control or make grandiose claims that Barondes is inept for not controlling for it.
Third, much of his argumentation is set up so that, if Barondes did everything he asked for, the study would fail (otherwise known as a straw man:). He picked contentions that he knew could not work. Lets account for judges ages. Well, do we assume that judges get better as they get older or worse? Do they become more open-minded or more conformist? Do they push the boundaries of the law more or rein back in their unbridled liberal passions? Before a variable should be included as a control one should know that there is some type of consistency across the sample in the behavior of that control. My assumption would be that no such consistency exists and, using Donohue’s logic, since Donohue provided no counter evidence I am going to assume my position is statistically correct. Also, suggestion of a panel regression is fool-hearty with the type of sample Barondes has and Donohue is experienced (I hope so) to know this. Compression of the 13,000 cases into a panel consisting of percentage overturn rates would limit the sample to a breadth of 95 (or less) and a depth of 4 years (maybe more if sliced into increments less than annual). No statistician having been awake for more than 5 minutes in class would know that such a small sample would provide no significance for anything if Barondes were to include all the proposed controls (his degrees of freedom would be non-existent). I suppose Donohue would prefer for Barondes to waste 40 hours of his time doing such analysis only to spend 5 minutes writing a pithy blog entry about “How he was right, the results were not significant.”
I write this response not solely to defend Barondes, but to correct the asinine logic of an apparently bitter, defensive law professor. Knowing that he has spent his life promoting empirics and economic analysis in the law I should say that I am ashamed he would knowingly stoop to such levels merely to defend his ivy league pride. In the process he is misinforming the lawyers of tomorrow and cheapening the academic debate about legal education. This is in no way a complete rebuttal, but intended to counter the tone and nature of his argumentation. As a PhD candidate myself I feel I must stand up and say enough is enough.
I am bad at posting comments, so the jist of this may appear previously. Nonetheless: the econometrics/statistics aside, this article points up that judges would be better off taking the best students from the spectrum of top law schools than the entire class from Yale. A middling Yalie is a Yalie nonetheless, but what has that student done to deserve the clerkship over the best students at Michigan, Penn, UVA, Duke, Cornell, Northwestern, etc...? Achievement after admissions ought to count for something.
What's ironic about your post is it confirms the rumors about Yale grads.
You are unable to connect theory with reality, and as a result, you are unable to apply theory.
You suggested a number of theoretical reasons why Barondes' analysis *may* be wrong. That is a cheap attack on the author's hard work. It is inappropriate because it dissuades other authors from trying. It is inappropriate because you attempt to marginalize the hard work he put into collecting and analyzing data with hot air. Simply stated, there *could* be something wrong with any statistical study. An indisputable study is absolutely impossible when you are studying the real world. That's an obvious point and a poor basis for criticism.
Barondes' analysis speaks for itself. No statistician ever claims their results are indisputable, and neither did Barondes. However, it's something. I appreciate the grueling effort required to collect and summarize statistical data. I will certainly not let you brush it aside with hot air.
In summary, you provided a number of theoretical reasons why his conclusion *could* turn out to be in error. But outside of Yale theory is not allowed to trump reality. Reality is allowed to have their say.
As a long time and well credentialed statistician and economist I am truly ashamed of your post. What's amazing is you did it without even attempting to hide your bias. Could you not at least have had someone from another school parrot this for you?
I only hope it doesn't dissuade others from putting in the hard word required to analyze and study data. Don’t worry about the windbags. We appreciate the work you did.
Dear Another Statistician,
As a non-statistician, I can obviously see that not controlling for the judge is a major flaw in this paper. The paper purports to explain the effect of hiring a Yale clerk. It fails to do that if it doesn't compare the same judge with and without a Yale clerk. Fancy stats aside, that should be clear.
I did find a typo, though. "then we might expect given the high ranking of Yale Law School." Should be "than." You're welcome.
As a statistician I should point out that the procedure Barondes used does account for judge effects. The Conditional Logit model used 'clusters' each judges observations together. So the results given are those effects present, conditional on each individual judge. Just because the model doesn't give a coefficient for each judge doesn't mean it doesn't account for each judge.
This is the kind of thing that frustrates me about Donohue's post. Anyone who reads this is going to think that if some statistical procedure doesn't produce output for every control variable under the sun then it is worthless (some models include effects not listed on output, and some controls are simply worthless and unnecessary).
If Barondes were to run a panel model, like Donohue suggests he would need to include variables for each judge, and interaction variables for each judge and time period (to account for his judges change over time argument). That would mean a sample of around 380 and at least 295 variables, not considering all the other appropriate controls. It is simply not feasible. I am sure that Barondes considered such options and appropriately disregarded them. Donohue should not mislead his readers (like k).
When I read this, I honestly forgot what blog I was visiting. I would not expect this type of entry or post at Balkinization. Halfway though the post I thought I was reading a post at that insufferably pedantic, often pretentious blog with the green color scheme, which is written by a bunch of ivy leaguers Anyway, I could not get past Donohue's ruminations about the need to factor in the judge's characteristics---specifically, advanced age. It seems unlikely that a lawyer, let alone a federal judge, would disclose cognitive deterioration. Short of brain imaging, which isn't dispositive anyway, how would one go about measuring the correlation between reversal or problematic judicial opinions/rulings and aging---at least without running into the same problem of reliability that is the subject of Donohue's entire critique of this Yale law clerk study? One cannot assume that aging inherently diminishes judicial decision-making and legal analytical skills. (If so, the USSC might as well close up shop.) At the end of the day, it seems as though the factors/constants/variables that Donohue anecdotally sets forth to demonstrate the unreliability of the study beg the same criticism he doles out, as one is left with the seemingly unmeasurable and unanswerable issue of causation: is it age, or is it the Yale law clerk?
So Yale has a lock on judicial clerkships? And what evidence is there to demonstrate the extent to which the Justices (at SCOTUS) are actually influenced by their Yale clerks (or other clerks)? Is there a suggestion that Yale has SCOTUS by the SCROTUS? And what do originalists have to say about the role and influence of such clerks? Are the Justices tnat naive (or incompetent or lazy) to be taken in by their clerks who have "bubkis" for experience? And is there a suggestion that there is a continuing pipeline between the Yale clerks and the Yale Law faculty? Maybe there is need for a locksmith.
HD kaliteli porno izle ve boşal.Post a Comment
Bayan porno izleme sitesi.
Bedava ve ücretsiz porno izle size gelsin.
Liseli kızların Bedava Porno ve Türbanlı ateşli hatunların sikiş filmlerini izle.
Siyah karanlık odada porno yapan evli çift.
harika Duvar Kağıtları bunlar
tamamen ithal duvar kağıdı olanlar var
2013 Beyaz Eşya modeller
Sizlere Güvenlik Sistemleri ayarliyoruz
Arayin Hırdavat bulun
Samsung Nokia İphone Cep telefonu alin.
Super Led Tv keyfi
Amatör Porno - Amcik Porno - Anal Porno - Asyali Porno - Bakire Porno - Erotik Porno - Esmer Porno - Fantazi Porno - Gay Porno - Götten Porno - Grup Porno - Hard Porno - HD Porno - Hemsire Porno - Latin Porno - Lezbiyen Porno - Liseli Porno - Olgun Porno - Oral Porno - Rokettube - Sarisin Porno - Sert Porno - Tecavüz Porno - Travesti Porno - Türbanli Porno - Türk Porno - Ünlü Porno - Yasli Porno - Zenci Porno - Kari Koca Porno - Hayvanli Porno