The Depth and Breadth of Misleading Employment Numbers by Law Schools (And How to Solve It)

Brian Tamanaha

It is by now well known that law schools have been posting misleading employment numbers. We get it. Additional reminders of this verge on tiresome redundancy.

But we don't get it. Law professors have no idea how deep and pervasive this practice is (top to bottom). It is still continuing. And the extent of the deception is shocking.

South Texas College of Law, for example, lists in the current US News profile its earnings quartiles for full-time private sector jobs as follows: $75,000 (25th percentile), $92,500 (median), $160,000 (75th percentile). That looks great. However, only 5 percent of the people employed in the private sector provided salary information. What this means is that those numbers represent 18 graduates (at most) out of a class of 376. A savvy prospective student would be able to discern that these salary numbers are extremely unrepresentative (the percent reporting is not indicated in the magazine, but available through subscription). Unsuspecting readers would be fooled into thinking that good money was to be had coming out of this school.

Misleading number reporting is rampant. (I'll get to top schools below.) Here are the schools that post salary numbers on full time private employment with the lowest percent reporting:

South Texas (5 percent)
Florida A & M (9 percent)
St. Mary’s (9 percent)
Mississippi College (10 percent)
Ohio Northern (13 percent)
Roger Williams (13 percent)
Florida International (14 percent)
Tulsa (14 percent)
Thomas Jefferson (17 percent)
Gonzaga (19 percent)
Atlanta’s John Marshall (20 percent)
Touro (21 percent)
U. Detroit Mercy (24 percent)
California Western (25 percent)
Seattle (25 percent)
Stetson (25 percent)
Florida Coastal (28 percent)
New York Law School (28 percent)
Southern Illinois (28 percent)
Cleveland State (29 percent)
Hamline (29 percent)
Loyola Chicago (29 percent)
Southwestern (30 percent)
Widener (30 percent)
Drake (32 percent)
Baltimore (32 percent)
Miami (32 percent)
John Marshall (33 percent)
Pace (33 percent)
Capital (34 percent)
Hofstra (34 percent)
Missouri (34 percent)

The list goes on. Nearly 70 law schools post salary figures taken from half or fewer of the graduates with private full time jobs.

Law schools complain that it is unfair to blame them for low salary reporting rates—it is not their fault, they say, that students do not provide this information.

But that is not the primary explanation for what is going on. Grounds for skepticism lie in the results of the following chart, which plots response rate against law school rank. As with the employment chart posted here, a strong downward drift is evident, with the top schools having uniformly high response rates and the lower ranked schools having lower rates (with exceptions to be mentioned).



Why would this pattern exist? It makes sense that “JD required” employment rates would show this pattern because fewer graduates from lower down schools are getting lawyer jobs. But this chart relates exclusively to graduates who have jobs. In response to the law school’s survey (calls, emails, letters), the graduate says “Yes, employed in a private firm,” but then fails to answer the “How much do you earn?” question. The further one goes down the ranking, the greater the proportion of graduates who do not disclose their salary. Why?

There are two obvious explanations. Greater numbers of students at lower schools are embarrassed to state their earnings. That is a part of it. Another part is that law schools with low reporting rates are not trying very hard to get this information. Salary breakdowns by type of private legal employer are well established, with small firms paying the least (except for high end boutiques) and large firms paying the most. Once law school career services personnel find out the law firm’s identity, they can make a ballpark guess of the graduate’s salary. A law school’s posted salary numbers would be kept higher (and the response rate lower) if the school did not make follow-up inquiries to graduates in small firms or working as solo practitioners.

Lower ranked schools tend to have lower response rates because the numbers are poor (hence the reluctance of graduates provide it) and obtaining the information would harm the school’s salary profile (hence the reluctance of schools to get it). That explains the downward slope. As between these two factors, the reluctance of law schools is the bigger part. (Another possible explanation is inadequate surveying, but it is not plausible to think that the administrations of lower ranked schools are systematically dysfunctional.)

Evidence for allocating most of the blame for low response rates on law schools comes from the outliers. A number of lower ranked schools have relatively high response rates: Texas Southern (97 percent); Charlotte (86 percent); La Verne (85 percent); Texas Wesleyan (84 percent). The performance of these schools shows that even low salaries will be reported in large numbers if an effort is made. There is another way to see this. Compare the disparate performance among several schools ranked 84th (in a tie) by US News: Louisiana State (99 percent) and Arkansas (94 percent) have high response rates, while Seattle (25 percent), Hofstra (34 percent) and Santa Clara (36 percent) have low response rates. The only explanation (aside from sheer incompetence by the latter law schools) for such a great disparity is a lack of enthusiasm on the part of the law school to obtain the salary numbers from all graduates.

This matters because it produces a distorted picture for prospective students who are trying to get a sense, before they make a huge investment of time and money, of what they are likely to earn on the back end. Santa Clara lists stunningly high earnings for its 2009 graduates in full time private jobs: $110,000 (25th percentile); $160,000 (median); $160,000 (75th percentile). These numbers are high not because the class did so well but because the response rate was so low. Indeed it appears that the 2009 class at Santa Clara did poorly when one also considers that (notwithstanding the listed 85 percent employment rate) only 50 percent of the graduates got jobs as lawyers, and 45 percent of these jobs were part time. An educated guess, although I cannot tell for sure, is that the above numbers reflect the earnings of about fifty graduates out of a class of two hundred fifty nine.

The problem is not just that prospective students are unlikely to grasp all of this, but that the actual employment results and earnings for the entire class cannot be reconstructed from the information provided. Law schools with low response rates are hiding the ball under the guise of providing a lot of detailed information. A bunch of law schools are doing this. Seattle and Hofstra have alluringly high salary numbers based on low response rates. Loyola of Chicago looks really good at $108,376 (25th percentile), $112 ,000 (50th percentile), $152,981 (75th percentile), although only 29 percent of the graduates in private full time jobs provided salary information. Numbers like these are bait for the unwary.

New York Law School is a useful example because its dean disclosed in an interview the real numbers behind the veil, allowing us see the size of the gap. NYLS posts dazzling employment numbers in its US News profile: $60,000 (25th percentile), $160,000 (median), $160,000 (75th percentile). Those unbelievably high salaries for graduates from a low ranked school are explained by the 28 percent response rate. When asked about these numbers, Dean Richard Matasar defended them as accurate but he admitted that they are incomplete. More detailed data is provided on the law school website. “In these materials and in our conversations with students and applicants,” Matasar asserted, “we explicitly tell them that most graduates find work in small to medium firms at salaries between $35,000 and $75,000.” These painfully low salary numbers that Matasar says “most” graduates earn are drastically different from the giddily high numbers advertised.

Although he grants that three or four years ago students might have been deceived by the $160,000 figure (a significant admission in itself), that is no longer the case, Matasar asserts. “Students are not stupid and they’re not naïve.”

He is right that most prospective students today, after the recent spate of bad news about law schools, are probably aware that they cannot trust the advertised salary numbers—a sad statement about the diminished credibility of law schools—but that is not an answer to the crucial problem. A person thinking about attending NYLS might realize that the numbers are inflated, without realizing by how much, and without knowing what the true figures are. The information provided on the NYLS employment statistics page on its website nowhere clearly states the range identified by Matasar. Nor does NYLS directly inform its students that only 62 percent of the class of 2009 (nine months after graduation) had obtained jobs as lawyers, and 27 percent of these lawyer jobs were part time. How many people would pay nearly $150,000 in tuition for three years if they were explicitly and clearly provided with this information? It is too late, of course, to communicate this to the incoming crop of students after they arrive.

While the focus above is mostly on lower ranked law schools, problems exist at higher ranked law schools as well, and students going to these schools must make the same hard decisions about return on investment.

The absolute best salary numbers a school can report are $160,000 at each quartile. That is especially enticing because it represents that students throughout the class (at least to the 25 percentile point) are landing top paying jobs, which reduces the necessity to score top grades.

There are thirteen members of the exclusive Triple $160,000 Club. But not all are equal members. Harvard, NYU, and Chicago have 100 percent response rates. Pennsylvania (99 percent) and Columbia (98 percent) are also impressively high. The response rates hold up for Berkeley (95 percent) and Stanford (94 percent), tailing off a bit with Yale (92 percent) and Northwestern (91 percent), and then stretching downward for Virginia (86 percent) and Cornell (86 percent). Two schools have response rates much lower than others in the group: George Washington (75 percent) and Georgetown (69 percent). If we assume that many of the non-responding graduates of these latter two schools did not earn $160,000, a fair assumption, then both fall out of the coveted group, and they would look more like a bunch of schools with a lower 25 percentile salary but higher response rates (Duke, Michigan, Texas, UCLA, Fordham). For that matter, Cornell and Virginia, and even Northwestern and Yale, could easily fall out as well if a significant number of the non-responsive graduates earned below $160,000, and the schools barely hit that number for the 25th percentile with the responses it had.

It gets worse.

We cannot disregard the possibility that all of these numbers all across the board, which are based on self-reporting by the law schools, are puff. In March 2011, Forbes asked Payscale—which has a huge body of employment data from self-provided information by people who want to compare earnings with others similarly situated—to examine its database for the median salaries of recent law graduates from 98 law schools. It had salary information on 8,500 law graduates from these schools working full time in the private sector—90 percent of whom were working as lawyers—within five years of graduation. No law school had a median of $160,000. Columbia graduates topped list with a median salary of $157,000, with Virginia second at $137,000. Median salaries quickly fell away—Yale was 12th at $105,000; G.W. was 22nd at $83,000. Law schools were $30,000, $50,000, $70,000, and more, below their advertised medians. (Mine included.) Sixteen schools—including many with medians listed between $130,000 and $160,000—came in around half of what they claimed. The biggest gap was Seton Hall, with an advertised median of $145,000, but a Payscale median of $64,500.

Strictly speaking, these medians are not directly comparable because the advertised medians I am using are from 2009, whereas the Payscale data search looked at employed law graduates from 2006 to early 2011 (both relate to full time jobs in private employment). Having said that, law schools have tended to list consistent numbers across this five year period.

What could explain such a huge disparity for so many schools? One possible contributing factor is that the layoffs of large numbers of lawyers in the past few years swept away a number of graduates who were previously counted as employed by law schools within nine months of graduation. Another factor is that many graduates voluntarily leave corporate law firms after two to four years, moving to lower paying (less hours-driven) positions. These two explanations, though likely a part of it, would not appear to be sufficient to explain such an earnings chasm across so many schools. Another possible explanation is that the Payscale database has a downward bias that massively under represents true medians. What might produce this is unclear, but there could be some of it at work.

The most obvious explanation is that law schools—including top schools—have been telling whoppers about the salaries of their graduates. The strongest evidence to support this is that 14 of the 98 schools had Payscale medians that are close to their advertised medians—a few thousand dollars below or above. If the database is systematically underreporting medians, for some reason these schools are not affected. They are from all over—University of San Francisco, De Paul, Northeastern, Utah, Penn State, New Mexico, Kansas, Oklahoma, Marquette, Oregon, Louisville, LSU, Buffalo, Maryland. The main characteristic this group has in common (aside from a heavy representation of public schools) is that their advertised medians are not extravagant, between $50,000 and $70,000.

Looking at the schools with the largest disparities, two patterns emerge. The top 15 schools all report a $160,000 median, and further down schools in major legal markets also report $160,000 medians (Santa Clara, Hastings, Brooklyn, Fordham, NYLS, Boston University, Boston College, G.W., Catholic). These patterns suggest that the medians of law schools are not determined in the first instance by the actual salaries of the mid-point of the graduating class, but rather by what their competitors are claiming their median to be. If a school’s peers are claiming a $160,000 median, then that school has the same median (whatever it takes).

Law schools can protest that this skepticism is unwarranted. They can swear by their numbers. But why should anyone believe them. (Just to be clear: I am not accusing law schools of outright fabrication, but of aggressive massaging, like corporations that work the numbers to slightly surpass analysts' earnings forecasts.)

The law school funny number problem is out of control. And it won't stop on its own. Anyone who thinks the fix will come from the current ABA efforts to provide greater transparency is deluded.

There is only one possible solution in the short run. The deans at the top 20 law schools must sit together in a room, agree on the standards, and personally guarantee the veracity of what they report. All the other law schools will follow (or be embarrassed by continuing to post ridiculously implausible salary numbers). This must be done soon, before the next cycle of numbers comes around.

Leadership, collective action, and integrity--that's what we need to end this terrible situation.
Posted 12:30 AM by Brian Tamanaha [link]