A Judicial Whodunnit: Shedding Light On Unsigned Opinions

Within legal circles, the mystery of “Whodunnit?” has increasingly become “Who wrote it?” as courts, including the U.S. Supreme Court, keep issuing opinions without divulging who actually authored them. Since 2005, for example, the Roberts Court has disposed of at least 65 cases through unsigned per curiam opinions. Many cases also came with unsigned concurring or dissenting opinions.

We place a high value on transparency in our democracy, and that should certainly apply to Supreme Court justices, who, after all, are already protected by lifetime tenure. Obscuring authorship removes the sense of judicial accountability, making it harder for experts and the public alike to understand how important issues were resolved and the reasoning that led to these decisions, especially in controversial cases. We’ve all heard the charge that judges are legislating from the bench — but assessing that claim requires, at the least, the ability to link opinions to individual decision makers.

Obscuring authorship removes the sense of judicial accountability, making it harder for experts and the public alike to understand how important issues were resolved and the reasoning that led to these decisions...

This is why my colleagues and I decided to apply machine learning algorithms to predict the authorship of opinions that are unsigned or whose attribution is disputed — we wanted to see if technology could help uncloak judicial anonymity. The challenging nature of parsing legal text algorithmically required broad collaboration across several different disciplines and institutions including computer science and engineering at MIT, the Berkman Center for Internet & Society at Harvard, and a practicing lawyer. [Full credits below]

To us, this task was more than just an academic parlor game. And our findings will help citizens and legal scholars alike.

Our results, which appeared in a study just published in the Stanford Technology Law Review, applied language processing to 568 Supreme Court opinions between 2005 and 2011. We leveraged advances in computational power and machine-learning algorithms to infer the authorship of cases with a high degree of accuracy, including the Supreme Court’s controversial decision in National Federation of Independent Business v. Sebelius concerning the 2010 Patient Protection and Affordable Care Act, better known as Obamacare.

After the court’s unexpected June 2012 ruling that left Obamacare constitutionally standing, speculation suggested that Chief Justice John Roberts wrote the majority and the dissenting opinions (both were unsigned), supposedly switching sides midstream. But when we applied our algorithms to evaluate the dissenting opinion against the diction used by justices in signed opinions, we concluded the probable author to be either Justice Antonin Scalia or Justice Anthony Kennedy. The unsigned opinion did not have the verbal characteristics used by Roberts in his other signed opinions.

Our research went well beyond this particular case to reveal an interesting insight into the judicial politics behind unsigned opinions. We found that justices commonly described as “conservative” were the predicted authors of 45 out of the 65 per curiam opinions and justices commonly described as “conservative-swing” are predicted authors of 13 of the remaining 20 opinions. According to our study’s algorithms, only seven of the unsigned opinions since 2005 were predicted to be authored by a “liberal” justice.

To us, this task was more than just an academic parlor game. And our findings will help citizens and legal scholars alike.

To further test our methodology, we applied it to another recent, high-profile Supreme Court decision, the Defense of Marriage Act (United States v. Windsor). Our algorithm predicted, with 99.7 percent probability, that Justice Kennedy was the author of the majority opinion. And because this was in fact a signed opinion, we know that that Kennedy prediction was correct.

Beyond our specific findings, this research demonstrates the potential impact of new technologies on legal scholarship. We have shown that word-level features can distinguish authorship with substantial accuracy. And by identifying authorship, the public is better able to understand the origin of important judicial rulings that can affect peoples’ lives.

The article “Using Algorithmic Attribution Techniques to Determine Authorship in Unsigned Judicial Opinions” by William Li, Pablo Azar, and Robert Berwick (MIT Computer Science and Engineering), David Larochelle and Phil Hill, (Berkman Center for Internet & Society at Harvard University), James Cox (associate attorney, Jenner & Block) and Andrew W. Lo (MIT Sloan School of Management) appears in Volume 16 of the Stanford Technology Law Review and can be downloaded here.