When: Third Friday of each month at 1 PM Central Time (sometimes fourth Friday; next workshop: Friday, October 30, 1:00 to 3:00 p.m. Central Time).
What: First 90 minutes: Two presentations of CS+Law works in progress or new papers with open Q&A. Last 30 minutes: Networking.
Where: Zoom
Who: CS+Law faculty, postdocs, PhD students, and other students (1) enrolled in or who have completed a graduate degree in CS or Law and (2) engaged in CS+Law research intended for publication.
A Steering Committee of CS+Law faculty from Berkeley, Boston U., U. Chicago, Cornell, Georgetown, MIT, North Carolina Central, Northwestern, Ohio State, Penn, Technion, and UCLA organizes the CS+Law Monthly Workshop. A different university serves as the chair for each monthly program and sets the agenda.
Why: The Steering Committee’s goals include building community, facilitating the exchange of ideas, and getting students involved. To accomplish this, we ask that participants commit to attending regularly.
Computer Science + Law is a rapidly growing area. It is increasingly common that a researcher in one of these fields must interact with the other discipline. For example, there is significant research in each field regarding the law and regulation of computation, the use of computation in legal systems and governments, and the representation of law and legal reasoning. There has been a significant increase in interdisciplinary research collaborations between researchers from CS and Law. Our goal is to create a forum for the exchange of ideas in a collegial environment that promotes building community, collaboration, and research that helps to further develop CS+Law as a field.
Please join us for our next CS+Law Research Workshop online on Friday, October 30, 1:00 to 3:00 p.m. Central Time (Chicago Time).
Workshop 39 Organizer: Northwestern University (Dan Linna)
This edition of the CS+Law workshop will feature two presentations on AI and the evaluation of expert judgment, in scientific peer review and in the federal courts.
Agenda:
20-minute presentation and discussion - Nihar B. Shah
10-minute discussion
20-minute presentation and discussion - Nathan Reitinger
10-minute discussion
30-minute discussion about both talks
30-minute open discussion
Presentation 1: LLMs for Scientific Reviewing/Interviewing
Presenter: Nihar B. Shah (Associate Professor, Machine Learning and Computer Science, Carnegie Mellon University)
Nihar Shah (Carnegie Mellon University) will discuss what is driving the explosion of submissions to scientific conferences and journals, how quotas and author interviews can address it, and how AI reviewers should be evaluated before they are deployed.
Nihar B. Shah is an associate professor at Carnegie Mellon University, with joint appointments in the Machine Learning and Computer Science departments. His research is centered around the evaluation of science, including peer review and autonomous AI scientist alignment. His research has already been used in the evaluation of several hundred thousand research papers and thousands of grant proposals, in over 200 venues. The algorithms we have developed are also deployed in diverse applications such as admissions decisions and competition judging. He is a recipient of The Allen Newell Award for Research Excellence at CMU, a Young Alumnus Medal from the Indian Institute of Science, the David J. Sakrison memorial prize from UC Berkeley for a "truly outstanding and innovative PhD thesis," and several Best Paper awards.
Presentation 2: Easy Case Déjà Vu
Presenter: Nathan Lawrence Reitinger (Empirical Fellow in Law and Computer Science, Northwestern Pritzker School of Law)
How do judges decide cases? Legal scholars have debated this question for well over a century. Some consider law to be a science, where fact and law mechanically yield a ruling; others reduce outcomes to little more than what a judge ate for breakfast. And yet, both positions agree on something—some cases are “easy.” An easy case is one where the facts and law fit so neatly together that an outcome is nearly compelled, resulting in consistent, mechanical-like rulings. This is known as the easy-case hypothesis, a theory without much empirical support in the district courts. But that is not all. Today, this hypothesis is creating a déjà vu.
Around 50 years ago, a shift occurred in the formalist–realist divide. Instead of asking “do gaps in law exist,” scholars began asking, “what next, after gaps in law exist.” This pivot was the byproduct of a consensus on easy cases. Easy cases were bracketed off, shifting the debate toward harder questions of what to do in the face of gaps. But before that scholarship moved on, no one stopped to verify it: Do easy cases really exist in the judiciary? The last 50 years of empirical scholarship, and the Priest-Klein Hypothesis, suggest “no” as empiricists routinely find outcomes not based in law and fact—i.e., judicial decisions are based on difficult-to-predict non-law factors like ideology or strategic motivations. This question, therefore, is ripe for review.
Perhaps more surprisingly, the unverified assumption is again back in vogue, this time thanks to Artificial Intelligence (AI). Claims that AI is “perfectly formalistic” and can compute the “legally correct answer” 100% of the time stand on a hefty assumption: Law is predictable, consistent, and nearly compelled from the facts—in not so many words, law is easy. And yet, it is patent that AI cannot find an optimal solution if there is no optional solution; if reasonable minds disagree about an outcome of a case, then any output by AI is merely a guess. And that brings the debate back to where it was 50 years ago: Do easy cases exist?
This Article addresses each question by supplying the first empirical test of the easy-case hypothesis at the federal district court level, using an AI-leveraged pipeline to identify, measure, and scrutinize easy cases. The Article builds one of the largest federal district court datasets ever compiled—nearly 200,000 cases, over ten million docket entries—and uses it to semantically cluster case complaints on a fact-by-fact basis, using motion to dismiss outcome consistency as a proxy for the easy case. The Article’s primary empirical finding is that when multiple judges within the same district court consider a substantially similar fact pattern, judges rule consistently. This finding bookends the theory debate, but also adds a much-needed texture to AI judging—the novel pipeline permits scrutiny for within-district splits, adds a quality check for AI training data, and, ultimately, provides a pathway for augmented AI judging.
Nathan Reitinger is the Empirical Fellow in Law and Computer Science at Northwestern Pritzker School of Law. His research integrates empirical and technical methods to study privacy, artificial intelligence, and the regulation of digital technologies. His scholarship has appeared or is forthcoming in leading law reviews, including the Stanford Technology Law Review, American University Law Review, and Jurimetrics, as well as premier computer science venues such as the IEEE Symposium on Security and Privacy, ACM Conference on Computer and Communications Security, USENIX Security, and the Privacy Enhancing Technologies Symposium. He holds a Ph.D. in Computer Science from the University of Maryland, an M.S. in Computer Science from Columbia University, and a J.D., magna cum laude, from Michigan State University College of Law. Prior to joining Northwestern, he worked on technology and privacy matters with the Federal Trade Commission and the University of Michigan, and served in the United States Peace Corps in Kenya.
Join our group to get the agenda and Zoom information for each meeting and engage in the CS+Law discussion.
Submit a proposed topic to present. We strongly encourage the presentation of works in progress, although we will consider the presentation of more polished and published projects.
Friday, October 30, 1:00 to 3:00 p.m. Central Time (Organizer: Northwestern University)
Friday, November 15, 1:00 to 3:00 p.m. Central Time (Organizer: University of Chicago)
Friday, January 17, 1:00 to 3:00 p.m. Central Time (Organizer: UPenn)
Friday, February 21, 1:00 to 3:00 p.m. Central Time (Organizer: Cornell)
Friday, March 21, 1:00 to 3:00 p.m. Central Time (Organizer: Tel Aviv University + Harvard)
Friday, April 18, 1:00 to 3:00 p.m. Central Time (Organizer: TBD)
Friday, May 16, 1:00 to 3:00 p.m. Central Time (Organizer: Georgetown)
Ran Canetti (Boston U.)
Bryan Choi (Ohio State)
Aloni Cohen (U. Chicago)
April Dawson (North Carolina Central)
James Grimmelmann (Cornell Tech)
Jason Hartline (Northwestern)
Dan Linna (Northwestern)
Paul Ohm (Georgetown)
Pamela Samuelson (Berkeley)
Inbal Talgam-Cohen (Technion - Israel Institute of Technology)
John Villasenor (UCLA)
Rebecca Wexler (Berkeley)
Christopher Yoo (Penn)
Northwestern Professors Jason Hartline and Dan Linna convened an initial meeting of 21 CS+Law faculty at various universities on August 17, 2021 to propose a series of monthly CS+Law research conferences. Hartline and Linna sought volunteers to sit on a steering committee. Hartline, Linna, and their Northwestern colleagues provide the platform and administrative support for the series.