Legal AI may outperform human lawyers on some benchmarks, but could an AI ever outperform a human judge? Judges don’t just argue rules, they interpret them and sometimes make new ones! As a result, autonomous judges are a microcosm of the alignment problem with potentially large societal impacts and costs.

My parent's dreams - like every other Jewish parent - may soon perish. Their two favorite professions, lawyers and doctors, could be fully replaced by AI in our lifetimes.

Beyond AI tools for professionals (Harvey/OpenEvidence), there’s a wave of AI native companies (Crosby/Nolla Health) that empower AIs to autonomously provide legal or medical services without any human supervision. In July, Crosby announced they were “taking out professional liability insurance for [their] agents, so that they can do autonomous legal work,” and Nolla Health became “the first organization in the U.S. authorized by a state regulator for AI to issue an initial prescription.”

While still early, the Crosby/Nolla type of firm are fundamentally different from the Harvey/OpenEvidence firms: they threaten to remove humans from the loop. This may not happen immediately - there are regulatory hurdles to jump through even when AI capabilities surpass that of humans - but it seems clear there’s a possibility AI lawyers and doctors fully replace humans in the medium-ish term (3-10 years).

If you don’t agree, then I suggest you stop reading here. The rest of this article will build on that assumption and only go further off the deep end. It will also focus on the legal half of this equation, because I am a mere Juris Doctor, not a Doctor of Medicine.

If you’re still here, I have a question for you: if you were arrested for a crime and put on trial, would you rather have a human lawyer or an equally effective AI lawyer?

If you answered either choice, you’re wrong. You see, I’ve rigged the question - it’s the classic “what’s heavier, a ton of bricks or a ton of feathers.” The AI is equally effective, in other words, it’s exactly effective enough for you to be indifferent. That is my premise and therefore your answer.^[1]

Now, what if that AI lawyer were more effective than its human counterpart. Well at that point, you should necessarily prefer it! Once again, I see no reason an AI wouldn’t be able to provide superior legal services to a person in the very near future given the deeply bureaucratic nature of the law.

According to historian and author Yuval Noah Harari, humans have built a civilization governed by language-based systems - laws, contracts, regulations - that are fundamentally unnatural for human cognition but ideally suited to AI. AI need not surpass humans in every dimension of intelligence to outperform us within institutions whose rules and structures are better suited to machines than to the humans who created them. The legal system, with its elaborate procedural rules, competing interpretations, and reliance on precedent, may be one of the clearest examples of this phenomenon.

You could imagine a future adversarial system where two agents - an Angel and Devil - perch on the judge’s shoulders and argue on behalf of their clients. This would have a number of advantages:

1. Equity: There would no longer be a disparity between the fancy shmancy lawyers that keep Wall Street alive and the overworked public defenders that sometimes keep their clients alive (when they accept plea bargains that avoid the death penalty).
2. Speed: AIs can process information much faster than humans, allowing them to dispense their legal assistance on demand.^[2]
3. Efficiency: The AI agents are better than humans at conveying relevant arguments to the human judge, and therefore, reduce the amount of time and cognitive load needed by the human judge for each case.
4. Ethics: There wouldn’t be any incentive issues with AI lawyers needing to hit a quota on convictions or taking a cut of their clients settlements.^[3]

Since the system is adversarial and moderated by a human arbiter, alignment wouldn’t be a huge issue - you’d simply set the effort to ultracode and say, “do your best within the law!” and the system would work! There’s probably more to be said there, but that’s close enough to get to my main point:

It’s way harder to substitute AI for the judge.

Lawyers have a relatively straightforward objective: advocate for their clients as hard as possible within the rules of the game. Judges, on the other hand, have to decide what the rules mean, how they should be applied, and even make new rules when there aren’t any that apply. There's no objective function for judges. Two perfectly reasonable judges might often reach entirely different conclusions on the same case, and neither is necessarily wrong. So how do you tell an AI judge what to optimize for?

If you imagine our adversarial system with the Angel and Devil AIs but add an AI Judge, you unlock an order of magnitude more issues (that look a whole lot like capital A-Alignment). These include:

-Context: The judge may want to know do its own research (e.g. check public opinion). For things outside the scope of the trial, deemed inadmissible, you’d want to sandbox the judge and avoid a HuggingFace-esque incident.

-Interpretation: The intention of a law, as written, is not always how it gets interpreted (see: tax loopholes).

-Scheming: You could imagine an AI judge could, for whatever reason, simply ignore the law, make his own assessment internally, and then fit his decision to that. I’m 95% sure this is how the supreme court works. You’d need mech interp tools to make sure the judge is adhering to due process.

-Sentencing: Even within statutory maximums/minimums, judges have a lot of discretion in sentencing and are supposed to weigh the severity of the crime, the defendant’s circumstances, odds of re-offending, etc.

-New Laws: This might come as a surprise to you, but laws are not the only source of laws! Judge’s ruling and interpretations make up “common law,” which is oftentimes a novel source of laws.

If the judge were fully aligned these would all be fine. But, like any superintelligence, if there is even a modicum of misalignment we end up in a shitty Chris Pratt movie.

Still, it’s tantalizing to think of all the good a fully autonomous legal system could do. No more fighting parking tickets, chasing down security deposits, or showing up for jury duty. Oh, and no more systemic racial biases, no more disparities in access to services, no more ambulance chasers, and no more waiting.

Still, this probably won’t get built because it’s quite scary to our mammalian brains. Much like self-driving cars, we’d rather take our own chances at crashing - even if they’re 20x higher.

Personally, I’ll take the Waymo.

1. ^{^}

   There’s something to be said about bedside manner - I’m not sure if that cuts in favor of the AI or the human - we don’t count the emotional weight of plucking feathers in the ton of bricks, so we’re not counting that here.

2. ^{^}

   I say this having worked in a public defender’s office. My boss once took a week of vacation to dedicate herself to prep for a murder trial (and still lost).

3. ^{^}

   I’m assuming here that the AIs are provided more like a utility than by a for profit company, which would of course open up a whole different set of incentive issues.