AI Can Now Read Court Rulings — But It Still Misses the Point

A researcher has built a new test to answer a simple question: can AI tell how a judge is arguing? The paper, accepted at an AI-and-law workshop, uses rulings from Germany's Federal Constitutional Court — the country's highest court — broken into single sentences. Each sentence was hand-labeled with the legal "method" a judge was using: looking at the exact words, the surrounding context, or how the rule fits with other laws.

Four AI models from three different companies were then asked to sort those sentences. They got it right roughly 70 to 79 percent of the time. Picking up on plain wording (what lawyers call "grammatical" interpretation) was easiest; judging how a rule fits into the wider legal system was hardest. The researcher also tried letting software automatically rewrite the instructions given to the AI — a technique called prompt optimization. It didn't reliably beat the prompts a human legal expert wrote by hand.

Why should you care? Legal AI is already showing up in law offices, court document review, and do-it-yourself legal tools. Most of it is good at finding text. Deciding why a ruling means what it means is the harder and more valuable skill — and getting it wrong one time in four is not good enough for advice that can cost someone their home, job, or freedom.

The catch: this is one court, in one country, in one legal tradition, with seven yes-or-no questions. The study shows AI can spot patterns in judicial language. It does not show AI understands the law. Think of it as a ruler for measuring progress, not a robot judge — and definitely not a replacement for a lawyer.