AI Legal Tools Miss The Point By Deleting Tiny Words

When computers analyze legal text — court opinions, statutes, rulings — they usually strip out what programmers call "stopwords." These are tiny, frequent words like "the," "of," "and," and "shall." The habit dates back to 1950s search technology, when storage was precious and those words seemed like pure noise. A law professor at a U.S. university decided to test whether that old habit still holds up.

He matched thousands of U.S. Supreme Court opinions to a well-known database of case labels, then measured how accurately a simple AI model could guess two things: whether a ruling leaned liberal or conservative, and whether it involved constitutional law. For each task he tried removing roughly 18,500 different words, one at a time, to find the best possible list of words to delete. The results were blunt. Standard word lists used everywhere made accuracy worse. Even a hand-optimized list performed no better than deleting nothing at all. And a second AI model trained to predict which deletions would help simply couldn't.

Why does this matter beyond academia? Because legal scholars, journalists, and policy researchers increasingly draw conclusions about judges and courts from these automated analyses. If a silent cleanup step changes which words the model sees, the conclusions can shift too — and nobody would notice, because the cleanup happens before anyone looks at results.

Legal language is unusually dependent on small words. "Freedom of speech" and "freedom from speech" mean different things. "Shall" versus "may" decides whether something is required or optional. Removing those words is like judging a recipe by deleting every mention of salt: you might still spot the dish, but you've lost the thing that tells you how it actually tastes. The broader takeaway applies to any inherited tech default — if a step was never tested, it's a guess, not a standard.