SIFT lets document classifiers teach themselves without human labeling

A team of researchers has unveiled SIFT (Self-Improving, Frozen-gate Training), a dynamic document classifier designed to solve the enterprise labeling bottleneck. The architecture splits classification into two stages: a cheap, CPU-bound pipeline using a SPLADE sparse encoder feeding a LightGBM head handles the majority of pages, and only low-confidence pages are escalated to an LLM judge for review. The judge's verdicts are automatically written back into the labeled corpus, creating a self-feeding loop where the expensive model continuously teaches the cheap one. Over time, the escalation rate falls, accuracy compounds with use, and the labeled corpus grows from production traffic rather than costly upfront annotation efforts. Onboarding a new document family requires only a declarative bundle—label space, anchor phrases, and a judge glossary—not a months-long labeling project.

The hardest challenge SIFT addresses is safety: an autonomously retraining classifier can silently regress. The paper introduces a two-part promote gate to prevent that. First, a critical-label F1 regression check compares the new model's performance on key labels against the previous version. Second, a frozen golden regression set—a static holdout the model is never trained on—serves as an additional veto. Either check can block a model promotion, making "retrain monthly without a human" routine rather than reckless. In an illustrative multi-domain deployment, the system demonstrated steadily falling escalation rates and rising accuracy, with marginal labeling cost trending toward zero. The approach effectively turns the classifier into a service that improves with every production request, opening the door to dynamic document classification at enterprise scale.