Local AI Cascade Outperforms GPT and APIs for Student Data De-Identification

Educational dialogue transcripts contain personally identifiable information (PII) entangled with curricular content, creating a privacy vs. accuracy tradeoff. Commercial LLMs handle ambiguity but require sending student data off-site, while local NER systems over-redact academic terms. A new paper from Haocheng Zhang and colleagues reframes de-identification as constrained privacy triage: a recall-first union proposer over-generates candidate spans using two lightweight encoders plus deterministic rules, then a context-aware reviewer makes a binary Redact/Keep decision using surrounding dialogue and speaker roles.

Tested on math tutoring transcripts from two large platforms, the strongest local configuration achieved 0.958 macro F1, compared to 0.767 for a same-family LLM-only baseline and 0.706 for a commercial API—all while running on a single laptop. On a challenge set designed to test curricular-personal name ambiguity (e.g., 'Riemann'), the local system degraded by only 0.03 F1 versus 0.19–0.25 for smaller reviewers. These results demonstrate that problem formulation, not model scale, is the key to effective educational de-identification.