New framework exposes Llama Guard's child safety blind spots

As generative AI becomes common in classrooms and children's apps, existing safety evaluations fall short by ignoring risks unique to minors. A paper accepted at the HEAL Workshop at CHI 2026 proposes a framework that integrates expert-defined hazard categories with real-world AI incident databases to create synthetic test sets for child safety. The framework applies domain-specific risk factors—rather than generic content filters—to evaluate how well models detect unsafe interactions.

Applying the framework to the education sector, the study tested three Llama Guard models on detecting unsafe user prompts. Results show that current Llama Guard models struggle to identify education-related unsafe prompts, highlighting a significant gap in current safety measures. The authors emphasize the need to extend evaluation to more risk categories and involve domain experts throughout the pipeline to ensure AI systems are safe for children.