Trust scores drift across LLM checkpoints, audit of Yi, Qwen, Mistral, Gemma finds

A longitudinal audit published on arXiv (July 2026) examines trustworthiness drift across twelve successive checkpoints of four open-source LLM families: Yi, Qwen, Mistral, and Gemma. The researchers applied a fixed basket of trust benchmarks using multiple prompt templates to measure how scores change from one checkpoint to the next. Mean absolute adjacent-generation drift was significantly above an independence-based no-drift reference null, and the result persisted even when dropping single benchmarks, dropping single release lines, or switching to strict scoring.

The paper argues that model cards routinely quote trust-benchmark scores without recording when they were measured, and the same number is carried across successive checkpoints as if the model had not shifted. The authors conclude that trust scores should not be carried forward without remeasurement; instead, each checkpoint's score should be reported as a dated artefact. They package this as a longitudinal model card. The study explicitly excludes closed APIs, larger models, canonical benchmark protocols, and fixed month-cadence rules from its scope.