New Study: AI Assistants Mistake Repeated Facts for Real Proof

A new benchmark called GraphEcho asks whether large language model (LLM) agents — systems that can follow graph paths on their own — mistake repeated encounters with the same evidence for additional corroboration. The benchmark varies path counts and evidential origins while holding evidence content fixed, and evaluates both judgments and active exploration. In controlled synthetic experiments, judgment shifts turned out to be model-dependent, but redundant supporting paths increased the share of repeated walks across all evaluated frozen agents. Provenance-aware post-training (PAPT) reduced revisits and improved synthetic accuracy, yet covered fewer distinct sources. On scientific claims, it continued to reduce repetition while accuracy declined. According to the article, these findings expose a gap between efficient exploration and effective evidence use: an agent can learn to stop repeating itself while overlooking information it needs. GraphEcho is offered as a controlled way to evaluate both what graph agents conclude and whether their exploration reaches distinct evidential sources.