New arXiv study reveals pooling data can reverse AI decisions

A new paper titled "More Data, Worse Decisions? Preference Reversals in Neural Networks under Gram Incompatibility" (arXiv:2607.27255) from authors Yanli Yan, Yuanzheng Li, Yong Zhao, Hongbo Guo, and Shoudong Han investigates a counterintuitive failure mode: combining datasets can make a model worse at making decisions. The team formalizes this through Case-Based Decision Theory (CBDT), specifically its composition axiom, which requires that preferences supported by individual sources survive when data is pooled and the model refitted. They show that for fixed-representation neural networks with ordinary least squares (OLS) output heads, pooled refitting recomputes the inverse-Gram geometry that weights source evidence. That recomputation can reverse shared preferences, violating the axiom and leading to worse decisions even though accuracy may improve.

The authors derive both exact and approximate conditions under which preference preservation holds, and introduce a scale-invariant Gram mismatch measure to prioritize which candidate pools are safe to combine. They also propose geometry-oriented regularization that shapes source geometry during training to reduce reversal risk. To make the problem operational, they develop a three-stage audit that traces strict pairwise preference reversals through decision changes to task-defined utility loss. Experiments span a load-based bidding proxy and medical and financial decision proxies. The load audit identifies a measurable nonzero class of source-consensus-relative harmful decisions under proxy utility, while cross-domain audits show that comparable Gram mismatch can correspond to sharply different preservation rates. The framework positions screening, analytic certification, geometry-oriented training, and decision-consequence auditing as complementary tools to make compositional reliability measurable in real-world ML pipelines.