New Paper Shows Scoring Functions Fail at Fairness-Utility Trade-offs

A new arXiv paper by Shubham Singh, Ian A. Kash, and Mesrob I. Ohannessian challenges a core assumption in ranking systems: that scoring functions can adequately balance utility and fairness. The authors demonstrate that scoring—the standard method of assigning relevance scores to items—is fundamentally sub-optimal for achieving all utility-fairness trade-offs. They provide counterexamples under a generic fairness formulation, showing the issue holds regardless of whether scoring is deterministic or randomized, and whether fairness is measured per query or across multiple queries.

On the positive side, the research shows that semi-greedy post-processing methods can achieve significantly better trade-offs, often approaching the ideal of exhaustive post-processing while remaining computationally tractable. This finding has direct implications for search engines, recommendation systems, and any AI that ranks items—suggesting that simply tweaking scoring functions is insufficient for fairness goals, and that post-hoc adjustments may be necessary.