Fortress framework stabilizes search recommendations by pruning volatile features

In large-scale search and recommendation systems, predictive models often suffer from temporal instability when certain input features—particularly engagement-based ones—cause scores to fluctuate unpredictably over time. This degrades user experience and reliability, especially in multi-stage systems where consistent predictions are critical. The paper presents Fortress, a general framework that stabilizes predictions by pruning volatility-inducing features while keeping the model's predictive accuracy intact.

Fortress operates in four steps: (1) collect historical snapshots of score fluctuations, (2) identify samples with unstable predictions, (3) isolate and remove the offending features, and (4) retrain using only stable signals. The authors validate Fortress on a real-world query-to-app relevance model from a large app marketplace. Offline results show significant improvements in both stability (measured by Coefficient of Variation) and classification performance (PR-AUC), demonstrating that suppressing engagement signal volatility doesn't come at the cost of accuracy.