Yandex's Long-History Transformer Boosts Ad Ranking 2.77% with Zero Latency

Online advertising faces a hard constraint: ad scores must be computed within a few hundred milliseconds to participate in real-time auctions. Full user interaction histories are highly informative for click-through rate (CTR) prediction, but running a large sequence encoder at request time is impossible. Yandex researchers solve this with a decoupled architecture: a high-capacity offline transformer asynchronously encodes the user's cross-surface history into a compact representation stored in a feature store, while a lightweight runtime model combines this cached history with the most recent events and request context.

The offline encoder is pre-trained autoregressively using a dual objective—feedback prediction and next-item prediction—on large-scale interaction logs. The two-stage pipeline is then fine-tuned for CTR prediction on the target advertising surface. In offline evaluations, the split design recovers 72–80% of the quality of a full-history runtime transformer that would be too expensive to deploy. The cached representation is robust to staleness, allowing inexpensive refresh policies.

In production A/B experiments on Yandex's search advertising and advertising network, the system improved the primary ranking metric by +2.77% and +2.1% respectively. Revenue increased by +2.26% on search and +0.43% on the network, with no increase in serving latency. The paper demonstrates that deep user history can be practically leveraged in real-time ad auctions through intelligent offline pre-computation and caching.