CoRe: Continuously finetuned LLM query rewriter boosts video search relevance

LLM-based query rewriters in production face a fundamental tension: the training reward must reflect how the ranker actually consumes the rewrite, but the training procedure must remain cheap enough for continuous redeployment as data drifts. CoRe (Context Relevance) solves this with a reward function that uses the deployed multimodal relevance model as its source, in a multiplicative ratio form that mirrors the production fusion algebra. This closes the simulation-production gap that offline reward proxies leave open. A semi-online Mixed Preference Optimization loop makes training affordable at multi-million-instance weekly scale: a DPO-style pairwise objective restricts gradient pass to a small top-k/bottom-k subset of sampled trajectories, and a phase structure reduces parameter syncs from per-step to per-phase.

CoRe was redeployed weekly for over five months in a major short-video search engine. The rewriter's output is consumed as parallel relevance signals at recall, raw-rank, and fine-rank stages without displacing the original signals, bounding the blast radius of any rewriter failure. An automated promotion gate over reward-like and stability metrics detected and recovered from a real reward-hacking incident in production. Online A/B tests from two sequential launches—first deploying at fine-rank, then extending to recall and raw-rank—delivered statistically significant reductions in change-query rate on rewrite-impacted queries, with all headline relevance and engagement metrics moving in the expected direction.