Quantized Inference for OneRec-V2

A research team has successfully applied low-precision, FP8 quantization to the OneRec-V2 generative recommendation model, overcoming a major hurdle in industrial AI. Traditional recommender systems have been notoriously difficult to quantize due to their high-magnitude, high-variance weights and activations, which cause significant performance drops when precision is reduced. However, the team's empirical analysis revealed that OneRec-V2, built on newer generative paradigms, exhibits weight and activation statistics that are far more controlled and resemble those of large language models (LLMs). This architectural shift, coupled with a more compute-intensive inference pattern, created a unique opportunity.

Leveraging this property, the researchers developed a tailored FP8 post-training quantization framework and integrated it into an optimized inference system. The results were dramatic: a 49% reduction in end-to-end inference latency and a 92% increase in system throughput. Crucially, extensive online A/B testing confirmed the quantized model introduced no degradation in core recommendation metrics like click-through rate. This breakthrough demonstrates that as recommender systems evolve to adopt LLM-like architectures, the powerful algorithm and system optimization techniques proven in the LLM domain—such as quantization—can be effectively ported over, unlocking massive efficiency gains for large-scale, real-world services.