NetEase's Melo agent cuts LLM hallucinations 7.8pp with reflective retry
Melo boosts playlist retention 2pp and engagement 1 min by fixing AI mistakes.
NetEase Cloud Music deployed Melo, a production LLM-powered music recommendation agent structured as a deterministic five-node state graph over heterogeneous tools. Rather than fine-tuning the controller, Melo uses a prompt- and state-machine-driven orchestration policy. The key insight: at scale, the bottleneck isn't how smart the LLM is but how the system detects and recovers from its mistakes. Two failure modes drove design: entity hallucination (agent commits to catalog-unsupported interpretations) and long-tail degradation (over-constrained requests collapse to generic popular fallbacks).
To fix these, Melo introduces two complementary mechanisms. Inference-time entity grounding repurposes the production search index as a verification primitive, gating entity decisions before propagation. Reflective retry verbalizes failure reasons from broken tool chains and feeds them into the next planning step, enabling constraint relaxation instead of blind fallback. A one-month online A/B test reported over 2pp lift in playlist retention and over one minute lift in engagement. Offline ablation showed 7.8pp reduction in entity misidentification from the three-layer grounding stack, and reflective retry fired on 5.8% of sessions with 59% process-level recovery.
- Melo uses inference-time entity grounding via NetEase's production search index to prevent hallucinated entities from propagating downstream.
- A one-month online A/B test on playlist surfaces showed over 2 percentage point lift in retention and over one minute lift in engagement.
- Reflective retry achieves 59% process-level recovery on 5.8% of sessions by verbalizing failure reasons and relaxing constraints.
Why It Matters
Proves that production LLM agents need robust error correction, not just smarter models, for real-world recommendation systems.