Prime Video's Diff-Aware AI cuts deployment risk by 83% recall

Amazon Prime Video faced a critical operational challenge: managing code deployments during live events and rapid feature releases without causing service outages. Traditional change control relied on blanket deployment freezes that blocked all changes regardless of risk, creating significant developer toil. Researchers at Prime Video—Mayur Kurup, Hyunjae Suh, Swathi Vaidyanathan, Pranesh Vyas, Srinidhi Madabhushi, and Yegor Silyutin—introduced a framework that uses diff-aware features: characteristics derived directly from code modifications. Instead of relying on developer-specific metadata or extensive historical data (which raise privacy concerns and limit applicability to new projects), they employ LLMs as multi-language feature extractors. This approach works across programming languages without specialized tooling, analyzing both quantitative metrics (code-level and change-level) and qualitative indicators (coding style violations, change type classification).

The framework was evaluated on two datasets: Prime Video's production environment and the public ApacheJIT dataset. The best-performing model achieved an average recall of 0.83 and F1 score of 0.81 for detecting risky code changes. Notably, ablation analysis revealed that change-level volume metrics (e.g., lines added/deleted) are noisy predictors, while structural code complexity provides a substantially stronger risk signal. These results demonstrate that thoughtful feature curation enables effective change risk assessment across different programming languages and organizational contexts while avoiding privacy concerns. The paper is available on arXiv (2607.06766) and represents a practical step toward replacing blanket deployment freezes with intelligent, risk-based change control—a win for developer productivity and service reliability at scale.