Meta’s GRACE slashes ad retrieval latency by 11x
Meta’s new GRACE engine delivers real-time ad retrieval at 11x lower latency
GRACE, a serving system for ads generative retrieval, tackles real-time ad delivery challenges by ensuring generated ads meet targeting rules and by keeping latency and GPU costs within strict limits. It introduces Generative Target Matching (GTM), which lifts the final ad-level target matching pass rate from 23.55% to 40.42% over constrained decoding alone. On NVIDIA GH200, GRACE speeds up cross-attention by 68.0 times and self-attention by 23.4–25.8 times versus the faster of FlashAttention-2 and FlashAttention-3 baselines, reducing overall decoder latency by 11.1 times.
- GRACE improves ad eligibility pass rates from 23.55% to 40.42% using Generative Target Matching (GTM) with Semantic ID prefixes and Bloom filters
- Decoder latency drops 11.1x on NVIDIA GH200 by optimizing attention kernels, KV cache, and beam search for wide-beam, short-sequence regimes
- Cross-attention latency improves 68x and self-attention by 23.4–25.8x, enabling real-time ad retrieval at scale
Why It Matters
GRACE could revolutionize ad tech by enabling sub-second ad generation while cutting compute costs, critical for high-volume platforms like Meta’s ad systems.