Meta's AI Update Fixes a Bug That Slowed Down Big Models
Your AI assistant could get faster and cheaper thanks to this behind-the-scenes fix.
Deep Dive
A llama.cpp change (PR #31081) drops the `-sm tensor` option for the architectures that build the amax graph. The reason: the amax POOL_1D reduction spans the whole matmul row, so a device can't compute it from an axis-0 split. The Meta split-state case was reported as MIRRORED, which hands each device a partial max of its own slice — so that case is removed, per the article.
Key Points
- Meta fixed a bug that could cause AI models to give wrong answers when running on multiple chips.
- The fix ensures that large AI models, like those used in Facebook and Instagram, work correctly.
- You might not notice, but your apps will be more reliable and less prone to weird errors.
Why It Matters
More reliable AI means fewer glitches in apps you use daily, like social media and translation tools.