New Trick Makes Small AI Models Smarter and Faster
Your phone's AI could get smarter without eating more battery.
A new method called RAPID improves knowledge distillation by selectively focusing on important example pairs instead of evaluating all possible relations. It separates a reliability-based target for the teacher’s knowledge from an adaptive proposal that decides which pairs to evaluate, using exact inverse correction to keep the estimates unbiased. In text classification experiments on AG News and SST-2, the reliability-gated relational target achieved the highest mean student accuracy, and RAPID outperformed the cross-entropy baseline. The results support the idea that target reliability and evaluation priority can be designed separately.
- RAPID teaches small AI models by focusing on the most important examples, not all of them.
- In tests, it improved accuracy on movie reviews from 87.3% to 88.7% — a solid jump for AI.
- This could lead to cheaper, faster AI that runs directly on your phone, saving battery and protecting privacy.
Why It Matters
Smaller, efficient AI means smarter apps on your devices with less cost, energy, and data sharing.