When to Think Fast and Slow? AMOR: Entropy-Based Metacognitive Gate for Dynamic SSM-Attention Switching
This hybrid AI architecture could make transformers 5x more efficient overnight.
Deep Dive
Researchers have unveiled AMOR, a new hybrid AI model that dynamically switches between a fast State Space Model (SSM) and a slower, more precise attention mechanism. Inspired by human 'fast and slow' thinking, it uses prediction entropy to decide when to engage costly attention. In tests, it achieved perfect accuracy on retrieval tasks while using attention on only 22% of positions, a 78% reduction in the most expensive compute operation.
Why It Matters
This breakthrough could dramatically lower the cost and energy required to run powerful AI models, making them more accessible.