AI agent autonomously designs compression, slashes model memory 75%

A new paper from Jiangwei Zhang et al. introduces a physically grounded multi-agent discovery engine that autonomously architects hardware-compliant computing systems. Traditional AI agents lack physical grounding and often hallucinate designs incompatible with real hardware. To solve this, the framework uses an Evolutionary Knowledge Graph that structures past innovations, paired with an algorithmic Chain-of-Thought to transform random search into directed evolution. The engine focuses on foundation model deployment as an extreme testbed.

The engine evolved two hardware-aware compression methods: Q-Enhance for mitigating long-context accuracy loss in dense models, and MoE-Salient-AQ, which outperforms state-of-the-art manually designed sparse Mixture-of-Experts (MoE) by 3.7% at sub-3-bit precision. Using a bandwidth-efficient Sensitivity Profile, the team deployed a massive 235-billion-parameter model onto a constrained dual-A100 server, reducing memory requirements by 75% while incurring only a marginal 0.64% accuracy degradation. This scalable hardware-software co-design paradigm enables machine-driven discovery within strict physical boundaries, potentially revolutionizing how large models are deployed on limited hardware.