MoECodec uses Mixture-of-Experts to unify image compression for humans and machines
Dynamic token-level routing boosts downstream vision tasks without retraining separate codecs.
Image compression has traditionally optimized for human perception, but with the rise of AI-powered applications, codecs must now also serve machine vision tasks like object detection and segmentation. Existing solutions either train separate models per task or apply heuristic transfer learning, both leading to static computation that ignores the varying semantic importance of different image regions. To address this, researchers from a leading institution (authors: Zhao, Ji, Zhan, Wan, Zheng) propose MoECodec, a transformer-based compression model that introduces Mixture-of-Experts (MoE) at the token level.
MoECodec replaces the feed-forward networks in each transformer block with a set of expert modules. A stable routing strategy—expert-choice routing combined with spatial total variation regularization—ensures that tokens are assigned to experts in a spatially coherent way, adapting computation to local image complexity. A lightweight Group Shuffle MLP (GShMLP) architecture keeps parameter growth manageable. The result is a single unified codec that simultaneously delivers high-quality reconstruction for humans and strong performance on object detection, instance segmentation, and image classification. Experiments on standard benchmarks show consistent improvements over task-specific and transfer-based baselines, demonstrating the viability of dynamic, token-aware compression for joint human-machine perception.
- Replaces static FFN layers with token-wise Mixture-of-Experts (MoE) for dynamic computation per image region.
- Employs expert-choice routing + spatial total variation regularization to enforce spatially coherent expert assignments.
- Introduces Group Shuffle MLP (GShMLP) to control parameter growth while maintaining expressive power.
Why It Matters
One model handles compression for both humans and machines, reducing deployment overhead and improving machine vision accuracy.