InternLM releases Intern-S2-Preview-397B, a 397B-parameter open-source model
This massive MoE model rivals GPT-4 in reasoning and coding tasks.
The InternLM team at Shanghai AI Laboratory has unveiled Intern-S2-Preview-397B, the latest iteration in their open-source large language model lineup. With 397 billion total parameters and a Mixture-of-Experts (MoE) architecture that activates only around 30% of them per token, the model balances massive scale with practical inference costs. Early benchmarks suggest it matches or exceeds GPT-4 on key evaluations like MMLU (knowledge understanding), HumanEval (code generation), and GSM8K (math reasoning), making it one of the most capable openly available models to date.
Intern-S2-Preview-397B is available on Hugging Face under the Apache 2.0 license, enabling unrestricted commercial use and fine-tuning. The release includes model weights, a detailed technical report, and inference scripts optimized for multi-GPU setups (e.g., 8x A100 80GB). This marks a significant step for open-source AI, providing researchers and enterprises with a GPT-4-class model that can be self-hosted and customized without vendor lock-in.
- 397B total parameters with MoE architecture, activating ~120B per token for efficiency
- Matches GPT-4 on MMLU (90.1%) and HumanEval (87.3%), surpassing Llama 3.1 405B
- Apache 2.0 license allows commercial use; weights and inference code available on Hugging Face
Why It Matters
Open-source AI now has a GPT-4-class model for self-hosting, enabling private deployment and customization.