Tencent Releases Hy3 preview - Open Source 295B 21B Active MoE
A massive 295B-parameter MoE model, but only 21B active per token, rivals GPT-4...
Tencent has open-sourced Hy3-preview, a massive 295B-parameter Mixture-of-Experts (MoE) language model that activates only 21B parameters per token. This sparse architecture dramatically reduces inference cost and latency while maintaining the capacity of a much larger dense model. The weights are available on Hugging Face under the Apache 2.0 license, enabling researchers and developers to fine-tune for specialized domains.
Early community benchmarks suggest Hy3-preview achieves competitive results against proprietary models like GPT-4 and Claude 3.5 on reasoning and coding tasks. The model's 21B active parameters make it feasible to run on high-end consumer GPUs (e.g., 4× RTX 4090 or A100), opening the door for local deployment. This release signals Tencent's aggressive push into open-source AI, challenging Meta's Llama 3 and Mistral's Mixtral line.
- 295B total parameters with only 21B active per token via MoE architecture
- Open-sourced on Hugging Face under Apache 2.0 license
- Benchmarks show GPT-4-level performance on reasoning and coding tasks
Why It Matters
Tencent's Hy3 preview democratizes GPT-4-class AI, enabling local deployment and custom fine-tuning for enterprises.