InclusionAI releases 8B Ling-3.0-tiny MoE model
8B parameter model with 1.3B active experts hits 105 tokens/s on DGX Spark
InclusionAI has released Ling-3.0-tiny, a lightweight variant of the recently open-weighted Ling-3.0-flash model. This 8-billion-parameter mixture-of-experts (MoE) architecture activates just 1.3 billion parameters per forward pass, positioning it between 4B and 8–12B open models like Qwen and Gemma in downstream performance metrics.
The model’s efficiency shines in real-world throughput benchmarks: on an NVIDIA DGX Spark system, Ling-3.0-tiny achieves 100–105 tokens/second using FP8 quantization, while running at 86–90 tokens/second on an Apple M4 Pro MacBook. Memory footprint remains modest at ~8.34 GiB for an 8K context window, making it viable for laptops and edge devices without sacrificing raw speed.
- Ling-3.0-tiny is an 8B-parameter MoE model with only 1.3B active experts, balancing performance and size
- Achieves 100–105 tokens/sec on DGX Spark and 86–90 tokens/sec on M4 MacBooks with FP8 quantization
- Peak memory usage is ~8.34 GiB at 8K context, enabling local deployment on consumer hardware
Why It Matters
Brings near-flagship LLM performance to laptops and edge devices without cloud dependency