Moonshot AI's Kimi K3 hits 2.8T parameters on AWS

Moonshot AI’s Kimi K3 is a breakthrough 2.8 trillion parameter open-weight Mixture of Experts (MoE) model, released on July 27, 2026. The model leverages a differentiated architecture with Kimi Delta Attention (KDA), Gated Multi-Head Latent Attention (MLA), and Stable LatentMoE framework, distributing its massive parameter count across 896 experts (16 active per token). This design enables 1M token context windows and native multimodal (text + vision) capabilities, excelling in long-horizon coding, complex reasoning, and agentic workflows. The model’s weights are available on Hugging Face in MXFP4 (4-bit Microscaling Floating Point) format, optimized for memory efficiency and large-scale inference.

Deploying Kimi K3 requires purpose-built infrastructure, specifically AWS’s p6-b300 instances (ml.p6-b300.48xlarge) with 8 NVIDIA B300 Blackwell Ultra GPUs and high-bandwidth interconnects. AWS offers two deployment pathways: Amazon SageMaker HyperPod, which simplifies orchestration via a dedicated Inference Operator, and Amazon EKS, which consumes reserved GPU capacity through Flexible Training Plans or Capacity Blocks. Both approaches rely on vLLM’s inference container (vllm/vllm-openai:kimi-k3) for serving, supporting MoE architectures, tensor parallelism, and MXFP4 quantization. This integration ensures organizations can self-host Kimi K3 at scale while maintaining performance and efficiency.