RadixArk's Miles: Open-source framework scales LLM RL post-training
Miles unifies SGLang, Megatron-LM, and Ray for production RL at scale.
Miles addresses a critical challenge in post-training large language models: as models grow and transition to mixture-of-experts (MoE) architectures, RL post-training becomes a distributed systems problem. RadixArk built Miles as an open-source framework that composes SGLang (rollout generation), NVIDIA Megatron-LM (distributed training), Ray (cluster orchestration), and PyTorch (numerical layer) behind a small, pluggable trainer core. This design lets researchers customize rollout logic, reward functions, and loss functions without forking the framework.
Key architectural choices include Ray actors for every long-lived process (trainer ranks, SGLang servers, proxies) with GPU-aware placement supporting disaggregated or colocated layouts. Miles handles the rollout-training boundary carefully: weight synchronization via NCCL/RDMA, sample transfer via Ray tasks, and MoE-aware routing alignment. It ships with unified low-precision recipes, built-in observability and fault tolerance (checkpointing), making it suitable for long-running jobs on NVIDIA Blackwell/Hopper hardware. The goal is composable, reproducible, and scalable RL training for frontier-level LLMs.
- Composes SGLang for rollout, Megatron-LM for training, Ray for orchestration, and PyTorch for extensibility
- Pluggable trainer core lets users swap rollout logic, reward functions, and loss functions via Python modules
- Built-in support for MoE alignment, low-precision recipes, NCCL weight sync, and fault-tolerant checkpointing
Why It Matters
Simplifies building reproducible, production-grade RL post-training for the largest LLMs and MoE models.