Belayer cuts LLM agentic RL recovery time by 42x with selective GPU reuse

LLM agents trained via reinforcement learning in long-horizon, sandboxed environments face a unique challenge: a single trajectory can span many rounds of generation and environment interaction, and failures in stateful containers can discard completed work or expose the model to inconsistent states. Existing systems lack efficient recovery mechanisms for this distributed execution model. Belayer, a new system from Jiecheng Zhou and colleagues, tackles both rollout engine and environment failures head-on.

Belayer equips each pre-initialized shadow worker with a selective GPU-state reuse protocol. On failure, it performs owner and GPU health checks, then retains independently owned weights and raw KV-arena allocations while rebuilding request-specific KV contents from logged token prefixes. This avoids a full engine cold start. For environment failures, Belayer uses full checkpoint and full restore to capture container filesystem and runtime state, coordinating the recovered environment with the LLM context to preserve prefix consistency. An adaptive policy overlaps full-state checkpointing with natural LLM inference bubbles when the predicted interval is long enough. Empirically, Belayer shows low overhead during failure-free training, up to 42x faster worker recovery, and 1.5-3.5x faster environment-recovery time.