New arXiv paper optimizes LLM-agent GPU usage by 43%

Cornell researchers Josef Liyanjun Chen have published an arXiv paper titled *Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control*, proposing a method to optimize GPU utilization for LLM-agent systems by minimizing redundant data transfers between GPUs and host systems.

The study formalizes the 'ready-cohort boundary' using metrics like fixed-partition share (F=30.19%), exact offline share (P*=43.00%), and online achieved share (U=45.85%) under a 50ms deadline for 100,000 active sessions. By keeping GPU-computed route decisions on-device instead of returning them to the host, the approach achieves 1.19x–2.39x speedups across 36 configurations. The paper validates its findings with 14.5M batched invocations and a reproducibility appendix, offering a new framework for GPU-efficient agent control.

The research highlights two key gates for GPU agent control: deadline-feasible cohort supply and observation placement, with implications for reducing CPU displacement and improving service-level benefits in distributed LLM systems.