AutoThread speeds RL inference by 83.8% with adaptive thread tuning

Simulation-in-the-loop reinforcement learning systems often hit bottlenecks on the simulator side, not the model side. Thread configurations that work at one moment can become inefficient as workloads shift, causing resource contention and scheduling overhead. A new paper on arXiv (2608.06025) from Jiming Su and colleagues at what appears to be a Chinese defense research group tackles this problem with AutoThread, a hybrid adaptive thread-tuning method that dynamically adjusts thread counts during RL inference.

AutoThread's core insight is that the ratio of task execution time to scheduling time determines the optimal thread count. To predict this, the authors employ a Physics-Informed Neural Operator (PINO) as a thread-count predictor, constrained and guided by a finite-source M/M/1 queueing model. This combination gives fast, accurate estimates even as workloads change. The system also performs load-aware online fine-tuning to correct prediction errors and refine resource allocation in real time.

Experimental results show AutoThread delivers an 18.4% average speedup over static threading strategies, and achieves 1.7x and 1.8x the throughput of XGBoost-based and Reinforcer-based approaches, respectively. Compared to state-of-the-art methods, it reduces execution time by up to 83.8%. The code and dataset are open-sourced, making the approach reproducible for anyone battling similar bottlenecks in high-performance RL systems.