R's 'rush' package brings async distributed computing with Redis shared state

R has long lagged behind Python in supporting modern asynchronous distributed algorithms, where workers share state rather than relying on a central controller. Marc Becker and Bernd Bischl introduce rush, an R package that fills this gap by providing a shared-state coordination layer built on Redis. Workers read and write task data through a Redis key-value store and execute their own loops independently, enabling patterns like asynchronous Bayesian optimization. The package offers a high-level API for managing task lifecycles, with sub-millisecond overhead per task, robust error handling that automatically detects lost workers, and efficient caching. This decentralized approach is now available to the R community, matching capabilities of Python frameworks like Optuna, DeepHyper, and Hyperopt.

rush optionally integrates with the mlr3 ecosystem, powering asynchronous optimization in the bbotk and mlr3tuning packages. The authors demonstrate its practical utility by implementing asynchronous decentralized Bayesian optimization (ADBO) and benchmarking it on hyperparameter optimization of LightGBM across four datasets using 448 concurrent workers. This showcases rush's ability to scale and handle expensive objective functions in parallel. For R users working in statistical modeling and machine learning, rush unlocks a new paradigm for distributed computing that was previously exclusive to other languages, potentially accelerating experimentation and tuning workflows.