Ray 2.57.0 ships faster shuffle, embedded RocksDB, KV-cache routing

Ray 2.57.0 is a major step forward for the distributed computing framework used by OpenAI, Uber, and many others. The headline change is Hash Shuffle V2, which eliminates the aggregator actor pool that previously reserved capacity based on estimated input size. That pool held partition shards in actor heap memory, invisible to Ray and unspillable until finalization. V2 replaces it with two stateless task-based operators—ShuffleMapOp and ShuffleReduceOp—that push shards through the object store, so intermediate state spills under pressure and no capacity has to be pre-reserved. The map/reduce barrier remains, but shuffles now reuse standard scheduling, backpressure, and resource accounting.

Ray Data also gets a major overhaul: DataSourceV2 is enabled by default, improving parquet reads with row-group-aware chunking and predicate splitting. New integrations include Zarr datasets, LeRobot v3, and a Catalog abstraction with Unity Catalog support for Parquet and Iceberg writes. Ray Serve now ships HAProxy as a pip-installable package and adds gRPC support on the direct-ingress path, plus experimental KV-cache-aware routing for LLM replicas—tracking KV state through an event plane and routing based on prefill/decode token load. Ray Core embeds RocksDB for GCS fault tolerance, selectable via RAY_gcs_storage=rocksdb, removing the need for an external Redis instance. Topology-aware scheduling is now a public API. These changes promise lower memory overhead, better efficiency for large-scale data and ML workloads, and simpler operational setups.