OAT tokenizer gives robot policies high compression and anytime action fidelity

Action tokenization maps continuous robot action chunks to discrete tokens, forming a key interface for modern visuomotor policies. Existing approaches either rely on analytical discretization that produces prohibitively long token sequences or learned latent tokenizers that lack structure, limiting compatibility with downstream policies. In a new paper, researchers from Harvard, Stanford, and other institutions identify three desiderata for action tokenization—high compression, total decodability, and an ordered token space—and introduce Ordered Action Tokenization (OAT), a learned tokenizer that satisfies all three. OAT uses a transformer with registers, finite scalar quantization, and ordering-inducing training mechanisms to discretize action chunks into an ordered sequence of tokens. By training each token prefix to decode into a valid action chunk, OAT places coarse control information in early tokens and uses later tokens to refine residual detail, yielding an anytime tradeoff between inference cost and action fidelity.

OAT is validated in two prevailing uses of action tokens: autoregressive policies that generate tokens for control, and token co-training policies that use token losses to shape the vision-language model context consumed by a flow-based action expert. Across three policy backbones and more than 60 tasks spanning five simulation benchmarks and real-world settings, OAT consistently delivers strong policy performance while offering significantly greater flexibility at inference time. This work promises to make robot learning more efficient and adaptable, allowing systems to trade off speed and precision on the fly without retraining.