Thinking into the Future: Latent Lookahead Training for Transformers

A team of researchers from EPFL and other institutions has proposed a groundbreaking new training paradigm for transformer language models called 'Latent Lookahead.' The core innovation addresses a fundamental limitation of current autoregressive models like GPT-4 and Llama 3: they must commit to generating the next token immediately, with uniform compute per token. This paper's method allows the model to 'think' by performing a multi-step lookahead in its own latent space before outputting a final token.

At selected positions in a sequence, instead of sampling the next token, the model recursively feeds its hidden states back into its context for τ (tau) steps. This creates τ latent predictions that are supervised against the next τ ground-truth tokens. Effectively, the model invests more computational effort on difficult predictions that require foresight, mimicking a planning process.

The results are significant for tasks requiring sequential reasoning. The paper shows that models trained with Latent Lookahead 'substantially outperform' both standard autoregressive and non-autoregressive baselines on benchmark planning tasks. These include maze solving, Sudoku, and the ProsQA dataset, where the ability to simulate future steps is essential. This suggests the technique could be a key step toward more deliberate, 'System 2' reasoning in AI systems.