New PPT method steers transformers without backprop by optimizing latent posterior

A new paper from Garrett Baker, Vinayak Pathak, Daniel Murfet, and Susan Wei introduces Posterior Prefix Tuning (PPT), a method that steers the behavior of Bayes-filtered transformers (BFTs) without any backpropagation through the model. The key insight leverages the latent posterior model: in BFTs, the next-token distribution arises from a posterior over latent predictive models conditioned on context. PPT optimizes a distribution over hard prompts to elicit continuations that maximize expected utility. The gradient of this objective is estimated by drawing prior samples once via predictive Monte Carlo and then applying importance sampling — no transformer forward passes are needed after that initial sample set.

PPT is validated on two types of BFTs (Beta-Bernoulli and reinforced urn) across three utility families: reverse cross-entropy, frequency matching, and Dyck validity. The method achieves effective steering with a critical efficiency gain: because the prior samples are utility-independent, a single set of samples can drive elicitation for any number of utilities at negligible marginal cost. This contrasts sharply with traditional fine-tuning or prompt optimization that requires repeated backpropagation. While currently limited to BFTs where the posterior is exact, the approach opens a new paradigm for model steering that decouples inference from optimization.