New Study Reveals Instructions Shape Language Production in AI Models

A recent study led by Andreas Waldis and co-authors explores the role of instructions in shaping the language production of AI models, as outlined in their paper 'Instructions shape Production of Language, not Processing.' The research emphasizes a production-centered mechanism that operates differently from traditional input processing. Through their experiments across five binary judgment tasks, the authors found that while task-specific information in input tokens remains stable, the same information in output tokens varies significantly and correlates strongly with the model's behavior. This suggests that instructions govern how language models generate output rather than simply dictate how they interpret the input.

The study further reveals that interventions targeting instruction flow can significantly impact model performance. For instance, blocking instruction flow to subsequent tokens reduces both the behavioral output and the information content in the output tokens, while blocking it only to input tokens has minimal effects. This asymmetry becomes more pronounced with larger model scales and enhanced instruction-tuning. The findings underscore the importance of jointly assessing model internals and behavior to fully understand AI capabilities, advocating for a more detailed analysis that distinguishes between input processing and output production.