Structured output reduces answer diversity across 44 LLMs by 23%

A new paper by Tapan Parikh, "Structured Output Collapses Answer Diversity Across 44 Language Models," reveals a significant, systematic bias introduced when models are asked to return answers in structured formats like JSON. Re-running the One-Word Census with 31 open-ended category prompts across 44 models, the author finds that simply appending "Reply with JSON only" dramatically reduces response diversity. The modal answer increases from 41% to 64% of the pool, distinct answers fall from 52 to 36, and mean answer-choice surprisal drops from 1.80 to 1.58 bits. Notably, six of the most distinctive models shift toward the majority, while less diverse models remain unchanged.

The effect is specific to formats models are trained to output—JSON and XML show significant compression, while YAML and CSV do not, and an arbitrary bracket wrapper actually increases diversity. Importantly, enforcing the schema at the decoder (response_format) does not further compress responses, meaning the homogeneity is driven by the model's response to the register itself, not by post-processing. As AI is increasingly consumed programmatically via structured outputs, this study suggests that the surface developers interact with is measurably more homogeneous than the chat surface on which models are evaluated and compared.