Peer Identity Bias in Multi-Agent LLM Evaluation: An Empirical Study Using the TRUST Democratic Discourse Analysis Pipeline

A new empirical study by Juergen Dietrich systematically measures peer identity bias in the TRUST democratic discourse analysis pipeline, a multi-agent LLM system designed for deliberative tasks. The research crosses four model families with two anonymization scopes across 30 political statements, finding that single-channel anonymization produces near-zero bias effects because individual channels act in opposite directions and cancel each other out. This creates a misleading impression that identity bias is absent.

Only full-pipeline anonymization reveals the true pattern: homogeneous ensembles amplify identity-driven sycophancy when model identity is fully visible, while heterogeneous configurations show the reverse. One tested model exhibits baseline sycophancy two to three times higher than others and near-zero deliberative conflict on ideological topics, making it structurally unsuitable for pipelines requiring genuine inter-role disagreement. The study concludes that heterogeneous model ensembles are more robust, achieving higher consensus rates and lower identity amplification, and that full-pipeline anonymization is essential for valid bias measurement in quality-critical multi-agent LLM applications.