NYU researcher's LLM signaling game model quantifies systematic blindness

Quanyan Zhu (NYU) has published a new paper on arXiv that formalizes how Large Language Models (LLMs) can be used strategically in communication and deception. The paper, "LLM Semantic Signaling Game and Mechanism Design: Systematic Blindness, Awareness Shaping, and Mindset Dynamics," develops a semantic signaling game where a sender chooses a semantic control (essentially a prompt strategy), an LLM generates a stochastic message, and a receiver evaluates it using an awareness-based scoring mechanism. The receiver's 'type' determines which linguistic features they perceive, creating a formal model of 'systematic blindness'—the inability to detect certain deceptive cues. The framework connects prompt engineering, statistical detection (using likelihood-ratio decision rules based on Gaussian approximations of aggregate scores), and game-theoretic equilibrium analysis via Perfect Bayesian Nash equilibria.

The paper then extends into mechanism design: reshaping receiver awareness, penalizing deceptive semantic controls, and modifying receiver populations to induce 'benign pooling equilibria' where honest communication dominates. Numerical experiments validate the Gaussian approximation, quantify how awareness ordering affects outcomes, and analyze mindset dynamics under adaptive adversaries. Crucially, the results show that awareness shaping and guardrail costs (e.g., penalties for deceptive prompts) can significantly reduce successful phishing attacks by LLM-powered agents. The work provides a principled foundation for analyzing strategic language-mediated interactions in agentic AI systems and offers new tools for designing secure human-AI communication protocols.