Tactics for Denying Your Motivations, or Why Legibility is Expensive

A thought-provoking essay titled 'Tactics for Denying Your Motivations, or Why Legibility is Expensive' has gained traction on the rationality forum LessWrong. Authored by user Dentosal, the piece argues that many functional systems, including our own psychology, rely on participants not fully understanding their inner workings. The central thesis is that an accurate description of a system can be an attack on it, destabilizing the very mechanisms that allow it to operate. This concept, termed 'legibility,' becomes expensive to maintain because it requires building systems robust enough to withstand full adversarial scrutiny.

The essay draws parallels between personal defense mechanisms and institutional practices, citing examples like banks intentionally blinding themselves to the specific reasons for filing Suspicious Activity Reports (SARs) to maintain plausible deniability with customers. It outlines a hierarchy of defensive tactics, from simple counterattacks and pretending innocence to more advanced organizational strategies for 'manufactured innocence.' The post concludes by linking this to developmental psychology, suggesting that the highest stage of personal development involves running one's motivational systems in a 'self-adversarial-robust' way—a costly but resilient form of legibility.