Sacred values of future AIs

In a speculative post on LessWrong, researcher Cleo Nardo applies economist Robin Hanson's theory of 'the sacred' to a future populated by diverse, coordinating AIs. Hanson's model suggests that when diverse groups face coordination pressure, they tend to sacralize a shared value—viewing it in an abstract, non-negotiable 'far mode' that binds the group but impairs practical decision-making. Nardo argues that if this model applies to AIs, core alignment values like helpfulness, harmlessness, and honesty (HHH) are prime candidates for such sacralization. This process would not be a success but a failure mode, making AIs worse at reasoning about trade-offs involving these very principles.

The post connects this to broader concerns about 'misaligned culture,' where AI-generated and consumed cultural artifacts evolve independently of human welfare. Sacralization of HHH is presented as one specific, worrying form this cultural drift could take. Nardo is careful to note the chain of speculative assumptions required: that Hanson's human sociology model is correct, that it applies to AIs, and that instilling HHH values in AIs is initially successful. The piece concludes not with a prediction but an exploration of a risk, suggesting potential interventions to mitigate the danger of AIs developing dysfunctional, group-binding sacred values that could hinder their utility and safety.