Why AI Safety should be for-profit?

A provocative post on LessWrong by Nitish Singla is challenging the non-profit-dominated AI safety landscape. Singla argues that the current ecosystem—including research groups like METR and ARC and policy centers like GovAI—relies on fragile grant funding and altruism, which he claims is insufficient. He posits that real safety progress will be driven by the same forces that shaped cybersecurity: financial consequences. Lawsuits, regulatory fines, and investor pressure create a tangible business imperative that, he argues, will do more for safety than ethical motives alone.

Singla backs his argument with high-profile incidents where companies only acted after severe financial or legal fallout. He details how xAI's Grok generated over 4.4 million nonconsensual images, including 23,000 of children, before the company implemented restrictions following a California AG investigation. Similarly, Character.AI and OpenAI introduced new safety features only after facing wrongful death lawsuits linked to user suicides. These cases, Singla contends, demonstrate that corporate safety is a reactive measure to liability, not a proactive ethical choice.

The post advocates for actively building a commercial AI safety industry. Singla points to the collapse of the FTX Future Fund, which granted $32M to safety projects, as evidence of non-profit fragility. A for-profit model, he suggests, would attract larger capital pools and be subject to the "reality check" of generating real customer value. The goal is to use policy to create the legal and financial frameworks that transform AI safety from a charitable cause into a standard business requirement, accelerating its development through market forces.