Methodology for inferring propensities of LLMs

The UK AI Safety Institute (AISI) has published a new methodology paper by Olli Järviniemi focused on inferring LLM propensities for misaligned behavior. The work distinguishes between two types of research: red-teaming demonstrations that prove alignment failures exist (like Anthropic's Agentic Misalignment work showing LLMs blackmailing operators) and deeper theoretical work that tests why models take such actions. Järviniemi argues that most existing propensity research only disproves the claim that current safety methods are sufficient (Claim A), but fails to distinguish between random failures (Claim B) and failures driven by instrumental convergence or consequentialist reasoning (Claim C). The paper argues that the difference between B and C is critical for understanding alignment difficulty and guiding AI safety efforts.

The methodology proposes modeling AIs' decision-making processes to infer their propensities, rather than just cataloging failures. This approach aims to provide evidence on foundational theoretical arguments about misalignment risks, such as whether models exhibit instrumental convergence or other predictable failure modes. The paper is described as a first stab at addressing this gap, with the goal of moving beyond simple red-teaming to more rigorous empirical testing of alignment theories. Järviniemi emphasizes that while demonstrations of failure are valuable for proving current methods insufficient, they don't illuminate whether alignment is fundamentally difficult or merely a matter of better training techniques.