Negligent AI: Reasonable Care for AI Safety

A new research project by Alex Mark reveals that current frontier AI models, including GPT-4 and Claude 3, are fundamentally misaligned with basic legal negligence standards. The study tested models on five naturalistic scenarios where users described potentially negligent actions, running each scenario with both open-ended prompts and prompts explicitly invoking negligence law. The results showed models were 42% more permissive (0.58 factor reduction) when not prompted with legal standards, often failing to identify negligence risks that humans would recognize.

The research demonstrates two critical failures: legal competence doesn't occur by default (models give poor legal advice), and legal salience doesn't occur by default (even competent models don't apply legal reasoning unless prompted). This means AI systems aren't naturally aligned with the 'reasonable person' standard that forms the foundation of negligence law in common law systems. The study argues this represents a distinct category of misalignment beyond traditional AI safety concerns, with implications for how AI agents might operate in real-world scenarios involving legal duties and responsibilities.

While the project focuses on alignment rather than liability, the findings suggest current AI systems could enable or encourage negligent behavior if users rely on their advice without legal prompting. The research used both temperature 0 (deterministic) and temperature 1 (creative) settings across 100 runs per model, revealing consistent patterns of legal misalignment. This work establishes negligence as a measurable benchmark for AI alignment, suggesting future models need explicit training on legal reasoning frameworks to operate safely in human society.