Unix Tools and the FITO Category Mistake: Crash Consistency and the Protocol Nature of Persistence

Computer scientist Paul Borrill's groundbreaking arXiv paper, 'Unix Tools and the FITO Category Mistake: Crash Consistency and the Protocol Nature of Persistence,' delivers a formal proof that the Unix filesystem abstraction is fundamentally broken. The research demonstrates that common tools like `ls`, `cp`, `mv`, and `rename` present a false illusion of atomic state transitions—a Forward-In-Time-Only (FITO) assumption that constitutes a 'category mistake' permeating every layer from ext4 journaling and NVMe device behavior down to x86-64 CPU architecture. Borrill proves that no syscall-based persistence primitive can define a reliable commit boundary under failure, because a syscall's return value is consistent with multiple, materially different persistence states across Linux filesystems.

This structural flaw has catastrophic real-world consequences documented in the paper's appendix. The recursive chain of non-atomic dependencies causes cross-layer 'temporal assumption leakage,' leading to cascading cloud outages at Google, AWS, Meta, and Cloudflare driven by retry amplification. It results in database corruption in PostgreSQL, etcd, and MySQL; silent data corruption at CERN and NetApp; and AI training waste consuming 12–43% of compute budgets at scale. Most critically, financial system failures traceable to this single cause total billions of dollars annually. The paper argues that systems designed around the FITO assumption compensate with retry-and-recover protocols that paradoxically amplify the very failures they attempt to mask, calling for a fundamental rethinking of persistence as a protocol rather than a state.