llama.cpp research: local AI inference shifts power to hardware vendors and Hugging Face

A new arXiv paper, "Open at the Edge, Captured at the Center: llama.cpp and the Political Economy of Local AI Inference," examines the infrastructure that makes open-weight models runnable on user devices — a layer largely ignored in open AI scholarship. The authors, Woohyeuk Lee, Hanlin Li, and David Gray Widder, analyzed 7,681 merged pull requests from llama.cpp's repository (March 2023–March 2026), combined with repository discussions, corporate statements, and contributor blogs. Their mixed-methods study finds that local inference broadens participation at the execution layer, but capture shifts into the infrastructure itself: hardware backends (like CUDA and Metal), model integration labor, and core maintainers become chokepoints. The February 2026 absorption of llama.cpp by Hugging Face is presented as a defining example of this centralization.

The paper documents how control over local AI increasingly rests with hardware vendors (through backend-specific optimizations), model distributors (via format compatibility and weights hosting), and a small set of core maintainers — while model owners and individual contributors bear the cost of making models runnable. This dynamic undercuts the promise of open-weight models as a full counterweight to cloud dominance. The authors argue that preserving openness outside the cloud requires policy mechanisms that extend beyond model release conditions: systematic analysis of format dependencies and vendor influence, enforceable model compatibility requirements, and sustained public funding for inference tooling. The paper is 18 pages with 7 figures, available on arXiv (2608.19001) and cross-listed in Computers and Society (cs.CY).