New paper defines 'agent harness' with clear test for 6 tools like Claude Code

The term 'agent harness' has become ubiquitous but ambiguous in generative AI software engineering—sometimes referring to products like Claude Code, sometimes to evaluation scaffolds like SWE-bench, and often conflated with frameworks or SDKs. Sanderson Oliveira de Macedo addresses this by systematically analyzing the concept's history, from the literal horse's tack to classic test harnesses in software testing, then through machine-learning evaluation harnesses, and finally to today's agentic systems. The result is a constitutive definition with clear necessary and sufficient conditions for a system to qualify as an agent harness, along with an operational inclusion/exclusion test.

The definition is validated against six real-world implementations (Claude Code, Codex CLI, Aider, Cline, OpenHands, and SWE-agent) and deliberate edge cases, including frameworks (LangChain), SDKs (OpenAI Agents SDK), and IDE plugins (GitHub Copilot). The test consistently includes only the harnesses and excludes the others. The paper closes with a research agenda organized around design tension axes, such as breadth vs. depth of agent capabilities, and tight vs. loose coupling to the underlying model. For professionals building or evaluating AI coding agents, this work provides the missing shared vocabulary needed to compare tools rigorously and guide engineering decisions.