DataFlow-Harness lets LLM agents build editable data pipelines with 93.3% pass rate

Large language models are increasingly used to automate data-processing workflows, but coding agents typically produce scripts that cannot be easily persisted or edited as platform-native artifacts—a gap the authors call the NL2Pipeline gap. To solve this, Runming He and six co-authors from an unnamed institution (paper submitted July 2026) present DataFlow-Harness, a grounded code-agent platform. Instead of generating free-form Python scripts, the platform guides an LLM agent to construct directed acyclic graphs (DAGs) through typed, incremental mutations. The system integrates three components: DataFlow-Skills for procedural guidance, a Model Context Protocol (MCP) layer that exposes the live operator registry and pipeline state, and DataFlow-WebUI, which synchronizes conversational authoring with a visual DAG editor. This architecture ensures that every pipeline built by the agent is automatically materialized as a persistent, editable artifact.

On a 12-task data-engineering benchmark, DataFlow-Harness achieves a 93.3% observed end-to-end pass rate. Compared to vanilla Claude Code, it reduces measured monetary cost by 72.5% and generation latency by 49.9%. Its pass rate is within 0.9 percentage points of the Context-Aware Claude Code baseline, while cost is 42.8% lower. Per-task analysis shows that the Skills component is most valuable when pipeline construction depends on implicit procedural knowledge. These results demonstrate that live platform grounding—combining a structured mutation protocol with a visual interface—can produce persistent, editable workflow artifacts with reliability approaching that of script-generation baselines, but at significantly lower cost and latency. The work has implications for productionizing LLM-based data engineering.