ExecuGraph boosts backend code accuracy by 22.5pp with multi-agent execution grounding
Six specialized agents validate code in a sandbox, slashing errors on complex benchmarks.
ExecuGraph is a new multi-agent framework that tackles the reliability problem in LLM-generated backend code. Instead of relying on a single pass, ExecuGraph uses six specialized agents coordinated by a typed directed workflow with a bounded retry budget, implemented on LangGraph with locally hosted models (Ollama) and an optional retrieval layer. Each piece of generated code is executed in a subprocess-isolated sandbox with a wall-clock timeout, ensuring runtime correctness.
Evaluated on a curated 30-problem DSA suite, HumanEval (n=64), and APPS-introductory, ExecuGraph showed significant gains. While on internal-30 the multi-agent setup was statistically indistinguishable from baselines, on HumanEval it edged ahead by +3.1 pp. The clearest signal came from cross-model testing: with DeepSeekCoder V2 Lite, graph-category accuracy improved from 57.5% (one-shot) to 80.0% (multi-full)—a striking +22.5 pp jump. The authors argue this supports a scaling hypothesis: multi-agent decomposition yields increasing returns as base-model capability grows. The framework's main contribution is methodological—a single codebase that can collapse into one-shot, execution-retry, or per-agent ablation conditions, enabling controlled measurement of each lever's marginal effect.
- Six-agent framework (Planner, Code Generator, Logical Reviewer, Evaluator, Optimizer, Explainer) coordinates via LangGraph with execution validation.
- On HumanEval, multi-agent outperforms single-agent by +3.1 percentage points.
- With DeepSeekCoder V2 Lite, accuracy jumps from 57.5% (one-shot) to 80.0% (multi-full), a +22.5 pp improvement.
- Includes subprocess-isolated sandbox with timeout and optional retrieval layer for algorithmic recall.
Why It Matters
Multi-agent code synthesis with execution grounding significantly boosts LLM backend reliability, especially as models scale.