BatchDAG: LLM-planned DAGs slash costs 47x for enterprise analytics

Large language models excel at analyzing single documents but struggle with cross-entity, exhaustive analytical queries over enterprise-scale data due to context overflow, loss of per-entity attribution, and linear latency from sequential tool calls. BatchDAG, by Anupreet Walia, solves this by having an LLM generate a typed directed acyclic graph (DAG) of operations—SQL queries, semantic searches, in-memory transforms, parallel fan-outs, and single-shot analyses. A deterministic engine evaluates this DAG with topological-wave parallelism and structured JSON data flow. The key innovation is entity-aware batching: grouping rows by logical entity before fan-out reduces LLM calls by up to 47x. This turns a system that previously required multiple hand-engineered workflows into a single natural-language-to-execution pipeline.

In controlled experiments on 12 transcript-heavy queries, BatchDAG achieves a quality score of 3.74/5, comparable to an expert-designed pipeline (3.25/5) and significantly better than a ReAct agent (3.09/5, p<0.01). It also provides superior provenance with a 77% transcript evidence rate versus 46-60% for baselines. Using structured JSON intermediates reduces hallucinations by 27% compared to prose summaries. The planner maintains a 98.8% valid-DAG rate across 300 planning calls. In production at this http URL, BatchDAG processes queries over 50,000+ meetings in under 60 seconds, with per-query costs ranging from $0.02 to $0.24 at published GPT-5.1 pricing—making it a practical, scalable orchestration layer for enterprise data analysis.