APIPilot validates LLM-inferred API dependencies, hitting 92.3% operation coverage

A team of Vietnamese researchers (Tu Nguyen, Thanh Nguyen, Huy Nguyen, Viet Nguyen, Tien N. Nguyen, and Vu Nguyen) has released a preprint for APIPilot, a framework that addresses a core flaw in LLM-based REST API testing: treating inferred dependencies as ground truth. Existing approaches generate test sequences from OpenAPI specs using LLM reasoning, but they rarely execute those sequences to verify that a producer endpoint truly creates a resource a consumer endpoint needs. This leads to spurious dependencies, missed feasible chains, and infeasible tests. APIPilot instead treats every LLM-inferred dependency as a hypothesis, validates it through concrete API executions, and only then builds a dependency graph for test generation. The graph enables coverage-aware workflows via bounded top-k traversal, cleanly separating semantic inference from sequence construction.

APIPilot also introduces response-driven refinement: after each test run, it analyzes runtime responses to update resource pools, adjust input-generation constraints, and prune or revise invalid dependency mappings. This closed loop grounds the LLM's reasoning in actual execution feedback, improving subsequent test iterations. In an empirical evaluation across 16 real-world REST API services, APIPilot achieved 92.3% operation coverage, up to 58.6% code coverage, and an 88.1% workflow execution success rate—outperforming both LLM-based and traditional testing baselines. It also uncovered 197 unique 5xx failures and specification-execution mismatches, demonstrating that execution-validated dependency inference is not just more reliable, but also surfaces real bugs. The paper is on arXiv (2608.17546) and submitted to ICSE 2027, signaling the approach is being positioned for top-tier software engineering venues.