ConcoLixir: LLM discovery oracles boost Python test coverage 17 points

Concolic testing combines concrete execution with symbolic constraint solving, but Python programs often hit limitations when library calls downgrade symbolic variables, or when complex operations like regex, checksums, or parsers resist solver analysis. The paper introduces ConcoLixir, a reactive LLM extension for Python concolic execution that acts as a discovery oracle rather than a solver replacement. It generates initial seeds, proposes concrete inputs after solver failures, and targets uncovered code when coverage plateaus. Each candidate is executed concolically, with observed coverage and path constraints guiding subsequent exploration.

Across benchmarks including synthetic, real-world, and library targets, ConcoLixir boosted mean line coverage by 8.6, 15.1, and 17.0 percentage points over the baseline concolic tester without an LLM oracle. The largest gains occurred near semantic and library boundaries. Crucially, the full evaluation cost only $1.63 in API charges, demonstrating that bounded LLM usage can complement symbolic reasoning without replacing it. This approach offers a practical, low-cost way to enhance automated software testing for Python projects.