Real-World Fault Detection for C-Extended Python Projects with Automated Unit Test Generation

A team of researchers from multiple universities has developed a novel method to automatically find critical bugs in Python libraries that use C-extensions. These extensions, common in performance-critical libraries like NumPy and Pandas, can cause the entire Python interpreter to crash when they fail, bypassing normal error handling. This crash halts traditional automated test generators, leaving dangerous faults undetected. The researchers' solution modifies the popular Pynguin test-generation tool to execute each generated test case in an isolated subprocess.

This 'subprocess-execution' approach means a crash in C code terminates only its own test subprocess, not the entire test generator. This allows the system to continue running, capturing the crash as a reproducible test case and moving on to test other parts of the code. The team evaluated their enhanced Pynguin on a new dataset of 1,648 modules from 21 major Python libraries with C-extensions. The results were significant: their method enabled testing of up to 56.5% more modules than a standard approach. It successfully detected 213 unique crash causes, which corresponded to 32 previously unknown, real-world faults in these widely-used libraries. The work is accepted for presentation at the IEEE ICST 2026 conference.