New tool dille catches silent ML faults with 91% precision

Machine learning pipelines often harbor silent semantic faults—like imbalanced datasets or misconfigured hyperparameters—that degrade model quality without throwing errors. These issues typically surface only after expensive training cycles, wasting time and compute. A new paper from Linköping University proposes a data-informed static analysis technique that catches these faults before a single epoch runs. The method extracts the ML pipeline (using the popular Random Forest classifier) into a directed acyclic graph and evaluates it against formalized API contracts covering structure, data properties, and hyperparameters. By relying on aggregated data properties, the analysis works even when datasets are confidential and inaccessible.

The technique is implemented in an open-source tool called dille. Evaluated on 200+ real-world Kaggle notebooks, dille identified relevant semantic faults with 91% precision and sub-second runtime overhead. The empirical study found that 12% to 18% of existing Random Forest notebooks are affected by such faults, underscoring the practical need for lightweight, pre-training validation. The authors highlight that dille can be integrated into IDEs, agentic workflows, and CI pipelines, helping engineers catch problems early without disrupting their workflow. This approach shifts ML debugging left, saving significant resources and reducing the frustration of silent model degradation.