Study reveals ML container issues: 10GB sizes, 71% wasted rebuilds

A new empirical study from researchers (Jebari, Ksontini, Barrak, Kessentini) dives deep into containerization practices in open source ML projects. Analyzing 1,993 ML-specific Dockerfiles, they found that containers are typically large—averaging 10.27 GB—and take nearly 9 minutes to build. A striking 44.4% of commits trigger a full rebuild, with 96.4% caused by changes to context files (e.g., code or data), not the Dockerfile itself. Worse, 71% of that rebuild work is redundant due to poor cache reuse, a massive inefficiency for teams iterating rapidly on ML workflows.

The study also categorized container roles: training, inference, and infrastructure. By examining stable projects, the researchers identified 7 recurring ML-specific Dockerfile refactoring patterns that improve build efficiency and shrink container size. These patterns include better layering, selective context copying, and multi-stage builds tailored to ML workflows. The findings are especially critical as ML models grow and deployment pipelines demand faster, leaner containers. For teams using Docker in ML, this research offers concrete, data-backed strategies to cut build times and reduce wasted cloud compute.