Researcher unveils Engine-Transfer-Bench for LaTeX engine selection
ETB tests 6 engines across 1,784 docs with revealing stability and speed insights...
Independent researcher Prajwal S. Venkateshmurthy has introduced Engine-Transfer-Bench (ETB), a first-of-its-kind benchmark designed to objectively evaluate and compare six leading document compilation engines: pdfLaTeX, XeLaTeX, LuaLaTeX, Tectonic, Typst, and pandoc PDF backends. The benchmark comprises 1,784 open documents and assesses engines across four critical dimensions—reliability, latency, text consistency, and failure modes—using a standardized harness deployed on GitHub Actions.
ETB's findings reveal stark contrasts in performance stability. Across 4,211 compilation runs on macOS, Ubuntu, and Windows, Tectonic demonstrated exceptional consistency with a success rate stable within 0.9 percentage points (96.3–97.2%). In contrast, classic TeX Live-style engines showed variability ranging from 12–20 percentage points depending on distribution policy (e.g., Ubuntu apt, MiKTeX auto-install, or macOS BasicTeX). For 702 portable LaTeX documents, all engines succeeded 100% of the time, positioning latency as the primary selection criterion. Failures were traced to architectural issues—such as font, layout, and asset handling—rather than missing packages, and concentrated within 107 engine-specific templates.
- ETB evaluates 6 engines (pdfLaTeX, XeLaTeX, LuaLaTeX, Tectonic, Typst, pandoc) across 1,784 documents and 4,211 GitHub Actions runs on macOS, Ubuntu, and Windows.
- Tectonic achieves 96.3–97.2% success rate with only 0.9pp variance, while TeX Live-style engines vary by 12–20pp depending on distribution policy.
- ETB-Porta, a companion recommender and portability gate, is released alongside the benchmark as open infrastructure.
Why It Matters
ETB delivers data-driven engine selection for LaTeX users, reducing compile failures and portability headaches in production workflows.