New BDTS framework slashes AI trace tokens by 87%

A new arXiv paper proposes Budgeted Dynamic Trace Structures (BDTS), a data-structural framework designed to maintain rooted trace graphs and append-only histories under a strict token or byte budget. Developed by Faruk Alpay and Levent Sarioglu, BDTS combines techniques like status-filtered reachability, soft-capped recency logs, and summary-plus-suffix compaction to compress long sequential traces without losing critical information. The goal is to enable AI models (especially LLMs) and debugging systems to process extensive execution traces efficiently on memory-constrained hardware.

In experiments with synthetic traces of 10k–40k vertices, the Rust implementation builds graphs in 0.58–2.72 ms and enumerates all descendants in 0.24–1.42 ms. Most notably, when applied to three public model targets (e.g., GPT-2, OPT-125M), BDTS reduces trace token counts from 3,359–3,360 to just 432–433 tokens—a compression ratio of nearly 8x. This breakthrough directly addresses token limits in AI inference pipelines and opens the door for more scalable trace-driven analysis in distributed computing and agentic workflows.