BOHM: Zero-cost attribution for compound AI systems from routing weights

Compound AI systems route tasks through hierarchies of specialized components, but attributing output to individual components has relied on Shapley-based methods (SHAP) that require evaluating the system on arbitrary subsets of components. That requirement fails for third-party APIs, opaque endpoints, and agentic orchestrators that concentrate routing on a few tools. Enter BOHM, a new method from researcher Joss Armstrong that extracts a hierarchical attribution tree directly from the routing weights these systems already maintain.

BOHM's key insight is that leaf attribution equals the path product of root-to-leaf routing weights, and level-k attribution is the induced distribution over depth-k nodes. This means zero marginal cost and no access to component internals. BOHM provides multi-resolution attribution at every level simultaneously — something flat methods like SHAP cannot offer at any evaluation budget. On 18 LLMs in a 3-level hierarchy over 880 LiveCodeBench problems, BOHM yielded Kendall tau=0.928, while SHAP reached tau=0.980 but required 9,000x more coalition evaluations per seed. On a 5-driver, 7-benchmark agentic study, drivers concentrated routing on a single tool (top-share median 0.65), and cell-level agreement between BOHM and SHAP predicted whether the driver's top pick was empirically best. On a US Census hierarchy with 475 leaves and 4 levels, BOHM recovered ground-truth rankings at every level (tau up to 0.722). BOHM satisfies efficiency, monotonicity, symmetry, and weak suppression but not Shapley's additivity — it is a complementary primitive for multi-resolution decomposition wherever routing state exists.