JouleShare framework cuts LLM energy attribution error 4x
Token-based GPU billing misses by 0.44 L1; JCalib slashes error to 0.116.
Batched LLM serving maximizes throughput but turns energy accounting into a mess — GPUs report power at the aggregate level, not per request. Existing benchmarks estimate energy by model, phase, or token count, but none provide measured request-level ground truth. That means the billing rules cloud providers and sustainability teams actually use have never been validated against a fair baseline. JouleShare, a new framework from Qi Luo and colleagues, fixes this by combining an offline harness with a learned calibration model.
The offline harness replays request subsets under vLLM using a reproducible protocol, integrates GPU power telemetry, and computes exact Shapley energy for each request — the gold-standard fair allocation. Then JCalib, a lightweight regression model, learns to predict those Shapley shares from cheap request features at serving time. Across 16 model/workload runs, token-proportional attribution deviated from exact Shapley by 0.440 normalized L1 under static batching and 0.458 under continuous batching. JCalib cut those errors to 0.116 and 0.177 respectively, outperforming even a standalone-measurement baseline that couldn't run online. The results prove token-based billing is a poor proxy for marginal energy, and that measured Shapley ground truth can calibrate low-cost features toward fair attribution.
- Token-proportional energy attribution shows 0.440 L1 error vs exact Shapley under static batching, 0.458 under continuous batching.
- JCalib calibration model reduces error to 0.116 static and 0.177 continuous across 16 workloads and 3 data-center GPUs.
- JouleShare preserves exact batch-energy efficiency while enabling fair per-request chargeback and sustainability reporting.
Why It Matters
Fair LLM energy billing is now feasible, enabling accurate sustainability reporting and equitable cost allocation at scale.