Gemma 3 1B and Llama 3.2 1B top energy efficiency in new LLM GPU benchmark

New research from Philipp M. Zähl and Anika Hennig provides the first reproducible, hardware-level energy benchmark for locally deployed LLMs. Using an RTX 4060 Ti 16GB GPU and the Ollama inference engine, they sampled GPU power draw at 2Hz via nvidia-smi across a fixed prompt set for nine open-source models ranging from 1B to 7B parameters. The study tracks mean/peak power, total energy per prompt (J/prompt), energy per output token (J/token), and throughput (tok/s). Unlike typical benchmarks that only report accuracy, this work quantifies the real-world energy cost of running models on consumer hardware—a growing concern as privacy-conscious users and enterprises move inference on-premise.

Key results show that raw parameter count isn't the primary driver of efficiency. Gemma 3 1B and Llama 3.2 1B achieve the lowest energy cost at 0.56 J/token and 0.65 J/token respectively, while leading throughput at over 170 tokens per second. In contrast, the 7B Mistral model consumes up to 4.4x more energy per token than the most efficient 1B model. Surprisingly, Qwen3.5 2B exhibited abnormally high per-prompt energy due to extended internal reasoning, highlighting that token generation mode—such as chain-of-thought or reasoning traces—must be separated from standard output when measuring efficiency. For professionals deploying local LLMs, this preliminary study suggests that selecting a smaller, well-quantized model can dramatically reduce energy costs and carbon footprint while maintaining adequate performance for many tasks. It also underscores the need for standardized energy benchmarks alongside accuracy metrics, and the authors plan to expand testing to more hardware and models.