Benchmarking Quantum Computers via Protocols, Comparing IBM's Heron vs IBM's Eagle

A new research paper from Technion researchers Nitay Mayo and Tal Mor presents a significant advancement in quantum computing evaluation. Instead of relying on traditional gate-level metrics, their work applies a protocol-based benchmarking methodology that uses well-defined quantumness thresholds to assess whether processors can demonstrate practical quantum advantage. This approach provides a more transparent and intuitive measurement of real-world quantum capabilities, moving beyond theoretical performance to evaluate what these machines can actually accomplish.

To demonstrate their methodology, the researchers conducted a direct comparison between two generations of IBM quantum processors: the older Eagle architecture and the newer Heron architecture. Their 42-page analysis with 51 figures reveals substantial performance improvements in the Heron generation, offering concrete evidence of hardware advancement. The protocol-level assessment shows genuine operational strengths and limitations of both devices, providing valuable insights for researchers and developers working with quantum computing systems.

The research builds on previous work (arXiv:2505.12441) and represents an important step toward standardized quantum benchmarking. By evaluating performance at the protocol level rather than individual gate operations, this methodology offers a more realistic assessment of quantum computing progress. The findings help establish clearer expectations for what current quantum hardware can achieve and provide guidance for future hardware development priorities in the rapidly advancing field.