A volumetric framework for quantum computer benchmarks
Robin Blume-Kohout, Kevin C. Young
Abstract
We propose a very large family of benchmarks for probing the performance of quantum computers. We call them {volumetric benchmarks} (VBs) because they generalize IBM's benchmark for measuring quantum volume \cite{Cross18}. The quantum volume benchmark defines a family of {square} circuits whose depth <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>d</mml:mi></mml:math> and width <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>w</mml:mi></mml:math> are the same. A volumetric benchmark defines a family of {rectangular} quantum circuits, for which <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>d</mml:mi></mml:math> and <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>w</mml:mi></mml:math> are uncoupled to allow the study of time/space performance trade-offs. Each VB defines a mapping from circuit shapes -- <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo stretchy="false">(</mml:mo><mml:mi>w</mml:mi><mml:mo>,</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math> pairs -- to test suites <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mi class="MJX-tex-caligraphic" mathvariant="script">C</mml:mi></mml:mrow></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mi>w</mml:mi><mml:mo>,</mml:mo><mml:mi>d</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math>. A test suite is an ensemble of test circuits that share a common structure. The test suite <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mi class="MJX-tex-caligraphic" mathvariant="script">C</mml:mi></mml:mrow></mml:mrow></mml:math> for a given circuit shape may be a single circuit <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>C</mml:mi></mml:math>, a specific list of circuits <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo fence="false" stretchy="false">{</mml:mo><mml:msub><mml:mi>C</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x2026;</mml:mo><mml:msub><mml:mi>C</mml:mi><mml:mi>N</mml:mi></mml:msub><mml:mo fence="false" stretchy="false">}</mml:mo></mml:math> that must all be run, or a large set of possible circuits equipped with a distribution <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>P</mml:mi><mml:mi>r</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>C</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math>. The circuits in a given VB share a structure, which is limited only by designers' creativity. We list some known benchmarks, and other circuit families, that fit into the VB framework: several families of random circuits, periodic circuits, and algorithm-inspired circuits. The last ingredient defining a benchmark is a success criterion that defines when a processor is judged to have ``passed'' a given test circuit. We discuss several options. Benchmark data can be analyzed in many ways to extract many properties, but we propose a simple, universal graphical summary of results that illustrates the Pareto frontier of the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>d</mml:mi></mml:math> vs <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>w</mml:mi></mml:math> trade-off for the processor being benchmarked.