Bayesian Sequential Verification for Budget-Aware Quantum Program Testing

Abstract: Quantum programs often produce probability distributions rather than deterministic outputs, making verification inherently statistical and increasingly costly on real hardware. In practice, developers still frequently rely on testing with fixed shot budgets on simulators, which are simple but time-consuming and poorly suited to noisy backends. What is missing is a verification approach that is both statistically explicit and budget-aware. This paper formulates Bayesian sequential verification as a reference-based Bayesian hypothesis testing workflow in which priors are derived from explicit reference sources, such as finite-shot reference runs or ideal/statevector-based computation, and verification decisions are updated batch by batch as measurement evidence accumulates. This approach is evaluated in Qiskit on two complementary workloads: Bell-state and QAOA-MaxCut. Across both case studies, the results show that Bayesian sequential verification can substantially reduce measurement costs compared to fixed-budget baselines when the success probability of the program exceeds the target threshold. The findings position Bayesian sequential verification as a practical verification workflow for quantum programs. The approach provides a foundation for future quantum continuous-integration pipelines that require reliable, budget-aware pass/fail decisions and motivates validation on real quantum hardware.

• The paper develops a reference-based Bayesian sequential verification workflow that updates Beta posteriors in 50-shot batches and stops after two credible-bound checks support a PASS decision.
• Experiments show up to 87.5% fewer Bell-state shots and up to 98.5% fewer easy QAOA shots, with equal or better PASS rates than matched fixed-budget testing when programs exceed targets.
• The results show savings disappear near decision boundaries and under noisy, low-probability predicates, highlighting the need for backend-specific thresholds, richer stopping rules, and real-hardware validation under drift.

Overview

This paper formulates quantum program verification as a budget-aware sequential decision problem, operationalizing Bayesian hypothesis testing (BHT) as a reusable verification workflow rather than proposing a new statistical method. The core observation is that quantum programs produce distributions rather than deterministic outputs, so verification must be statistical, and that prevailing practice—fixed shot budgets such as 10k shots per circuit—is statistically inefficient: it wastes shots when the program clearly meets its specification and may be underpowered when it does not. Given hardware costs on the order of $96 USD per minute for IBM quantum resources, the paper argues for a procedure that stops as soon as posterior evidence suffices. The contribution is implemented in Qiskit and evaluated on two case studies—Bell-state correlation verification and QAOA-MaxCut Top-$K$ verification—with artifacts released publicly for reproduction.

Method: reference-based Bayesian sequential verification

The verifier treats each program execution as inducing an unknown outcome distribution $p_B$ over bitstrings, and reduces correctness to a Bernoulli predicate $\phi:\mathcal{Y}\rightarrow\{0,1\}$ with unknown success probability $\theta$. A Beta prior $\mathrm{Beta}(\alpha_0,\beta_0)$ is constructed from an explicit reference source: either a shot-derived reference success rate from a noiseless simulator or an analytic/statevector-computed rate $\mu_0$, with prior strength $s=20$ pseudo-observations in all experiments. Measurements are collected in batches of $b=50$ shots, and the posterior updates in closed form via conjugacy. After each batch, once a minimum-shot gate ($n_{\min}$) is passed, the verifier computes a one-sided lower credible bound (LCB) via the inverse Beta CDF at tail probability $\delta$ and declares PASS when the LCB meets the target threshold $p_B$0 for $p_B$1 consecutive batch checks; otherwise it runs to the shot cap $p_B$2 and returns FAIL.

Two design choices deserve note. First, the pass-streak requirement guards against premature stopping on a single favorable batch. Second, the two prior sources are deliberately complementary: the Bell-state study uses empirical priors from AerSimulator, while the QAOA study uses exact statevector-derived priors, avoiding Monte Carlo variance in the prior itself. The paper assumes outcomes are conditionally i.i.d. and stationary within a run—an assumption it concedes does not hold under long-term hardware drift, which is explicitly out of scope.

Bell-state verification results

The first workload verifies a 2-qubit Bell state in both $p_B$3 and $p_B$4 bases, where success means observing only correlated outcomes $p_B$5. Across three IBM fake backends (FakeNairobiV2, FakeLimaV2, FakeManilaV2) and thresholds swept from $p_B$6 to $p_B$7 with 10 seeds per setting, the headline result is strong early stopping when the backend is comfortably above specification: at $p_B$8, all backends achieve 10/10 PASS with a median of 250 max-shots against a cap of 2,000—an 87.5% shot reduction with zero dispersion (IQR [250, 250]).

Behavior near the decision boundary is markedly different and backend-dependent. At $p_B$9, FakeNairobiV2 fails in all runs and exhausts the cap, while FakeLimaV2 and FakeManilaV2 still pass 10/10 with reductions of 71.25% and 58.75% respectively, but with wider IQRs reflecting threshold-adjacent variability. At $\phi:\mathcal{Y}\rightarrow\{0,1\}$0, nearly all configurations become budget-limited. The practical implication stated by the author is that regression workflows likely require backend-specific calibration of $\phi:\mathcal{Y}\rightarrow\{0,1\}$1 to balance assurance against test cost.

QAOA Top-$\phi:\mathcal{Y}\rightarrow\{0,1\}$2 verification results

The second workload is a stress test: an 8-qubit depth-$\phi:\mathcal{Y}\rightarrow\{0,1\}$3 QAOA MaxCut ring instance verifying Top-5 success mass, with ideal success probability $\phi:\mathcal{Y}\rightarrow\{0,1\}$4. Because Top-$\phi:\mathcal{Y}\rightarrow\{0,1\}$5 success is a low-probability event, the posterior LCB separates from the target much more slowly than in the Bell-state case.

On noiseless AerSimulator, all runs verify for $\phi:\mathcal{Y}\rightarrow\{0,1\}$6 using only 150–525 median shots, with degradation appearing only at $\phi:\mathcal{Y}\rightarrow\{0,1\}$7 (PASS 8/10, median 2,350 shots). Under realistic noise models the picture changes sharply: on FakeBrooklynV2 and FakeKolkataV2, verification becomes costly already at $\phi:\mathcal{Y}\rightarrow\{0,1\}$8 (PASS 7/10 and 6/10, medians of roughly 4,000–5,000 shots), and for $\phi:\mathcal{Y}\rightarrow\{0,1\}$9 most runs hit the 10,000-shot cap without passing. This confirms the intended role of the workload as a near-threshold stress test and indicates that low-probability predicates under noise may require more robust designs—larger $\theta$0, multi-threshold stopping rules, or backend-specific calibration—rather than the current single-predicate rule.

Matched fixed-budget baseline comparison

A key methodological strength is the matched baseline: both BHT and the comparator use the same success event, confidence level ($\theta$1), and decision target—for QAOA, $\theta$2—with the baseline evaluating the rule only at the full budget. This isolates the effect of sequential stopping rather than merely comparing against an arbitrary cap.

The comparison shows the advantage is not simply "stop earlier than the cap" but equal or better decisions at lower cost. Representative results:

Workload	Setting	BHT PASS	Fixed PASS	Median saving
Bell	FakeManilaV2, $\theta$3	10/10	10/10	1,750
Bell	FakeLimaV2, $\theta$4	10/10	10/10	1,425
QAOA	FakeBrooklynV2, $\theta$5	7/10	5/10	5,400
QAOA	AerSimulator, $\theta$6	10/10	10/10	9,850

Notably, at several settings BHT improves the PASS count itself (e.g., 7/10 vs. 5/10 on FakeBrooklynV2 at $\theta$7), because the sequential rule can accumulate evidence across batches whereas the fixed test commits to a single evaluation. Conversely, savings vanish entirely once runs become budget-limited near the boundary—at $\theta$8 on noisy backends, both methods return median savings of zero. The gains are therefore conditional on the gap between true success probability and target threshold, a conditionality the paper states plainly.

Limitations and open questions

The threats-to-validity section is candid on four fronts. Internally, stopping behavior depends on fixed hyperparameters (batch size, minimum-shot gate, streak requirement, prior strength, cap), and different choices could shift PASS rates near the boundary; the matched binomial baseline is also only one comparator among many possible frequentist or sequential alternatives. Construct validity is limited because both workloads reduce correctness to a single Bernoulli predicate, which does not capture richer notions of program quality. Externally, all experiments use AerSimulator and IBM fake backends; real devices introduce calibration drift, queue delays, and nonstationary noise that could undermine the stationarity assumption underlying the posterior updates—and the paper explicitly defers real-hardware validation due to cost and access constraints. Finally, the study is a proof-of-concept over two workloads, so conclusions about savings magnitude hold only relative to the reported budgets and baseline configuration. Open questions left by the paper include how to systematize prior selection and refresh across repeated CI runs, what fault models should be attached to FAIL outcomes, and whether the stopping rules remain reliable under nonstationary hardware behavior.

Conclusion

This paper contributes a concrete, reproducible workflow that replaces fixed-shot quantum testing with reference-based Bayesian sequential verification, evaluated on two qualitatively different predicates and benchmarked against a carefully matched fixed-budget baseline. The evidence supports large shot savings (up to ~87.5% for Bell-state, up to ~98.5% for easy QAOA settings) when programs are comfortably above specification, equal-or-better PASS rates at moderate thresholds, and honest conservatism near the boundary. Its main unresolved dependency is validation on real hardware, where drift and nonstationarity may interact with the i.i.d./stationarity assumptions of the posterior update—the natural next step toward budget-aware quantum continuous-integration pipelines.