A Penalty-Free Pipeline for Direct Quantum-Annealer Portfolio Optimization

Abstract: Direct quantum-annealer portfolio optimization is commonly formulated as a penalty-encoded QUBO and submitted to D-Wave hardware. We show that this standard formulation fails on current devices and identify the structural reason: the cardinality penalty contributes a dense rank-one term proportional to the all-ones matrix that makes the logical interaction graph complete regardless of the covariance structure. On Pegasus and Zephyr, chain-break fractions reach 83 percent at N equal to 24 and 92 percent at N equal to 49, producing no feasible samples. Attempting to fix this through topology-aware sparsification reveals a second problem: any sparsifier that removes off-diagonal entries also dilutes the cardinality constraint, so raw samples remain infeasible even when chains no longer break, and an ablation shows that for structurally favorable cases such as betting with settlement-graph priors the classical feasibility projector alone explains the result rather than the QPU. We propose dropping the penalty entirely: build an objective-only QUBO from the expected returns and the risk-scaled covariance, sample it on hardware, and enforce the cardinality constraint classically as a post-processing step. On D-Wave Advantage and Advantage2 for equities up to N equal to 49 and betting up to N equal to 48, mean chain-break fractions per sample averaged over reads drop from the range of 71 to 92 percent down to at most 0.04 percent. The QPU returns lower-energy feasible portfolios than the greedy heuristic on betting at N equal to 39 and 48, which is an energy comparison and not a proof of optimality, and the equity post-processed regret is at most 0.03 percent at all tested scales. These results establish that the penalty encoding, not the sparse hardware topology, is the binding constraint for direct QPU portfolio optimization at currently accessible scales.

• The paper shows that penalty encoding in portfolio optimization creates dense QUBOs that lead to high chain-break fractions on quantum annealers.
• The study rigorously compares sparsification methods and reveals that while they reduce chain lengths, they weaken constraint fidelity, necessitating classical post-processing.
• The proposed penalty-free pipeline, by removing the dense penalty term and enforcing constraints classically, delivers viable portfolios with significantly improved embedding and performance.

Penalty Encoding as the Structural Bottleneck in Quantum-Annealer Portfolio Optimization

Background and Motivation

The application of quantum annealing to combinatorial portfolio optimization typically proceeds by encoding the cardinality constraint (exact-$K$ asset selection) into a QUBO via a quadratic penalty term: $A(1^\top x - K)^2$. This induces a dense rank-one matrix, $A11^\top$, regardless of the underlying objective's sparsity. Previous literature has largely overlooked the embedding cost this penalty imposes given the bounded-degree hardware connectivity of quantum annealers such as D-Wave's Pegasus and Zephyr topologies. The present work rigorously diagnoses this structural failure mode, empirically dissects sparsification as a putative remedy, and demonstrates that penalty-free pipelines yield superior outcomes under practical scale constraints.

Structural Diagnosis of QUBO Failure

The QUBO associated with cardinality-constrained portfolio selection, commonly written as $$Q = -\mathrm{diag}() + \lambda \Sigma + A11^\top - 2AK\,I$$ (with $A$ a penalty weight), embeds a complete logical graph due to the penalty. Embedding dense logical graphs onto sparse hardware topologies necessitates long chain qubits, which break frequently during annealing and thus degrade the fidelity of returned solutions. Empirical results on D-Wave hardware at up to $N = 49$ confirm this: chain-break fractions approach 83–92%, with feasible sample rates collapsing to zero for all tested QUBO sizes.

Figure 1: Penalty encoding transforms sparse objective interaction graphs into fully connected QUBOs, forcing long chains for embedding that break at high rates during annealing.

For both equity and sports betting datasets, dense QUBOs require substantial qubit overhead and exhibit monotonically increasing chain lengths with $N$. Embedding is still possible at the tested scales, but the penalty encoding, not hardware topology limitations, is revealed as the primary constraint on solution quality.

Figure 2: Comparison between the natural settlement graph in betting instances (left, 3-cliques per match) and the complete logical graph induced by the penalty (right, $K_9$), which drastically increases edge count and embedding complexity.

Sparsification: Remedy and Failure Mode

In response to the embedding bottleneck, topology-aware sparsification is a natural response: zeroing out weaker objective interactions to produce a sparse QUBO, facilitating short chains and robust embedding on sparse hardware. Four families are studied: threshold, top-$k$, domain-prior, and domain-prior with residuals. Mean chain lengths are reduced to unity, and chain-break fractions are almost zero across all scales.

Figure 3: Mean chain length versus problem size for both dense and best-sparse QUBOs; only sparsified QUBOs maintain unit chains across tested $N$.

However, all sparsification methods inadvertently dilute the penalty term (since it is dense), weakening enforcement of the cardinality constraint. Empirically, raw QPU samples from sparsified QUBOs overwhelmingly fail to satisfy $A(1^\top x - K)^2$0, typically yielding all-ones vectors. Thus, sparsification solves the embedding problem but does not restore constraint fidelity. Feasibility must be enforced in a post-processing classical step, which dominates solution quality in favorable settings such as sports betting (block-diagonal covariance).

Post-Processing and Role of Classical Projection

Greedy feasibility projection (backward elimination to guarantee exact-$A(1^\top x - K)^2$1 support) recovers feasible portfolios from infeasible quantum samples. Ablation studies show that classical post-processing, rather than quantum sampling, is responsible for the recovery of high-quality portfolios in sparse cases—such as betting with settlement-graph priors.

Figure 4: Ablation reveals that classical projection (all-ones and greedy construction) achieves near-zero regret, indicating that quantum annealing merely supplies candidate landscapes that post-processing exploits.

In these structurally favorable cases, all-ones projection plus greedy elimination converges to the optimal portfolio, rendering the QPU’s contribution redundant. For dense QUBOs, post-processing cannot fully repair the damage inflicted by chain breaks, leading to suboptimal portfolios.

Penalty-Free Pipeline: Empirical Validation

Motivated by the above diagnosis, the paper proposes simply dropping the penalty term: formulate a penalty-free, objective-only QUBO ($A(1^\top x - K)^2$2), sample on QPU, and enforce cardinality classically. This pipeline eliminates the penalty-induced density, preserves objective structure, and delivers robust chain integrity on hardware.

Empirical results confirm dramatic improvements. Chain-break fractions collapse from 71–92% (penalized) to at most 0.04% on all tested instances ($A(1^\top x - K)^2$3). Post-processed solutions achieve at most 0.03% regret relative to greedy classical references on equities, and outperform backward elimination on betting at $A(1^\top x - K)^2$4 and $A(1^\top x - K)^2$5 (yielding lower-energy feasible portfolios).

Figure 5: Head-to-head comparison of the standard penalty-encoded pipeline versus the penalty-free alternative; the latter yields near-zero chain breaks and short chains, with classical feasibility projection restoring exact-$A(1^\top x - K)^2$6 compliance.

Figure 6: Chain-break fractions in penalized (red) versus penalty-free (green) pipelines across scaling instances; penalty-free workflow achieves robust chain integrity at all scales, independent of logical density.

Notably, in the penalty-free regime, the classical all-ones projection baseline is no longer optimal, and the QPU’s landscape exploration yields meaningful improvements for betting portfolios, establishing a visible quantum contribution within practical workflow constraints.

Embedding Analysis and Topology Effects

Zephyr consistently outperforms Pegasus in embedding quality: lower qubit overhead and shorter chain lengths at fixed logical graph density. For both dense and sparse QUBOs, all tested instances ($A(1^\top x - K)^2$7) embed successfully. The primary performance bottleneck at these scales is chain-break fraction and not embeddability.

Figure 7: Zephyr achieves lower physical qubit overhead and chain lengths compared to Pegasus for equivalent logical graphs.

Figure 8: Qubit overhead and chain length increase with logical graph density; sparsification delivers substantial reductions, with Zephyr maintaining best performance throughout.

Financial Performance and Practitioner Implications

Out-of-sample validation demonstrates competitive realized Sharpe ratios (equities) and ROI (betting) for penalty-free and sparsified pipelines, but not outright dominance. The main contribution is operational: direct QPU sampling with penalty-free QUBO plus classical projection yields feasible, investable portfolios—enabling integration into periodic selection workflows such as betting slate sizing or equity screening.

Figure 9: Realized Sharpe ratio and ROI for equities and betting; error bars denote confidence intervals, confirming competitive but not exceptional financial validation.

Within quantum-computation research, the results establish that penalty encoding, not sparse hardware topology, is the primary constraint for direct quantum annealer portfolio optimization at currently accessible scales. Practically, penalty-free pipelines should be preferred, and constraints handled classically when fast projection is possible.

Broader Theoretical and Practical Implications

This work delineates a sharp separation between objective sampling and constraint enforcement in quantum portfolio optimization. It contends that penalty encoding enforces constraints at the cost of embedding viability and solution quality, and demonstrates that classical projection is both efficient and effective for exact-cardinality in relevant application domains.

Theoretically, the findings strongly suggest generalization to any combinatorial optimization problem where constraints are currently enforced by dense quadratic penalties. Future studies should investigate applicability to other sparse-and-structured domains, and alternative encoding/constraint-handling strategies (e.g., native CQMs, structural encodings).

Conclusion

This research rigorously isolates the cardinality penalty as the dominant structural bottleneck in direct quantum-annealer portfolio optimization. Topology-aware sparsification trivially solves embedding but inadvertently disables constraint enforcement, shifting the solution burden to classical post-processing. Removing the penalty upstream (penalty-free pipeline) restores chain integrity and produces feasible, competitive portfolios across tested scales. These results recommend abandoning penalty-encoded QUBOs in favor of objective-only sampling and downstream classical projection—both for portfolio optimization and broader classes of quantum annealing problems with tractable constraints.