Quadratic Knapsack Problem (QKP)

• The Quadratic Knapsack Problem (QKP) is a combinatorial optimization problem that generalizes the classical knapsack by integrating quadratic (pairwise) interactions among items.
• Advanced solution techniques, including semidefinite programming, parametric convex quadratic programming, and heuristic methods, yield small optimality gaps and efficient runtimes.
• QKP has practical applications in portfolio selection, project planning, social network analysis, and even PDE-constrained optimization in scientific computing.

The Quadratic Knapsack Problem (QKP) is a fundamental combinatorial optimization problem that generalizes the classical knapsack problem by incorporating quadratic (pairwise) terms in the objective. It serves as a prototypical testbed for advanced discrete optimization methodologies and holds significance in both theoretical algorithm research and practical applications involving portfolio selection, project planning, and social network analysis.

1. Mathematical Formulation and Problem Variants

The standard QKP is defined as follows. Given $n$ items with profit parameters $\{p_i\}_{i=1}^n$ (linear, or singleton) and $\{p_{ij}\}_{1\leq i<j\leq n}$ (pairwise, or quadratic), weights $w_i>0$, and a knapsack capacity $W$, the goal is

$$\begin{aligned} \text{maximize} \quad & H(x) = \sum_{i=1}^n p_i x_i + \sum_{1\leq i<j\leq n} p_{ij} x_i x_j \ \text{subject to} \quad & \sum_{i=1}^n w_i x_i \leq W \ & x_i \in \{0,1\} \quad \forall i. \end{aligned}$$

Variants of QKP considered in the literature include:

• The $k$-item QKP (kQKP), adding a cardinality constraint $\sum x_i = k$ (Létocart et al., 2016).
• The continuous or relaxed versions, allowing $x_i \in [0,1]$ or higher-order weights (for bounding or pre-processing).
• QKP with negative or indefinite $p_{ij}$, leading to NP-hardness and inapproximability in general (Taylor, 2015).

Beyond the standard form, applications may introduce multiple knapsack constraints, side constraints, or context-specific quadratic terms, e.g., for mesh diffusion or group assignment.

2. Exact Algorithms and Relaxations

The combinatorial complexity of QKP drives research into exact algorithms and tight relaxations:

Semidefinite Programming and Polyhedral Methods

• The current dominant exact approach for moderate $n$ is semidefinite relaxation (SDP), which lifts $x x^\top$ to a full matrix variable with positive semidefiniteness and linear constraints (Létocart et al., 2016, Tang et al., 2023).
• The standard relaxation writes $\max \operatorname{tr}(QX)$ subject to $w^\top x\leq W$, $\operatorname{diag}(X)=x$, $X \succeq x x^\top$.
• In kQKP, the SDP is strengthened with rank-1 and triangle (metric) inequalities, yielding a relaxation with gaps as small as 7.4% at the root node and significant pruning in branch-and-bound (Létocart et al., 2016).
• Recent advances in Burer–Monteiro factorization exploit the geometry of the feasible set to enable manifold optimization, even for $n\sim 10^6$, under weak rank requirements derived from graph minimum rank invariants. This framework ensures global optimality of second-order stationary points for generic data (Tang et al., 2023).

Parametric Convex Quadratic Programming

• Parametric convex quadratic programming constructs a family of relaxations by optimally perturbing the nonconvex quadratic objective into a concave part (retained) and a linearized remainder, then jointly optimizing the perturbation and the primal variables (Fampa et al., 2019). Unlike SDPs, these relaxations admit efficient primal-dual interior point implementation and can be further tightened by lifted cover and knapsack inequalities.

Solvers and Practical Considerations

• Interior-point methods for SDPs with rank-1 data matrices achieve $O(n^3)$ per iteration, while bundle methods dynamically incorporate only those polyhedral inequalities most violated, controlling computational load (Létocart et al., 2016).
• Riemannian gradient-based solvers for Burer–Monteiro relaxations maintain feasibility efficiently, using Newton or quasi-Newton retractions, with $O(n r^2)$ per iteration for rank $r$ (Tang et al., 2023).

3. Heuristic, Approximation, and Large-Scale Methods

Given strong NP-hardness, several heuristics, approximation algorithms, and specialized methods complement exact solvers for large $n$ or real-time applications:

Heuristic and Metaheuristic Methods

• Breakpoints heuristic algorithms (QKBP) construct the full set of optimal solutions for "breakpoint" budgets via parametric min-cut on an associated flow network. For budgets between breakpoints, a local greedy insertion/deletion efficiently recovers near-optimal solutions (Hochbaum et al., 22 Aug 2024).
• Classical heuristics include greedy-addition and exchange, local search, simulated annealing (SA), GRASP, and evolutionary or tabu-search methods, often achieving sub-1% optimality gaps in practical settings (Ohno et al., 28 Mar 2024).
• Two-stage post-processing (repair–improvement) pipelines, applied to raw solutions from fast Ising or annealing-based machines, guarantee both feasibility and optimality comparable to leading classical heuristics with minimal overhead (Ohno et al., 28 Mar 2024).

Approximation Theory

• QKP admits a polynomial-time $O(n^{2/5+\epsilon})$-approximation in the nonnegative case (Taylor, 2015). This is achieved by reducing to a logarithmic number of knapsack and densest $k$-subgraph subproblems, using rounding, bucketing, and averaging over sub-instances.
• For arbitrary profit signs, constant-factor approximation is believed impossible unless P=NP, due to reduction from the CLIQUE problem.

Specialized Algorithms

• Quantum search methods (e.g., Quantum Tree Generator, QTG) construct a superposition over all feasible $x$, with amplitude amplification yielding high-quality solutions. Empirically, QTG offers $10^2$–$10^4\times$ speedup over Gurobi for attaining 2–5% optimality gaps on $n=1000$–$2000$ instances, under favorable assumptions on quantum hardware (Wilkening et al., 28 Mar 2025).

4. Ising Machine and Quantum Annealer Implementations

The QKP is a challenging benchmark for Ising machines and related quantum hardware due to the need to encode an inequality constraint:

QUBO and Ising Mappings

• The standard QUBO encoding induces a quadratic penalty $\lambda(\sum w_i x_i + z - W)^2$, where a slack variable $z$ is introduced and can be binary-encoded, unary, or one-hot, among other schemes (Ohno et al., 28 Mar 2024).
• Conversion to Ising spin variables $s_i = 2x_i -1$ leads to pairwise and linear coupling fields whose explicit form is given by direct term expansion.
• The extended Ising Machine (EIM) further bypasses auxiliary slack binaries by maintaining a real-valued dependent variable $r=\sum w_i x_i - W$, penalized via a ReLU-type function $\lambda \max(0, r)$. This leads to smoother optimization trajectories, improved feasibility, and reduced variable counts at moderate $n$ (Akishima et al., 9 Aug 2025).
• Both conventional and extended Ising approaches can leverage simulated or digital annealing, replica-exchange, and advanced MCMC sampling strategies. Choice of penalty weight $\lambda$ is crucial, typically tuned by scaling with instance density and constraint tightness.

Postprocessing and Encoding Robustness

• Without postprocessing, different constraint encodings yield dramatically different feasible rates and solution quality. However, applying two-stage repair/improvement postprocessing "erases" encoding differences, leading all methods to converge to gaps $\sim$0.01% on moderate size ($n\leq 3000$) (Ohno et al., 28 Mar 2024).
• For large $n\sim2000$, two-stage postprocessing combined with digital annealing is competitive with state-of-the-art heuristics and exact methods (Ohno et al., 28 Mar 2024).
• Penalty and encoding parameter recommendations are provided as practical guidelines, with minor-embedding and topology overheads deferred to future research.

5. Applications in PDE-Constrained Optimization and Scientific Computing

QKP-based solvers arise in continuous optimization when enforcing positivity or entropy constraints in discretized PDEs:

• In discontinuous Galerkin spectral element methods, enforcing a cell entropy inequality can be cast as a continuous QKP: $\min_{\theta \in \mathbb R^L} \|\theta\|_2^2$ s.t. $a^T\theta \geq b$, $0\leq\theta_i\leq 1-\lambda_i^c$ (Christner et al., 19 Jul 2025).
• This convex QKP admits a reduction to scalar root-finding for a clipped linear function, enabling solution in $O(L)$ arithmetic operations with quasi-Newton updates. Compared to the linear knapsack analog (which requires a greedy fill), the quadratic version yields greater temporal and spatial regularity and guarantees uniqueness.
• Numerical benchmarks indicate that QKP-limited schemes achieve second-order time convergence, improved step counts in shock regimes, and robust enforcement of bounds, outperforming linear-knapsack-limited schemes (Christner et al., 19 Jul 2025).

6. Further Algorithmic Enhancements and Valid Inequalities

Progress toward closing the relaxation–integrality gap and accelerating convergence centers around:

• Parametric convex QP relaxations optimized over the perturbation matrix, alternating with rounds of newly derived valid inequalities in lifted matrix space. Cutting planes derived from classical cover and knapsack inequalities can be lifted to enforce tighter bounds, separated via mixed-integer programming subproblems (Fampa et al., 2019).
• Alternating rounds of parametric perturbation and cut generation yield primal gaps $<0.1\%$ and dual gaps $<0.5\%$ for random instances with $n\leq 100$, outperforming LP and standard SDP bounds (Fampa et al., 2019).
• For combinatorial variants (multiple constraints, multi-dimensional QKP), further research is required to develop multidimensional root-finding and extend the efficacy of continuous relaxations.

7. Open Problems and Research Directions

Current frontiers in QKP research and application include:

• Establishing constant-factor approximation bounds or hardness results in the most general setting.
• Scaling Ising and quantum solvers to $n>3000$ and incorporating more complex constraint sets.
• Automating penalty parameter selection for annealing and QUBO formulations.
• Exploring multidimensional and multi-objective QKP, including robust and stochastic variants.
• Bridging the gap between continuous relaxations and integer program solvers, especially in applications with large $n$ and sparse or structured $Q$.

Ongoing research suggests that no single approach dominates across all QKP regimes. SDP and lifted relaxations remain the tool of choice for moderate $n$ with demands for global optimality. Heuristic, Ising, and quantum methods provide state-of-the-art performance for large-scale and time-sensitive applications, provided care is taken with penalty parameters and postprocessing. The QKP continues to serve both as a modeling framework and as a benchmark for the development and evaluation of advanced optimization algorithms.