Convex-Relaxation-Then-Quantization

• Convex-relaxation-then-quantization is a two-stage method that replaces a nonconvex discrete problem with a convex surrogate and recovers a feasible discrete solution through rounding or projection.
• It encompasses diverse mechanisms such as simplex relaxation, convex regularization, semidefinite lifting, and exact convex-hull characterization to tailor recovery based on the problem structure.
• The approach balances computational tractability with solution accuracy, demonstrating versatility in applications from quantized neural networks to combinatorial signal design.

Searching arXiv for the cited works on convex relaxations, quantization, and related recovery pipelines. Search query: "Soft Convex Quantization Revisiting Vector Quantization with Convex Optimization (Gautam et al., 2023)" Convex-relaxation-then-quantization denotes a family of methods in which a discrete, combinatorial, or otherwise nonconvex problem is first replaced by a convex optimization problem over continuous variables and then mapped back to a discrete or structurally feasible object. In the literature, this pattern appears in several distinct forms: exact convex-hull formulations followed by randomized sign quantization, convex regularization paths whose proximal maps converge to hard quantization, and relaxation-based recovery procedures that project continuous surrogates back onto nonlinear feasible sets. The term is therefore best treated as a methodological umbrella rather than a single algorithmic recipe. Equally important, not every convex relaxation used near a discrete problem actually performs a final quantization step; some methods replace hard assignment by a soft convex substitute and stop there (Gautam et al., 2023, Jin et al., 19 Mar 2025, Mo et al., 2018).

1. Definition and scope

At its most literal, the pattern has two stages. The first stage replaces a hard feasible set—such as one-hot assignments, binary sequences, rank-one lifted matrices, or quantized weights—by a convex superset or convex surrogate. The second stage converts the continuous optimizer into a discrete object by rounding, projection, thresholding, randomized sampling, or an asymptotic hardening mechanism. This two-stage structure is explicit in Piecewise-Affine Regularized Quantization, where a convex regularizer attracts weights toward a finite codebook and the associated proximal map converges to hard quantization, and in randomized SDP methods for binary sequence design and quantized neural networks, where a convex lifted solution is converted into binary outputs by sampling and sign quantization (Jin et al., 19 Mar 2025, Mo et al., 2018, Bartan et al., 2021).

A broader reading also includes methods whose second stage is not literal scalar quantization but recovery of a feasible nonlinear object from a convex surrogate. In pairwise interaction problems over probability measures, the relaxed variable is an auto-correlation $F$, and recovery is performed by minimizing a Kullback–Leibler divergence over feasible $\rho$ such that $F_\rho=\rho\circ\rho$ approximates the relaxed solution; this is structurally analogous to rounding, even though the output is a measure rather than a quantized codeword (Bandegi et al., 2015). By contrast, Soft Convex Quantization is explicitly described as a strong example of the “convex-relaxation” side but only a partial example of the full “then quantization” story, because its forward output is already a soft convex reconstruction $CP^\star$ rather than a rounded discrete code (Gautam et al., 2023).

Setting	Convex stage	Recovery or endpoint
Soft Convex Quantization	simplex-constrained convex coding	soft latent $CP^\star$
PARQ	convex piecewise-affine regularization	proximal map hardens to quantization
Spectrally shaped binary sequences	SDP over lifted matrix $S$	randomized projection and sign quantization
Quantized two-layer networks	SDP over lifted moments	sampled binary weights
Pairwise interaction energies	convex relaxation in auto-correlation $F$	KL-based recovery of $\rho$

This diversity matters. A common misconception is that any convex relaxation near a discrete model is automatically a relax-then-round method. The literature does not support that simplification.

2. Canonical convexification mechanisms

One common mechanism replaces vertices of a simplex by the simplex itself. In vector quantization, standard VQ assigns each latent to a single codeword, whereas SCQ replaces the one-hot assignment by a simplex-valued vector. Its core forward problem is

$$P^\star := \arg\min_{P\in \mathbb{R}^{K\times N\widetilde H\widetilde W}} \|Z_e^{\mathrm{flattened}}-CP\|_F^2+\lambda\|P-\widetilde P\|_F^2$$

subject to

$$P\ge 0,\qquad P^\top 1_K = 1_{N\widetilde H\widetilde W}.$$

Here hard VQ corresponds to one-hot columns, while SCQ allows any point on the simplex. The relaxation is geometric: the finite codebook is replaced by its convex hull, and the output may lie anywhere in that convex hull (Gautam et al., 2023).

A second mechanism replaces an indicator of discrete feasibility by a convex regularizer. PARQ starts from

$\rho$0

or equivalently from a nonconvex penalty enforcing $\rho$1, and substitutes the convex piecewise-affine regularizer

$\rho$2

The resulting objective

$\rho$3

is convex whenever $\rho$4 is convex. Quantization then emerges through the proximal map of $\rho$5, which has flat segments at the target levels and converges to hard quantization as the aggregate regularization strength grows (Jin et al., 19 Mar 2025).

A third mechanism is semidefinite lifting. In spectrally shaped binary sequence design, the binary vector $\rho$6 is lifted to $\rho$7, the quadratic objective and constraints become linear in $\rho$8, and the nonconvexity is isolated in $\rho$9. Dropping rank yields the convex SDP

$F_\rho=\rho\circ\rho$0

An analogous lifting is used for quantized two-layer neural networks with polynomial activations, where SDP moment matrices replace binary first-layer weights during optimization (Mo et al., 2018, Bartan et al., 2021).

A fourth mechanism is exact convex-hull characterization. For a single quadratic equality intersected with a bounded polyhedron,

$F_\rho=\rho\circ\rho$1

the exact convex hull $F_\rho=\rho\circ\rho$2 is second-order-cone representable. This does not itself quantize anything, but it supplies an exact convex module that can serve upstream of later rounding or branching procedures (Santana et al., 2018).

3. Recovery, rounding, and hardening

The second stage varies sharply across the literature. In some methods it is randomized. For binary sequence design, after solving the SDP one computes $F_\rho=\rho\circ\rho$3, samples

$F_\rho=\rho\circ\rho$4

and quantizes by

$F_\rho=\rho\circ\rho$5

The final binary sequence is selected from feasible candidates maximizing message-band power. The theoretical link between relaxed and rounded solutions is the arcsine law

$F_\rho=\rho\circ\rho$6

which explains why randomized rounding preserves second-order structure more faithfully than deterministic thresholding (Mo et al., 2018).

In quantized neural network training via SDP, the recovery map is also randomized but distributional rather than deterministic. Grothendieck’s identity yields

$F_\rho=\rho\circ\rho$7

allowing sampled binary weights to reproduce the SDP moment matrix in expectation. Under sufficient width, the resulting quantized network is provably near the global optimum of the original discrete bilinear problem (Bartan et al., 2021).

In PARQ, recovery is integrated into optimization rather than appended afterward. Aggregate Proximal Stochastic Gradient maintains a latent full-precision variable $F_\rho=\rho\circ\rho$8 and applies

$F_\rho=\rho\circ\rho$9

As $CP^\star$0, the soft proximal map sharpens into hard quantization. The paper explicitly interprets the straight-through estimator as the asymptotic form of this proximal mechanism rather than as a heuristic gradient rule (Jin et al., 19 Mar 2025).

Other recovery rules are deterministic thresholding or support selection. In planted influence maximization under the independent-cascade model, the convex relaxation need not be integral even when it identifies the planted influencers. The paper therefore introduces a thresholding map $CP^\star$1 and, in practice, a top-$CP^\star$2 selector $CP^\star$3 that chooses the $CP^\star$4 largest coordinates of the convex solution (Elkin et al., 2013).

A different boundary case is recovery without quantization in the scalar sense. In pairwise interaction problems, the relaxed auto-correlation $CP^\star$5 is projected back onto the nonlinear set $CP^\star$6 by minimizing

$CP^\star$7

This is best described as structured recovery rather than quantization, but it occupies the same methodological slot (Bandegi et al., 2015).

4. Domains in which the pattern appears

Quantization-aware training provides perhaps the clearest modern machine-learning instance. PARQ is explicitly formulated as convex regularization inducing progressive attraction to discrete levels, with hard quantization recovered as a limit of the proximal map. Its theory covers convex losses and proves average-iterate and last-iterate convergence for AProx, while experiments show competitive performance on convolutional and transformer-based vision models (Jin et al., 19 Mar 2025).

Vector quantization in latent-variable models illustrates a more ambiguous use of convex relaxation. SCQ replaces nearest-neighbor assignment by a differentiable convex optimization layer and substantially improves reconstruction and codebook usage, but it does not recover a discrete token during the main forward pipeline. The method is therefore best understood as “convex relaxation instead of hard quantization,” not as a classical relax-then-round procedure (Gautam et al., 2023).

Combinatorial signal design offers a textbook relax-then-quantize case. Spectrally shaped binary sequences are designed by SDP, randomized Gaussian projection, sign quantization, and feasibility-aware selection. The paper emphasizes that naive direct quantization badly distorts the spectrum, and that randomized rounding is needed because the interferer quadratic form is not diagonal (Mo et al., 2018).

Quantized inverse problems can also invert the order. In quantized compressed sensing by ReLUs, measurements are quantized first and the convex program is applied afterward. The one-bit estimator minimizes

$CP^\star$8

which is a convex proxy for Hamming inconsistency with dithered quantization cells. This is adjacent to convex-relaxation-then-quantization, but not an instance of that forward pipeline (Jung et al., 2019).

5. Exactness, tightness, and the geometry of relaxations

The strongest possible relaxation is the exact convex hull. For quadratic equality over a bounded polyhedron, second-order-cone representability of $CP^\star$9 means there is no tighter convex relaxation in the original variables for that single constraint set, although the representation may require exponentially many variables or disjunctive components. Exact representability therefore does not imply compactness or computational ease (Santana et al., 2018).

Exactness can also appear as integrality. In the deterministic planted influence model, the LP relaxation is uniquely solved by the binary indicator of the influencers, so the convex program already returns the discrete optimizer. No rounding is needed there, even though a thresholding step becomes necessary in the probabilistic cascade variant (Elkin et al., 2013).

At the opposite extreme are relaxation-only methods. Tightened single-neuron relaxations for ReLU verification derive the exact convex hull of a single ReLU over a multivariate box input domain and supply linear-time separation, but the output is a verification bound, not a discrete rounded object (Tjandraatmadja et al., 2020). Convex relaxations of one-hidden-layer convolutional networks likewise provide a continuous shared filter under planted Gaussian assumptions, yet the paper does not analyze any subsequent quantization of that filter (Bartan et al., 2018).

Some papers argue for a broader unification. A recent convex-analytic synthesis casts compression, quantization, and decoding as convex projections of continuous information onto discrete manifolds, with Gibbs-type variational formulations, KL/Bregman geometry, reverse water-filling, and variational decoding all treated under one framework. This suggests that relax-then-discretize is not only an algorithmic trick but also an information-geometric pattern (Macchiavello, 12 Dec 2025).

6. Limitations and boundary cases

Several limitations recur. First, principled exact relaxations are often too expensive in raw form. SCQ’s exact differentiable convex optimization layer is estimated to scale as $CP^\star$0, so the practical method replaces it by an unconstrained linear solve plus approximate simplex projection, reducing the dominant cost to $CP^\star$1 (Gautam et al., 2023).

Second, many guarantees stop at the convex surrogate. PARQ’s convergence theory is for convex losses, not for the full nonconvex deep networks used in experiments; its online codebook estimation and slope schedules are practical deviations from the clean theory (Jin et al., 19 Mar 2025). Quantized neural network training via SDP is restricted to two-layer polynomial activations or a lifted bilinear architecture, with quantized first-layer weights and sufficiently large width (Bartan et al., 2021).

Third, some convexified problems fundamentally do not include a discrete recovery stage. The second-quantized SCE relaxation over binary occupations yields lower bounds and approximate dual potentials for Kohn–Sham self-consistent field iteration, but not a rounded transport plan on $CP^\star$2 (Khoo et al., 2019). SCQ likewise leaves open the problem of producing truly discrete latent tokens for downstream autoregressive or diffusion models (Gautam et al., 2023).

A final misconception is that “rounding” is always scalar or coordinatewise. The literature shows otherwise. Recovery may be randomized sign projection, thresholding of the largest coordinates, KL projection onto realizable auto-correlations, or direct integrality of the convex optimum itself. This suggests that the defining feature of convex-relaxation-then-quantization is not a particular rounding operator, but the ordered separation between a convex continuous stage and a later recovery of a discrete or nonlinear feasible representation (Mo et al., 2018, Bandegi et al., 2015, Elkin et al., 2013).