L-SPADE Linear Proxy Framework

• L-SPADE Linear Proxy is a method that utilizes sparse linear inequalities to approximate high-dimensional PSD cones in convex optimization effectively.
• It employs a separation SDP to generate E-PSD cuts, ensuring the LP relaxation matches tight SDP bounds while reducing computational complexity by up to 100×.
• In neural decoding, L-SPADE enables early-exit strategies in transformers via learned linear maps, accelerating inference without compromising accuracy.

The term L-SPADE Linear Proxy refers to a methodology for efficiently projecting, approximating, or substituting high-complexity or high-dimensional mathematical objects by linear or polyhedral proxies using sparse or low-cost transformations. Contemporary instantiations of "L-SPADE linear proxy" span several fields, including convex optimization (particularly semidefinite programming relaxations of nonconvex quadratic problems) and machine learning (notably, transformer-based LLMs). Across these domains, the L-SPADE framework employs sparsity, linearity, and projection to achieve substantial reductions in computational cost without sacrificing representational or bound tightness. The concept is most prominently developed in recent works by Günlük et al. on positive semidefinite (PSD) cones (Günlük et al., 10 Mar 2026), as well as in efficient decoding and early-exit policies for large neural networks (Zheng et al., 23 Jul 2025).

1. Linear Proxy for the Positive Semidefinite Cone

The L-SPADE linear proxy in convex optimization, especially for semidefinite programming (SDP) relaxations of quadratically-constrained quadratic programs (QCQPs), is based on the construction of sparse linear inequalities—termed "E-PSD cuts"—that approximate or outer bound the PSD constraint. For an $n$-variable QCQP, the standard Shor relaxation lifts the problem to a matrix variable $Y \in S^{n+1}$ with a PSD constraint $Y \succeq 0$ and additional affine constraints. This renders the solution of large SDPs burdensome, especially in branch-and-bound frameworks.

The key insight is that if the problem data $(Q^k)$ are sparse, only a subset $E$ of the indices in $Y$ need be enforced. Let $$E = \{(i,j): i=0 \lor j=0 \lor i=j \lor Q^k_{ij} \neq 0 \text{ for some }k\}$$ and define $H_E$ as the subspace of symmetric matrices with support in $E$. The L-SPADE proxy constructs an LP relaxation over $Z=(Y_{ij})_{(i,j)\in E}$ augmented by the family of E-PSD cuts: linear inequalities $Y \in S^{n+1}$0 for all $Y \in S^{n+1}$1. The main theorem establishes that the projection of the full PSD cone onto $Y \in S^{n+1}$2, enforced via E-PSD cuts, makes the LP relaxation as tight as the original SDP:

$Y \in S^{n+1}$3

provided all constraints and objective coefficients are supported on $Y \in S^{n+1}$4 (Günlük et al., 10 Mar 2026).

2. Sparse-Cut Generation and Separation SDP

To operationalize the L-SPADE proxy, separation oracles are used to identify violated E-PSD cuts. At a candidate $Y \in S^{n+1}$5, one solves the projection SDP:

$Y \in S^{n+1}$6

If the optimal value is negative, the minimizer $Y \in S^{n+1}$7 yields a valid sparse cut $Y \in S^{n+1}$8. In doubly-nonnegative relaxations (when $Y \in S^{n+1}$9), the cone is further tightened by $Y \succeq 0$0 for $Y \succeq 0$1. This separation subproblem is itself a small SDP, but is tractable due to the limited support $Y \succeq 0$2.

Practically, the algorithm proceeds iteratively: solve the root-node LP with accumulated sparse cuts, check for separation at LP solutions, and repeat until no violation is found or a maximum number of iterations is reached. In branch-and-bound (B&B), the LP proxy is re-used at every node, and cut separation can be optionally performed in selected nodes (Günlük et al., 10 Mar 2026).

Step	Input/Output	Operation
1	QCQP data $Y \succeq 0$3	Compute $Y \succeq 0$4; initialize cut set $Y \succeq 0$5
2	LP+E-PSD cuts; candidate $Y \succeq 0$6	Separation via projection SDP; add new cut if violated
3	B&B node	Solve LP over $Y \succeq 0$7 with all sparse cuts in $Y \succeq 0$8

3. Theoretical and Empirical Performance

Theoretically, the L-SPADE linear proxy delivers the exact relaxation bound obtainable from the full SDP when all problem structure is captured in $Y \succeq 0$9. This provides a substantial reduction in computational effort: the LP relaxation over variables in $(Q^k)$0 plus a few hundred sparse cuts can be solved $(Q^k)$1–$(Q^k)$2 faster than dense-cut approaches.

Empirical studies on BoxQCQP and QPLIB benchmarks confirm the method's efficiency: in BoxQCQPs, sparse cuts at the LP root node closed at least $(Q^k)$3 of the SDP gap, and often matched the SDP bound exactly for QPLIB instances. Overall B&B speedup ranged from $(Q^k)$4 to $(Q^k)$5 on difficult, high-dimensional quadratic instances, with dramatic reductions in node counts (Günlük et al., 10 Mar 2026).

4. Linear Proxy in Early-Exit Neural Decoding

The L-SPADE linear proxy has also been adapted as a linear approximation to the SPADE (Space Alignment Decoding) mechanism in LLMs. In this context, the goal is to enable early-exit at intermediate transformer layers by estimating the output-layer token distribution from intermediate representations via a learned linear map.

Let $(Q^k)$6 be the hidden state of token $(Q^k)$7 at layer $(Q^k)$8 and $(Q^k)$9 the output embedding matrix. The L-SPADE proxy is a matrix $E$0 such that

$E$1

where $E$2 is a proxy for the final-layer hidden state, yielding output logits and token probabilities without full sequential decoding (Zheng et al., 23 Jul 2025).

$E$3 is trained via cross-entropy distillation to match the SPADE output logits, minimizing

$E$4

across all training tokens.

5. Entropy-Based Early-Exit Policies

L-SPADE enables entropy-based early-exit in transformer inference. At each evaluation interval, the proxy distribution $E$5 is computed from the current hidden state, and its Shannon entropy

$E$6

is evaluated. If $E$7 falls below a threshold $E$8, indicating confident prediction, inference truncates and the final answer is produced via a lightweight SPADE forward on the minimal two-token sequence. This mechanism reduces inference complexity from $E$9 (for full-sequence, all-layer decoding) to a single matrix multiply and softmax per check, plus the truncated SPADE pass.

Layer $Y$0	Proxy logits $Y$1	Entropy $Y$2	Decision
Example	$Y$3	$Y$4 nats (with $Y$5)	Stop and decode via SPADE

6. Cross-Domain Significance and Comparative Summary

Across both optimization and machine learning, the L-SPADE linear proxy formalizes the substitution of high-cost, high-dimensional constraints (PSD-cone, deep feature transformation) with computationally efficient, sparse, or low-rank linear surrogates. In optimization, this leads to scalable LP relaxations matching SDP bounds, highly advantageous for large-scale branch-and-bound. In machine learning, it enables efficient layer-wise proxy decoding while maintaining predictive fidelity, supporting hybrid early-exit strategies with tight accuracy guarantees (Günlük et al., 10 Mar 2026, Zheng et al., 23 Jul 2025).

Both lines of work demonstrate that careful structural projection—onto a sparsity pattern for SDPs, or via learned linear maps for neural representations—can preserve essential problem structure without incurring prohibitive resource costs. This suggests further potential of L-SPADE proxies in high-dimensional scientific computing where constraint enforcement or model-depth bottlenecks dominate.