KernelSOS: RKHS-Based Sum-of-Squares Method

• KernelSOS is an RKHS-based generalization of sum-of-squares that represents nonnegative functions via PSD quadratic forms in feature space.
• It reformulates classical polynomial SOS into finite-dimensional semidefinite programs, supporting applications in global optimization, control, estimation, and Bayesian optimization.
• Recent advances include adaptive kernel calibration and custom SDP solvers that improve scalability and efficiency in trajectory optimization and conformal prediction.

Kernel Sum of Squares, commonly abbreviated KernelSOS or kSoS, is an RKHS-based generalization of sum-of-squares methodology in which nonnegative functions are represented as quadratic forms of kernel feature maps and positive semidefinite operators. In its scalar form, a kSoS function has the structure $g(x)=\langle \phi(x),A\phi(x)\rangle_H$ with $A\succeq 0$, so nonnegativity is enforced by construction rather than by direct pointwise verification. This framework extends polynomial SOS from finite-dimensional monomial bases to reproducing kernel Hilbert spaces, admits finite-dimensional semidefinite representations through Gram matrices and representer theorems, and has been used for global optimization, infinite-constraint approximation, PSD-valued regression, convex regression, trajectory optimization, Bayesian optimization, and adaptive conformal prediction (Aubin-Frankowski et al., 2023, Bach et al., 2023, Groudiev et al., 23 Jul 2025).

1. Formal definition and core construction

In the general RKHS formulation, KernelSOS parameterizes a nonnegative scalar function by

$$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$

where $H$ is a scalar RKHS on $\mathcal X$, $\phi(x)=k_\phi(\cdot,x)$ is the canonical feature map, and $A$ is a bounded self-adjoint PSD operator. The associated cone of kSoS functions is

$$\mathcal{K}_{\mathrm{kSoS}} = \{g:\mathcal X\to\mathbb R \mid \exists A\in S^+(H): g(x)=\langle \phi(x),A\phi(x)\rangle_H\},$$

which is the RKHS analogue of polynomial SOS cones (Aubin-Frankowski et al., 2023).

For sampled optimization, KernelSOS replaces a continuum nonnegativity condition by interpolation equalities at sampled points. In the trajectory-optimization specialization, given samples $\mathbf u^{(1)},\dots,\mathbf u^{(N)}$, kernel matrix

$$K_\sigma = \bigl(k_\sigma(\mathbf u^{(i)},\mathbf u^{(j)})\bigr)_{i,j\in[N]},$$

and Cholesky factorization $A\succeq 0$0, KernelSOS solves

$A\succeq 0$1

where $A\succeq 0$2 is the $A\succeq 0$3-th column of $A\succeq 0$4. The resulting surrogate is

$A\succeq 0$5

with

$A\succeq 0$6

By construction, $A\succeq 0$7 at all sampled points, and $A\succeq 0$8 globally because the quadratic form is PSD (Wei et al., 29 Apr 2026).

A central interpolation fact in the more general constraint-setting is that, for strictly positive definite kernels, any nonnegative values on a finite sample set can be interpolated exactly by a kSoS function. The paper on infinite-dimensional constrained optimization states this as a kSoS interpolation lemma, with explicit trace expressions and bounds for the witnessing PSD operator (Aubin-Frankowski et al., 2023).

2. Relation to classical SOS and theoretical guarantees

KernelSOS is directly descended from classical SOS optimization. In polynomial SOS, one seeks $A\succeq 0$9 such that $$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$0 is a polynomial sum of squares, typically written as $$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$1 with $$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$2. KernelSOS replaces the finite polynomial feature vector by a kernel-induced feature map and the polynomial Gram matrix by a PSD operator or its finite Gram representation. The general SOS note formulates the finite-dimensional and kernelized versions in the same lower-bound template, namely maximizing $$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$3 under a globally nonnegative certificate for $$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$4 (Bach et al., 2023).

This generalization preserves the lower-bound interpretation. In the sampled RKHS relaxation used for contact-rich manipulation, the paper states that, if $$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$5 lies in the chosen RKHS, then the optimal value satisfies

$$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$6

where $$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$7 is the true global minimum, and the relaxation gap asymptotically approaches zero as $$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$8 (Wei et al., 29 Apr 2026). The control-and-estimation paper makes the same point at the methodological level: KernelSOS replaces the classical degree-based hierarchy by a sample-based relaxation, and if a finite-degree polynomial kernel is chosen, KernelSOS reduces to classical polynomial SOS (Groudiev et al., 23 Jul 2025).

For infinite families of pointwise inequality constraints, the general constrained kSoS theory gives a stronger approximation-theoretic picture. Pointwise inequalities

$$g(x)=\langle \phi(x),A\phi(x)\rangle_H, \qquad A\in S^+(H),$$9

are replaced by equalities with kSoS slack at sampled points, and scattered-data inequalities propagate these sampled equalities to the whole domain. The main theorem yields nested constraint sets and error scales $H$0 and, under stronger smoothness conditions, $H$1, which the paper describes as “breaking the curse of dimensionality when $H$2 is large relative to $H$3” (Aubin-Frankowski et al., 2023).

3. Semidefinite formulations, representer theorems, and algorithmic realizations

A recurrent structural feature of KernelSOS is that infinite-dimensional PSD-operator optimization collapses to finite PSD-matrix optimization. In the scalar setting, the representer theorem states that optimal operators can be written in the span of rank-one feature operators $H$4, so the learned nonnegative function becomes

$H$5

or, after empirical reparameterization,

$H$6

(Allain et al., 27 May 2025).

The same finite-dimensionalization underlies several algorithmic variants. In global control and estimation, KernelSOS samples $H$7 points, constructs a kernel Gram matrix, solves one SDP per global step, and recovers a candidate minimizer through the dual variables as

$H$8

The method is then combined with a restart strategy that shrinks the sampling region around the current solution (Groudiev et al., 23 Jul 2025). In contact-rich manipulation, the framework is embedded inside Global-MPPI: log-sum-exp smoothing produces a differentiable surrogate objective, KernelSOS performs global exploration, and MPPI locally refines the returned candidate. That paper also introduces automatic kernel calibration by minimizing a Gaussian-process marginal likelihood over $H$9, uses a Laplace kernel for KernelSOS, and solves the SDP with a custom interior-point method exploiting the rank-one structure of the constraints; in the reported experiments, $\mathcal X$0 and $\mathcal X$1 samples are used per stage (Wei et al., 29 Apr 2026).

A distinct realization appears in acquisition optimization for Bayesian optimization. KSOS-BO formulates acquisition minimization as a semidefinite program with kernel-induced representations, recovers a candidate through a convex combination of sampled points, and treats acquisition optimization as a structured derivative-free problem rather than a generic black-box inner loop (Ou et al., 20 May 2026).

4. Global optimization, control, estimation, and Bayesian optimization

KernelSOS has become particularly prominent in black-box global optimization and robotics. The control-and-estimation study emphasizes that the method is applicable to non-polynomial and non-parametric formulations, works with function evaluations only, and can operate either as a standalone optimizer or as an initialization method for local solvers. Its sample-based character allows it to handle trajectory optimization with an integrated simulator treated as a black box (Groudiev et al., 23 Jul 2025).

The contact-rich manipulation specialization is more tightly integrated. Global-MPPI decomposes into three stages: graduated non-convexity through log-sum-exp smoothing, KernelSOS-based global search on the smoothed objective, and MPPI local refinement. The paper evaluates this pipeline on PushT and dexterous in-hand manipulation and reports “faster convergence and lower final costs compared to existing baseline methods.” The details identify PushT as a 12-dimensional control problem with a 1-second horizon and dexterous in-hand manipulation as a 64-dimensional problem with a 0.25-second horizon; in both tasks, removing auto-calibration, GNC, or local refinement degrades performance (Wei et al., 29 Apr 2026).

In estimation, KernelSOS has been applied to range-only localization without requiring polynomial reformulations. The cited control-and-estimation paper reports that KernelSOS is competitive with other SOS approaches on estimation problems while remaining applicable to non-polynomial costs, including settings where polynomial formulations would otherwise require squaring or auxiliary variables (Groudiev et al., 23 Jul 2025).

In Bayesian optimization, KSOS-BO uses KernelSOS to optimize acquisition functions. The paper reports that it “consistently outperforms derivative-free baselines using Sobol Search, Differential Evolution, or CMA-ES to optimize the acquisition function, achieving an average regret improvement of 81.16% on 10/15 benchmarks,” and that it “converges faster in wall-clock time with an average improvement of 93.55% on 10/15 benchmarks,” while also noting reduced effectiveness on functions with steep drops or plate-shaped regions (Ou et al., 20 May 2026).

5. Constrained learning, PSD-valued functions, convexity, and conformal prediction

Beyond scalar global optimization, KernelSOS has been extended to learning problems with structural constraints. In infinite-dimensional constrained optimization, it is used to replace pointwise affine inequalities by sampled equalities with nonnegative kSoS slack. The paper illustrates this on learning vector fields with side information, specifically the invariance of a set, by enforcing boundary inequalities through a kSoS parameterization (Aubin-Frankowski et al., 2023).

A matrix-valued extension models PSD-valued functions as

$\mathcal X$2

which guarantees $\mathcal X$3 pointwise. That work proves a representer theorem, a universal approximation theorem for continuous PSD-valued functions on compact sets, and eigenvalue bounds under subsampled equality constraints. The same framework is then used to model convex functions by imposing a KernelSOS representation on the Hessian, and the paper shows that any smooth and strongly convex function admits such a representation (Muzellec et al., 2021).

KernelSOS has also been used to learn adaptive conformal scores. In the symmetric-band formulation, the score is

$\mathcal X$4

where $\mathcal X$5 is learned as a nonnegative KernelSOS function under constraints that dominate squared residuals on the pre-training set. The paper proves a general representer theorem for the resulting statistical learning problem, derives a dual formulation that can be solved efficiently by accelerated gradient methods with “several hundreds or thousands of samples,” and proposes HSIC-based tuning of kernel lengthscales to target adaptivity (Allain et al., 27 May 2025). A later asymmetric extension replaces the single scaling function by lower and upper half-width functions $\mathcal X$6 and $\mathcal X$7, with score

$\mathcal X$8

and adds symmetry-promoting penalties that interpolate between symmetric and asymmetric bands (Allain et al., 30 Jan 2026).

6. Limitations, assumptions, and open directions

KernelSOS inherits the expressive flexibility of RKHS models and the convexity of PSD constraints, but its guarantees are conditional and its computational profile remains dominated by semidefinite structure. Several papers state that performance depends strongly on kernel choice and scale. In Global-MPPI, auto-calibration is introduced precisely because poor kernel-parameter choices degrade surrogate quality; the same paper lists kernel adequacy, sampling coverage, and the smoothness of the smoothed objective as key assumptions (Wei et al., 29 Apr 2026). The general constraint-approximation theory likewise requires smooth kernels, smooth constraint functions, and sufficiently dense sampling sets to control violations away from sampled points (Aubin-Frankowski et al., 2023).

Scalability is the most explicit practical limitation. The contact-rich manipulation paper notes that SDP cost “scales cubically with the number of samples,” even though the SDP size is independent of control dimension, and identifies GPU-accelerated SDP solvers and more efficient sampling as future directions (Wei et al., 29 Apr 2026). The broader control-and-estimation paper also describes a single-SDP-per-step method whose cost is governed by sample count rather than polynomial degree, and emphasizes custom solvers exploiting rank-one structure (Groudiev et al., 23 Jul 2025). In Bayesian optimization, the authors explicitly describe higher per-iteration cost as a limitation, even though end-to-end wall-clock time can still improve (Ou et al., 20 May 2026).

A second limitation concerns exactness under subsampling. For PSD-valued functions and convex regression, enforcing equality constraints only at finitely many points yields eigenvalue or convexity violations bounded by $\mathcal X$9 away from the sample, rather than exact global enforcement (Muzellec et al., 2021). The constrained kSoS theory makes the same point in a more general form: finite sampled equalities provide controlled approximations, not exact continuum feasibility, unless asymptotic or stronger representability assumptions are invoked (Aubin-Frankowski et al., 2023).

Open directions stated across the literature include richer kernels, low-rank or sparse parameterizations of PSD operators, adaptive sampling schemes, improved large-scale solvers, integration with learned dynamics or surrogate models, and extensions to more general derivative, convexity, and conic constraints (Aubin-Frankowski et al., 2023, Muzellec et al., 2021, Wei et al., 29 Apr 2026). Collectively, these directions indicate that KernelSOS is best understood not as a single algorithm, but as a family of RKHS-based SOS constructions whose central idea is stable across applications: represent nonnegativity through PSD quadratic forms in feature space, then exploit semidefinite optimization and finite representer structure to make that representation computationally usable.