Interpolation-Based Optimization (IBO)

• Interpolation-Based Optimization (IBO) is a methodology that builds surrogate models via interpolation to replace unavailable derivative information in optimization.
• It leverages trust-region frameworks and poisedness conditions to ensure stable model construction and robust performance in derivative-free and constrained settings.
• IBO extends to bi-level parameter estimation and privacy-preserving formulations, restructuring complex problems with interpolated surrogates.

Searching arXiv for recent and foundational papers on interpolation-based optimization and closely related model-based derivative-free optimization. Interpolation-Based Optimization (IBO) denotes a family of optimization methodologies in which interpolation is used to construct surrogate structure, feasible mappings, or intermediate optimization targets that replace, regularize, or augment direct optimization on the original problem. In derivative-free optimization, the term is often used synonymously with model-based derivative-free optimization: local Taylor-type approximations are replaced by interpolation models built from sampled function values, and these models drive trust-region steps (Roberts, 6 Oct 2025). More broadly, recent work uses interpolation in bi-level parameter estimation for differential equations, in mixed-derivative trust-region frameworks, in constrained optimization via interpolation-based projection, and in application-specific losses or privacy mechanisms, indicating that “IBO” now refers both to a core derivative-free paradigm and to a wider design principle in which interpolation restructures optimization while preserving critical problem properties (Prabhu et al., 31 May 2025, Larson et al., 27 May 2026, Akrour et al., 2020, Qiu, 15 Jan 2026).

1. Scope and definitions

In its canonical derivative-free form, IBO studies nonlinear optimization when gradient or Hessian information is unavailable, expensive, or noisy. The defining operation is the construction of local models by interpolation over sampled points rather than by explicit differentiation (Roberts, 6 Oct 2025). The basic trust-region loop follows the structure of derivative-based nonlinear optimization: construct a local model $m_k$, compute a trust-region step by approximately minimizing that model within radius $\Delta_k$, evaluate the true objective at the trial point, and update both the trust-region radius and the interpolation set (Roberts, 6 Oct 2025).

The standard local quadratic model is written as

$$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$

with coefficients obtained from interpolation conditions rather than derivatives (Roberts, 6 Oct 2025). Linear interpolation uses $n+1$ well-poised points, whereas full quadratic interpolation uses $\frac{(n+1)(n+2)}{2}$ points (Roberts, 6 Oct 2025). The same broad interpolation principle also appears outside DFO. In differential equation inverse problems, measured trajectories are interpolated to avoid repeated numerical integration and expensive solver sensitivity calculations (Prabhu et al., 31 May 2025). In convex optimization, interpolation defines an inexpensive feasible mapping $g(x)$ that replaces exact projection (Akrour et al., 2020). In metric differential privacy, perturbation distributions are optimized at anchor points and interpolated elsewhere by log-convex combinations that preserve privacy guarantees (Qiu, 15 Jan 2026).

This suggests a useful conceptual distinction. In a narrow sense, IBO refers to interpolation-based local surrogate optimization in DFO (Roberts, 6 Oct 2025). In a broader sense, it refers to optimization frameworks in which interpolation is the mechanism by which unavailable derivatives, dense constraints, or discontinuous objectives are replaced by tractable approximations or structurally faithful intermediate representations (Prabhu et al., 31 May 2025, Akrour et al., 2020, Qiu, 15 Jan 2026).

2. Core trust-region methodology in derivative-free optimization

The standard IBO trust-region method starts from $x_0$ and $\Delta_0>0$. At each iteration, an interpolation-based model is built near the current iterate from function values at a selected point set; a trust-region subproblem

$$s_k \approx \arg\min_{\|s\| \leq \Delta_k} m_k(x_k + s)$$

is then solved approximately (Roberts, 6 Oct 2025). The step is accepted or rejected using the ratio

$$\rho_k = \frac{f(x_k) - f(x_k + s_k)}{m_k(x_k) - m_k(x_k + s_k)},$$

and the interpolation set is updated to maintain model quality (Roberts, 6 Oct 2025).

A central technical concept is poisedness: the interpolation points must be geometrically well placed so that the interpolation system is invertible and numerically stable (Roberts, 6 Oct 2025, Larson et al., 27 May 2026). In Birkhoff interpolation models, data consist of pairs $\Delta_k$0, where $\Delta_k$1 denotes which derivative information is interpolated at $\Delta_k$2. The associated interpolation matrix must be invertible for the set to be poised, and the quality of this poisedness is quantified by a Birkhoff poisedness constant $\Delta_k$3 (Larson et al., 27 May 2026). The same issue appears in classical model-based DFO, where Lagrange polynomial norms and related geometry constants determine the quality of the interpolation set and hence the model accuracy (Roberts, 6 Oct 2025).

A persistent theme in modern IBO is selective enrichment of the interpolation conditions. Classical derivative-free interpolation uses only function values. The Birkhoff framework generalizes this by allowing arbitrary patterns of derivative data: function values at some points, selected partial derivatives at others, and even higher-order derivatives where available, without requiring the Hermite condition that all available derivatives be queried at every interpolation point (Larson et al., 27 May 2026). This naturally bridges derivative-free and derivative-based settings (Larson et al., 27 May 2026).

3. Approximation theory, complexity, and geometric management

The approximation-theoretic backbone of IBO is the notion of fully linear and fully quadratic models. A model $\Delta_k$4 is fully linear on $\Delta_k$5 if

$\Delta_k$6

throughout the trust region (Roberts, 6 Oct 2025). Fully quadratic models satisfy the stronger bounds

$\Delta_k$7

(Roberts, 6 Oct 2025). In the Birkhoff framework, analogous fully quadratic error estimates are established with constants depending on the problem dimension, Lipschitz constants, and the poisedness constant $\Delta_k$8: $\Delta_k$9 (Larson et al., 27 May 2026).

These model properties support worst-case evaluation complexity bounds. For first-order criticality, IBO requires at most

$$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$0

function evaluations to find a point with $$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$1 (Roberts, 6 Oct 2025). For second-order criticality, the bound is

$$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$2

to achieve $$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$3 (Roberts, 6 Oct 2025). These rates match the structure of gradient-based trust-region theory, although the constants reflect interpolation geometry and dimension (Roberts, 6 Oct 2025).

Geometric self-correction is a hallmark of the methodology. Poorly poised point sets degrade approximation quality, so modern IBO algorithms actively repair geometry by replacing or adding points in the trust region (Roberts, 6 Oct 2025). In one radial-basis-function trust-region method, a valid model is guaranteed by maintaining a well-poised subset that approximately exhibits regular simplex geometry; the stated consequence is improved point scattering and curvature in the surrogate model (Lefebvre et al., 2018). That method uses $$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$4 points rather than the usual $$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$5 simplex geometry, specifically to guarantee curvature and support ellipsoidal trust-region normalization (Lefebvre et al., 2018).

4. Model construction variants

Interpolation-based optimization encompasses several model classes, each targeting a different structural regime.

A prominent classical variant is sparse quadratic interpolation. When the Hessian is sparse but its sparsity pattern is unknown, one can construct a quadratic model by minimizing the $$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$6-norm of the quadratic coefficients subject to interpolation constraints,

$$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$7

thereby importing compressed-sensing recovery ideas into model construction (Bandeira et al., 2013). Under random sampling and sparsity assumptions, $$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$8 function values suffice to recover accurate fully quadratic models when the Hessian has $$m_k(y) = c_k + g_k^T (y - x_k) + \frac{1}{2} (y - x_k)^T H_k (y - x_k),$$9 nonzeros (Bandeira et al., 2013). The same paper reports that a trust-region method using minimum $n+1$0-norm quadratic models is more efficient than minimum Frobenius norm models and NEWUOA on sparse-Hessian problems (Bandeira et al., 2013).

Another line uses radial basis functions and Universal Kriging. In the ellipsoidal trust-region method, the surrogate has the form

$n+1$1

with parameters determined from the interpolation system

$n+1$2

(Lefebvre et al., 2018). This framework also introduces an ellipsoidal trust region

$n+1$3

where the affine transformation is chosen to normalize local curvature (Lefebvre et al., 2018).

A distinct interpolation tradition uses rational rather than polynomial models. Iteration methods based on barycentric rational interpolation were derived for univariate optimization and root search. For optimization, the derivative-free methods approach convergence order $n+1$4, while the first-derivative methods approach $n+1$5; in the root-search setting, the corresponding limits are quadratic and cubic (Cassel, 2020). These methods are “self-accelerating” in the sense that additional memory points improve asymptotic order, although practical gains saturate after a modest memory size (Cassel, 2020).

At the opposite end of the spectrum are structure-specific interpolation schemes. For second-order nonlinear ODEs with mixed boundary conditions, trigonometric interpolation is applied to the second derivative $n+1$6, represented by

$n+1$7

with FFT-based coefficient computation (Zou, 27 Apr 2025). The solution is then reconstructed by integration, the boundary conditions determine the integration constants, and an objective measuring ODE residuals on the grid is minimized (Zou, 27 Apr 2025). The reported complexity reduction for gradient evaluation is from $n+1$8 to $n+1$9 through FFT-based formulas (Zou, 27 Apr 2025).

5. Bi-level and structure-exploiting formulations

A major recent development is the use of interpolation inside bi-level optimization. In parameter estimation for ODEs and DDEs, the measured states $\frac{(n+1)(n+2)}{2}$0 are replaced by a smooth surrogate $\frac{(n+1)(n+2)}{2}$1, so that the differential equation right-hand side separates into parts linear in parameters $\frac{(n+1)(n+2)}{2}$2 and possibly nonlinear in $\frac{(n+1)(n+2)}{2}$3 (Prabhu et al., 31 May 2025). The resulting problem is cast as

$\frac{(n+1)(n+2)}{2}$4

and interpolation yields the approximation

$\frac{(n+1)(n+2)}{2}$5

(Prabhu et al., 31 May 2025). For fixed $\frac{(n+1)(n+2)}{2}$6, the inner problem is convex and linear in $\frac{(n+1)(n+2)}{2}$7, while the outer problem optimizes $\frac{(n+1)(n+2)}{2}$8 (Prabhu et al., 31 May 2025).

The Lagrangian and KKT system of the inner convex problem enable efficient sensitivity computation for the outer optimization by the implicit function theorem (Prabhu et al., 31 May 2025). The stated advantage over single- or multiple-shooting is that ODE solver sensitivities are eliminated or significantly reduced, since derivatives are taken through the interpolated trajectory instead (Prabhu et al., 31 May 2025). The paper applies this framework to parameter estimation for ODEs and DDEs and to model discovery, including an example involving simultaneous selection of functional forms and estimation of linear and nonlinear parameters (Prabhu et al., 31 May 2025).

Bi-level structure also appears in neural optimal interpolation for signal reconstruction. There, interpolation is written as a variational problem

$\frac{(n+1)(n+2)}{2}$9

and the overall learning problem is posed as

$g(x)$0

(Beauchamp et al., 2022). The trainable solver asymptotically converges to the classical optimal interpolation solution for stationary and non-stationary linear spatio-temporal Gaussian processes, while also extending to nonlinear and multimodal settings (Beauchamp et al., 2022).

6. Extensions beyond classical derivative-free local optimization

Several papers use interpolation as the key mechanism for enforcing constraints, preserving guarantees, or reshaping loss geometry.

For convex optimization over a feasible set $g(x)$1, an interpolation-based projection is defined using a strictly feasible anchor $g(x)$2: $g(x)$3 The algorithm does not use this as a post-step projection; instead, it optimizes the composite objective $g(x)$4, with update

$g(x)$5

(Akrour et al., 2020). For linear objectives and convex Lipschitz constraints, the paper proves

$g(x)$6

establishing an $g(x)$7 convergence rate (Akrour et al., 2020). The paper also reports that differentiable interpolation projection is about $g(x)$8 slower than unconstrained models but up to $g(x)$9 faster than differentiable orthogonal projection layers in the reported neural-network experiments (Akrour et al., 2020).

In continuous and fine-grained metric differential privacy, the anchor-based interpolation principle is explicitly formalized as an optimization framework. The domain is partitioned into orthotopic cells, perturbation distributions are optimized at anchor points, and non-anchor distributions are interpolated via log-convex combinations of anchor log-probabilities (Qiu, 15 Jan 2026). Because naïve high-dimensional interpolation can violate privacy, the interpolation is decomposed into axis-wise one-dimensional steps with per-dimension privacy budgets satisfying

$x_0$0

(Qiu, 15 Jan 2026). The paper reports that the proposed mechanism achieves $x_0$1 mDP violations at all locations, all budgets, and for both $x_0$2 and $x_0$3 norms on the evaluated location datasets (Qiu, 15 Jan 2026).

In object detection, interpolation enters directly into the regression objective. InterpIoU defines an interpolated box

$x_0$4

and uses the loss

$x_0$5

(Liu et al., 16 Jul 2025). The paper states that interpolation provides meaningful gradients in non-overlapping cases and avoids the box enlargement issue caused by misaligned geometric penalties (Liu et al., 16 Jul 2025). A dynamic variant uses

$x_0$6

to adapt interpolation strength to current localization quality (Liu et al., 16 Jul 2025).

7. Applications, misconceptions, and limitations

Interpolation-based optimization has been applied across black-box optimization, inverse problems, interpolation/reconstruction, privacy, treatment planning, and multi-objective optimization. In expensive multi-objective optimization, one two-phase framework first solves a small number of scalarized subproblems and then fits a Bézier simplex

$x_0$7

to interpolate a sparse Pareto approximation (Tanabe et al., 2022). On 55 bi-objective BBOB problems, the paper reports that this two-phase approach performs significantly better than HMO-CMA-ES and some state-of-the-art meta-model-based optimizers (Tanabe et al., 2022). In surrogate-assisted multi-objective optimization, KKT-guided manifold interpolation generates candidates along the approximated Pareto-set manifold, and experiments on 136 benchmark instances are reported to show faster convergence and stronger resilience to irregular Pareto-front shapes (Li et al., 2021).

A common misconception is that interpolation in optimization is merely a numerical convenience for estimating derivatives. The literature indicates a wider role. In differential equations, interpolation can change the problem structure so that an inner parameter-estimation problem becomes convex (Prabhu et al., 31 May 2025). In privacy, interpolation is not only an approximation device but the mechanism by which guarantees are extended from anchors to the full domain (Qiu, 15 Jan 2026). In convex constrained learning, interpolation is used to define a cheap feasible mapping whose differentiability is essential to the algorithm (Akrour et al., 2020).

Another misconception is that interpolation necessarily sacrifices rigor. In the DFO literature, fully linear and fully quadratic models carry explicit approximation constants and support worst-case complexity theory (Roberts, 6 Oct 2025). The Birkhoff framework establishes poisedness conditions and fully quadratic error bounds even when derivative information is heterogeneous and incomplete (Larson et al., 27 May 2026). The privacy setting provides provable preservation of $x_0$8-norm mDP under the corrected axis-wise interpolation scheme (Qiu, 15 Jan 2026).

The principal limitations are equally recurrent. Interpolation quality depends on geometry or data density. Poorly poised interpolation sets cause ill-conditioning in trust-region DFO (Roberts, 6 Oct 2025). Sparse or noisy measurements can degrade interpolated trajectories and hence parameter estimates in differential equation problems (Prabhu et al., 31 May 2025). High dimensionality increases the size and maintenance cost of interpolation sets, and in stochastic settings sample sizes may scale as

$x_0$9

so sampling effort grows rapidly as the trust region shrinks (Roberts, 6 Oct 2025). In application-specific IBO variants, the structural assumptions also matter: the Bézier-simplex approach assumes a simplicial Pareto front, and its advantage can be reduced on highly multimodal or non-simplicial problems (Tanabe et al., 2022).

Taken together, these lines of work indicate that interpolation-based optimization is no longer a single algorithmic template. It is a technically diverse optimization paradigm whose unifying principle is the replacement of inaccessible, unstable, or poorly structured optimization objects by interpolated surrogates, interpolated feasible mappings, or interpolated auxiliary targets, with the interpolation chosen so that optimization remains tractable while preserving the relevant geometry, physics, or guarantees (Roberts, 6 Oct 2025, Prabhu et al., 31 May 2025, Larson et al., 27 May 2026, Akrour et al., 2020, Qiu, 15 Jan 2026).