Optimization via Integration Techniques

• Optimization via Integration (OVI) is a framework that replaces direct objective minimization with integration-based approaches to exploit structural properties.
• It encompasses methods like Bayesian quadrature for expensive integrals, numerical integration of gradient flows, and gradient-free probabilistic or spherical consensus updates.
• OVI techniques balance computational efficiency and robustness while opening avenues for VOI extensions, hybrid strategies, and advanced dynamical formulations.

Searching arXiv for recent and foundational papers on "Optimization via Integration" to ground the article. Optimization via Integration (OVI) denotes a non-standard but recurring label for optimization procedures that exploit an integral, expectation, or continuous-time dynamical representation of the problem rather than treating the objective as an unstructured map. In the literature considered here, the label covers several distinct constructions: optimization of objectives that are themselves sums or integrals via Bayesian Quadrature Optimization; interpretation of accelerated first-order methods as numerical integrators of gradient flow; gradient-free schemes that update parametric or spherical search distributions by expectation-weighted averaging; and reformulations of parameter estimation or optimal control as boundary-value or initial-value problems solved by ODE integration (Toscano-Palmerin et al., 2018, Scieur et al., 2017, Andrieu et al., 2024, Varner et al., 5 Mar 2025, Kaschek et al., 2013, Zhang et al., 2017, Ackermann et al., 24 Jun 2026).

1. Terminology and scope

One paper states explicitly that “OVI” is not yet a standard term, and the surrounding literature confirms that the acronym has been used for several related but non-identical ideas. In all cases, however, the central move is to replace a direct optimization step by an operation built around integration, expectation, smoothing, or continuous-time evolution (Varner et al., 5 Mar 2025).

Usage of “OVI”	Core construction	Representative papers
Expensive-integrand optimization	Optimize $G(x)$ by modeling the integrand $F(x,w)$ and integrating it	(Toscano-Palmerin et al., 2018, Varner et al., 5 Mar 2025)
Integration view of first-order methods	Derive discrete algorithms by integrating gradient flow	(Scieur et al., 2017)
Gradient-free probabilistic or spherical optimization	Reweight samples and update a parametric distribution or consensus point	(Andrieu et al., 2024, Ackermann et al., 24 Jun 2026)
Dynamical-systems reformulation	Encode optimality conditions as BVP or IVP dynamics	(Kaschek et al., 2013, Zhang et al., 2017)

A separate source uses OVI for “Optimization-based Visual Inversion” and states explicitly that there OVI stands for Optimization-based Visual Inversion, not “via Integration.” That usage belongs to text-to-image generation and is terminologically distinct from the optimization literature summarized here (Dell'Erba et al., 25 Nov 2025).

This suggests that “Optimization via Integration” is best understood as a family of viewpoints rather than a single algorithm. The common thread is structural exploitation: the optimizer is designed around an integral, a smoothed functional, a GP-induced expectation, or an auxiliary dynamical system whose equilibrium satisfies the original optimality conditions.

2. Integral-structured objectives and Bayesian Quadrature Optimization

In the most literal sense, OVI is the problem of optimizing an objective that is itself an integral or a sum over an expensive-to-evaluate function. The canonical forms are

$$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$

with observations

$$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$

This formulation covers multi-task BO, cross-validation, optimization via simulation, and average-case design under random environments (Toscano-Palmerin et al., 2018).

The key methodological move in Bayesian Quadrature Optimization (BQO) is to place a GP prior on the integrand $F(\cdot,\cdot)$ rather than on $G$ directly. The posterior on $F$ induces a posterior on the integrated objective by Bayesian quadrature:

$a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$

with an analogous covariance expression obtained by integrating $\Sigma_n$. For finite sums, the same construction reduces to a weighted sum over $w$. This is the formal reason BQO can exploit smoothness in the integrated variable $F(x,w)$0 and perform variance reduction by conditioning on $F(x,w)$1 instead of averaging it away (Toscano-Palmerin et al., 2018).

Sampling is then driven by a one-step value of information criterion. With hyperparameters treated as known,

$F(x,w)$2

The BQO acquisition chooses $F(x,w)$3 to maximize the posterior expectation of this quantity over GP hyperparameters. The paper states that this criterion is Bayes-optimal if only one sample remains, and for finite-sum OVI it further proves asymptotic Bayes optimality. A crucial representation is

$F(x,w)$4

which makes unbiased Monte Carlo estimation of both the VOI and its gradient possible without discretizing the domain (Toscano-Palmerin et al., 2018).

The computational contribution is a discretization-free stochastic-gradient method for optimizing VOI. The paper gives an unbiased, strongly consistent estimator of $F(x,w)$5 and an unbiased stochastic gradient estimator via the envelope theorem. In contrast to discretized approaches, which scale exponentially in the dimension of $F(x,w)$6, the discretization-free approach has sub-exponential scaling and was demonstrated up to $F(x,w)$7 dimensions. Theoretical guarantees include consistency for continuum $F(x,w)$8 with finite $F(x,w)$9 and for finite $$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$0 and finite $$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$1, both expressed as

$$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$2

Empirically, the method performs as well or better than previous sum- or integral-aware methods and is especially effective when evaluations are noisy or when the integrand varies smoothly in the integrated variables; on GP-simulated problems, gains up to $$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$3 over KG are reported (Toscano-Palmerin et al., 2018).

The same problem class also motivates a different line of work in which objective evaluations are expensive because they require numerical integration while gradients can be obtained comparatively cheaply. That setting remains OVI in the literal “objective via integration” sense, but it leads to first-order event-triggered algorithms rather than GP-based Bayesian optimization; that variant is discussed separately below (Varner et al., 5 Mar 2025).

3. Numerical integration of gradient flow and accelerated optimization

A second, conceptually different meaning of OVI is the viewpoint that discrete optimization algorithms arise by numerically integrating the gradient flow ODE

$$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$4

For $$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$5-strongly convex $$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$6, the flow satisfies exponential energy decay, while for $$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$7-smooth convex $$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$8 it satisfies a sublinear bound. The paper’s central claim is that acceleration can be understood through the choice of integration scheme: if a discrete method tracks $$G(x) = \int F(x,w)\,p(w)\,dw \qquad\text{or}\qquad G(x) = \sum_{i=1}^m F(x,w_i)\,p(w_i),$$9 with $$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$0, then larger stable $$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$1 yields faster progress along the same continuous descent trajectory (Scieur et al., 2017).

The generic discretization is a linear multi-step method,

$$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$2

or, with shift-operator notation,

$$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$3

Within this framework, classical numerical integrators such as Adams–Bashforth, Adams–Moulton, and BDF methods become optimization algorithms once applied to gradient flow. The paper then shows that heavy-ball and Nesterov’s accelerated gradient correspond to specific two-step schemes with carefully chosen stability polynomials (Scieur et al., 2017).

For strongly convex quadratics, the relevant stability object is the characteristic polynomial

$$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$4

Gradient descent, viewed as explicit Euler, is stable only for $$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$5, with best uniform contraction at $$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$6. By contrast, the two-step designs corresponding to AGD and heavy-ball admit step sizes of order $$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$7, rather than $$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$8, while remaining stable on the spectral interval. This is the paper’s explanation for acceleration: multi-step schemes cancel lower-order discretization error, and the resulting larger stable time step manifests as momentum or lookahead (Scieur et al., 2017).

The discrete convergence statements align with standard optimization rates. Gradient descent contracts like $$y(x,w) = F(x,w) + \epsilon,\qquad \epsilon \sim \operatorname{Normal}(0,\lambda_{(x,w)}).$$9, while heavy-ball and AGD achieve $F(\cdot,\cdot)$0 in the strongly convex case, with $F(\cdot,\cdot)$1. In the merely convex regime, the time-varying-step AGD discretization yields a discrete estimate close to the standard $F(\cdot,\cdot)$2 rate, while gradient descent tracks $F(\cdot,\cdot)$3. The same integration viewpoint also recovers implicit Euler as the proximal point method and extends to composite and mirror settings through IMEX and non-Euclidean gradient flows (Scieur et al., 2017).

The paper is explicit about limitations. Heavy-ball can fail to converge on general non-quadratic smooth strongly convex functions, stochastic gradients reduce effective stability regions, and implicit schemes trade robustness for the cost of solving implicit equations. In this strand of the literature, OVI is therefore not about integral objectives but about deriving optimization algorithms from numerical integration theory.

4. Gradient-free OVI through probabilistic reweighting and spherical consensus

A third meaning of OVI is a gradient-free optimization framework in which the iterate is updated by fitting a parametric probability density to a reweighted version of the objective. In the exponential-family formulation,

$F(\cdot,\cdot)$4

or, for exponential dispersion models,

$F(\cdot,\cdot)$5

At iteration $F(\cdot,\cdot)$6, the method forms a reweighted density

$F(\cdot,\cdot)$7

and projects $F(\cdot,\cdot)$8 back onto the parametric family by minimizing $F(\cdot,\cdot)$9. For exponential families this yields the moment-matching recursion

$G$0

In the Gaussian case with fixed covariance $G$1, the update reduces to a weighted average of the samples (Andrieu et al., 2024).

This reweight-and-project step is the basis for both plain Monte Carlo and SMC implementations. The method requires only evaluations of $G$2 or noisy pointwise measurements $G$3; it does not use gradients of $G$4. Theoretical analysis proceeds through the Laplace-smoothed objectives

$G$5

or, in the Gaussian case,

$G$6

The paper proves epi-convergence of the smoothed objectives to $G$7 under lower-boundedness and strong lower-semicontinuity, gives a descent lemma with Bregman divergence for exponential dispersion models, and establishes a time-inhomogeneous gradient-descent theorem yielding a subsequence with vanishing gradients. With $G$8 and $G$9, the Gaussian OVI scheme falls under this theory (Andrieu et al., 2024).

A more recent spherical black-box optimization paper uses OVI for a parametric optimizer based on log-sum-exp aggregation under Gaussian smoothing:

$F$0

Gradient descent on this objective with effective step size $F$1 yields the weighted-consensus update

$F$2

so the weights are a softmax over negative losses. The same paper gives an equivalent spherical formulation and places OVI, ES, CBO, pCBO, cCBO, and Diffusion Evolution inside a single master update whose two main design axes are fitness aggregation and consensus scope (Ackermann et al., 24 Jun 2026).

Within that framework, OVI is the global-consensus, log-sum-exp, resampling case; ES corresponds to an affine transformation of fitness; and CBO adds transport and distance-scaled noise. The paper then introduces ES-OVI hybrids,

$F$3

to interpolate between sharpness preference and flatness preference, together with CBO-OVI hybrids such as SchedPol and AdaPol to interpolate between parametric efficiency and multimodality. Empirically, the hybrid methods are evaluated on BBOB, Brax, and LLM merging, and the paper reports that they can outperform their constituent algorithms (Ackermann et al., 24 Jun 2026).

Across these two gradient-free strands, the common structure is explicit reweighting by $F$4 or $F$5 followed by an integration or expectation step. The 2024 paper emphasizes KL projection, moment matching, and epi-convergence on nonsmooth objectives, whereas the 2026 paper emphasizes spherical smoothing, consensus updates, and a unifying master update for black-box optimizers (Andrieu et al., 2024, Ackermann et al., 24 Jun 2026).

5. Expensive objective integrals and event-driven objective evaluations

A fourth strand treats OVI as the class of problems in which the objective depends on an integral or expectation that is expensive to evaluate numerically, while the gradient can be computed comparatively cheaply. Canonical forms include

$F$6

The paper’s motivating examples come from semi-parametric statistics, control, and sub-problems in global optimization. It emphasizes that gradients can be cheaper because of the fundamental theorem of calculus, differentiation under the integral sign, or reuse of quadrature nodes and weights under AD/adjoint differentiation (Varner et al., 5 Mar 2025).

The central negative result is an anti-convergence statement for objective-function-free optimization (OFFO) methods. Under two abstract conditions—no objective dependence and diverging iterates on simple convex examples—the paper constructs a locally Lipschitz smooth, lower-bounded function for which the OFFO sequence satisfies

$F$7

The point is not that all gradient-only methods fail, but that for this OVI class the absence of occasional objective checks can be catastrophic even when gradients remain bounded or oscillatory (Varner et al., 5 Mar 2025).

The proposed remedy is an event-driven first-order method with outer accepted iterates $F$8 and inner trial iterates $F$9. The inner loop uses only gradients:

$a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$0

with adaptive

$a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$1

and local Lipschitz estimates from gradient differences. An expensive objective evaluation is triggered only if the trial point exits a trust ball of radius $a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$2, its gradient norm leaves a prescribed interval, or the inner-loop cap $a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$3 is reached. At that point a non-sequential Armijo test is applied:

$a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$4

Acceptance or rejection then updates $a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$5 and rescales $a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$6 (Varner et al., 5 Mar 2025).

The theory is unusually explicit. The paper proves boundedness of the adaptive step sizes, compactness of inner iterates, an acceptance theorem preventing OFFO-like divergence, a Zoutendijk-type global alternative, a complexity bound

$a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$7

and a local PL linear rate once an isolated local PL region is entered. Empirically, on quasi-likelihood estimation problems with four variance functions and multiple dimensions, the method found near-stationary points on roughly $a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$8–$a_n(x) := E_n[G(x)] = \int \mu_n(x,w)\,p(w)\,dw,$9 of runs, showed far less sensitivity to hyperparameters than OFFO baselines, and reduced the number of expensive integral evaluations relative to classical line-search approaches (Varner et al., 5 Mar 2025).

In this variant, OVI is again literal: the objective is an integral or expectation. The distinguishing contribution is the event-triggered separation between cheap gradient steps and sparse, strategically placed expensive objective evaluations.

6. Boundary-value and variation-time formulations in dynamical systems

In parameter estimation for ODEs, OVI appears as a reformulation in which solving the differential equation and optimizing the likelihood are performed in one step. The construction begins from the augmented dynamics

$\Sigma_n$0

so parameters become constant states. The discrete negative log-likelihood is then continued to time using an interpolant $\Sigma_n$1 of the observations, yielding an integral objective. With sensitivities

$\Sigma_n$2

the gradient of the continued likelihood with respect to the initial augmented state satisfies a gradient ODE

$\Sigma_n$3

with endpoint constraints

$\Sigma_n$4

The resulting BVP couples state, sensitivity, and gradient dynamics; solving it yields the initial conditions, parameters, and trajectories that satisfy the stationarity conditions of the continued likelihood (Kaschek et al., 2013).

The empirical illustration is the Lotka–Volterra system. In the fully observed case, with $\Sigma_n$5 initial parameter guesses drawn by Latin hypercube sampling over $\Sigma_n$6 orders of magnitude, the single-shooting Levenberg–Marquardt baseline found the global optimum in only about $\Sigma_n$7 of cases, while the BVP method was roughly $\Sigma_n$8 times more efficient at finding the best optimum. In the partially observed case, both methods had global convergence rates below $\Sigma_n$9, but restricting initial guesses to the negative orthant increased the BVP method’s success rate by about a factor of $w$0 over single-shooting (Kaschek et al., 2013).

A related but distinct OVI construction for optimal control is the Variation Evolving Method (VEM). Here the optimization variables evolve in an artificial variation time $w$1, distinct from the physical time $w$2, and the equilibrium of the $w$3-dynamics satisfies the necessary optimality conditions. For the OCP with Bolza cost

$w$4

and dynamics $w$5, the method introduces the Hamiltonian

$w$6

and a convex residual functional

$w$7

The $w$8-flow descends $w$9:

$F(x,w)$00

with accompanying boundary and terminal-time evolution equations. The paper states that the $F(x,w)$01-dynamics are globally stable and that any initial value will evolve to the extremal solution ultimately (Zhang et al., 2017).

Numerically, VEM turns the OCP into an IVP in $F(x,w)$02 after discretization in the physical time $F(x,w)$03, so mature ODE solvers can be used. The paper reports a calculus-of-variations example, a linear-quadratic example solved with ode15s, and a brachistochrone problem whose terminal time converged to $F(x,w)$04, compared with $F(x,w)$05 from GPOPS-II. The stated limitation is that the present functional $F(x,w)$06 is not directly applicable to general state/control path constraints (Zhang et al., 2017).

These two dynamical-systems variants differ technically—one is a BVP for likelihood stationarity, the other an IVP in variation time for Pontryagin residuals—but both exemplify optimization via integration in the strong sense that optimality is obtained by integrating an auxiliary dynamical system whose terminal or equilibrium conditions encode the target solution.

7. Shared principles, trade-offs, and open questions

The surveyed papers identify a common methodological pattern: optimization is made tractable by exposing latent structure that can be integrated, smoothed, conditioned on, or evolved. In BQO, the exploitable object is the integrand $F(x,w)$07; in the gradient-flow view it is the ODE $F(x,w)$08; in the gradient-free probabilistic view it is a parametric family whose reweighted moments can be computed; in event-driven OVI it is the separation between cheap gradients and expensive objective integrals; and in BVP/VEM formulations it is the dynamical system whose sensitivity or residual equations encode optimality (Toscano-Palmerin et al., 2018, Scieur et al., 2017, Andrieu et al., 2024, Varner et al., 5 Mar 2025, Kaschek et al., 2013, Zhang et al., 2017, Ackermann et al., 24 Jun 2026).

The trade-offs are correspondingly heterogeneous. GP-based OVI depends on kernel specification, tractable or numerically manageable integral terms, and nontrivial VOI optimization. Gradient-flow OVI clarifies acceleration but does not by itself remove conditioning or stochasticity issues. Reweight-and-project OVI and spherical OVI can suffer weight degeneracy, sensitivity to $F(x,w)$09 or $F(x,w)$10, and difficulties with multimodality or high dimension. Event-driven OVI depends on accurate gradients and local Lipschitz behavior, while the BVP and VEM formulations enlarge the dynamical system substantially and can become sensitive to initialization, stiffness, or missing observations (Andrieu et al., 2024, Varner et al., 5 Mar 2025, Kaschek et al., 2013, Zhang et al., 2017, Ackermann et al., 24 Jun 2026).

Several open directions are stated explicitly. For BQO, these include cost-aware VOI extensions and receding-horizon approximations; for the exponential-family gradient-free framework, mixture families, covariance adaptation, and almost-sure convergence in the noisy case; for spherical black-box OVI, better control of sharpness, robustness, and multimodality through hybrids; and for VEM, incorporation of path constraints through KKT or slack-variable machinery (Toscano-Palmerin et al., 2018, Andrieu et al., 2024, Ackermann et al., 24 Jun 2026, Zhang et al., 2017).

This suggests that “Optimization via Integration” is not a single canonical method but a family resemblance term for optimization strategies that relocate computational effort from direct objective minimization to integration-centric surrogates, update rules, or dynamical evolutions. The exact meaning therefore depends on which structure is being integrated: an expensive integrand, a gradient flow, a reweighted probability law, or the optimality conditions of a dynamical system.