Birkhoff interpolation models for optimization with some available derivatives

Abstract: We consider interpolation-based derivative-free optimization in settings where only some derivatives are available. Such situations arise naturally in scientific computing applications involving simulations, adjoint-enabled components, legacy software, or partially differentiable models. We introduce a Birkhoff interpolation framework that permits arbitrary patterns of derivative availability and enables the con- struction of local polynomial models using mixtures of function values and partial derivative informa- tion. In contrast to Hermite interpolation approaches, the proposed framework does not require all available derivatives to be queried at every interpolation point. We develop conditions under which the resulting interpolation systems are poised and establish corresponding model-accuracy bounds for fully quadratic interpolation models. We develop a trust-region framework that maintains poised interpolation sets while selectively incorporating derivative information. The method generalizes an established class of interpolation-based derivative-free optimization algorithms and naturally bridges derivative-free and derivative-based settings. We evaluate the proposed approach on a collection of CUTEst test problems with synthetically generated derivative-availability patterns.

• The paper extends derivative-free trust-region optimization by using Birkhoff interpolation to combine function values and selectively available partial derivatives without requiring every derivative at each sample point.
• The authors establish poisedness and conditioning results, fully quadratic model-error bounds with explicit constants, and convergence to second-order stationary points under Lipschitz continuity assumptions.
• Numerical tests on 89 CUTEst problems show that selective derivative incorporation is more robust than Hermite-style bundling, especially when only 25% of variables have available derivatives.

Problem setting and motivation

This paper addresses unconstrained optimization problems of the form $\min_{x\in\mathbb{R}^n} f(x)$ in which an oracle returns only a subset of the partial derivatives of $f$. The motivating applications are scientific computing problems—most prominently stellarator design in magnetic confinement fusion, where derivatives with respect to coil parameters are inexpensive (e.g., via adjoint methods) while derivatives with respect to plasma parameters are costly or unavailable. The authors formalize this through a finite available set $A$ of multi-indices $\alpha$, where $\alpha \in A$ indicates that $\partial^\alpha f$ may be queried, with $0 \in A$ and $\|\alpha\|_\infty \le 2$ for quadratic modeling.

The work generalizes the classical interpolation-based derivative-free optimization framework of Conn, Scheinberg, and Vicente [CSV2009], which assumes only zeroth-order information ($A = \{0\}$). The key distinction from prior partial-derivative approaches is that the proposed framework does not require all available derivatives to be queried at every interpolation point. In particular, it departs from Hermite interpolation—which mandates that if any derivative is interpolated at a point, all lower-order derivatives must also be interpolated there—and instead adopts Birkhoff interpolation, which permits arbitrary patterns of function-value and derivative conditions across interpolation nodes.

Birkhoff interpolation framework

A Birkhoff interpolation model is a quadratic polynomial $m(y^0 + s) = c + g^T s + \tfrac12 s^T H s$ centered at $f$0, satisfying conditions $f$1 for $f$2 inseparable pairs $f$3, where $f$4. Lagrange interpolation is recovered when all $f$5; Hermite interpolation is a special case of the more permissive Birkhoff setting.

Because entries of the interpolation matrix $f$6 scale with different powers of the trust-region radius, the authors introduce a row-and-column scaling yielding a normalized matrix $f$7 whose nonzero entries are of order one. All poisedness analysis is conducted on $f$8.

Poisedness theory

The paper defines poisedness via invertibility of $f$9, constructs Birkhoff interpolation polynomials $A$0 satisfying $A$1 on normalized points, and proves their existence and uniqueness for poised data. An interpolant admits the representation

$A$2

established via the chain rule. A $A$3-poisedness constant is then defined analogously to the Lagrange case: when $A$4, the Birkhoff definition coincides exactly with the Lagrange poisedness constant, so the framework strictly generalizes the classical theory.

The central structural result relates poisedness to conditioning: if $A$5, then $A$6 is $A$7-poised; conversely, $A$8-poisedness implies $A$9. This yields a uniform lower bound on $\alpha$0 for $\alpha$1-poised data. Notably, the geometry of a well-poised set depends jointly on sample locations and on which derivative multi-indices are paired with each point—a heatmap study shows that different distributions of derivative conditions over similar point sets can produce substantially different poisedness constants.

Model-building algorithms

Two greedy pivoting subroutines maintain well-poised data:

• Model completion: builds a poised set $\alpha$2 from arbitrary input data using pivot polynomials spanning $\alpha$3, selecting at each step the condition $\alpha$4 maximizing $\alpha$5, with fallback to fresh sampling within the trust region if no existing datum exceeds a threshold $\alpha$6. Termination is guaranteed for sufficiently small $\alpha$7.
• Model improvement: replaces a single interpolation condition when doing so improves the final pivot value by a factor $\alpha$8.

A subtle but important design choice is that the algorithm does not fix an a priori ordering of derivative multi-indices. The authors demonstrate by explicit counterexample in two dimensions that fixing $\alpha$9 at each pass can drive a pivot polynomial's relevant derivative identically to zero, causing failure; flexibility in pairing multi-indices with pivots avoids this. Weighting functions $\alpha \in A$0 can be introduced to prioritize conditions without breaking the guarantees, provided they are positive and uniformly bounded.

Convergence analysis

The main theoretical result establishes that a Birkhoff interpolant built from $\alpha \in A$1-poised data is a fully quadratic model. Writing $\alpha \in A$2 for the number of order-$\alpha \in A$3 conditions in $\alpha \in A$4, the error constants take the form

$\alpha \in A$5

with analogous expressions for $\alpha \in A$6 and $\alpha \in A$7 involving $\alpha \in A$8 and $\alpha \in A$9 error bounds respectively. The proof assembles Taylor-expansion identities across the three derivative-order cases into a linear system whose rescaled coefficient matrix $\partial^\alpha f$0 satisfies $\partial^\alpha f$1 as a submatrix, so $\partial^\alpha f$2. The result is a specialization of Ciarlet–Raviart Birkhoff error bounds with exact constants. Combined with standard trust-region machinery, this yields convergence of model centers to second-order stationary points under Assumptions 1–2 (Lipschitz continuity of $\partial^\alpha f$3 and $\partial^\alpha f$4 on the enlarged sublevel set, bounded below).

An important caveat: the analysis requires at least one Lagrange condition ($\partial^\alpha f$5) to pin down the constant term, and the available set $\partial^\alpha f$6 is assumed known and fixed in advance. The experiments do not exploit differential evaluation costs $\partial^\alpha f$7 of available derivatives—an avenue the authors explicitly leave open despite noting that reverse-mode AD costs are provably within a small constant factor of function evaluations.

Numerical results

The method was tested on 89 unconstrained CUTEst problems with $\partial^\alpha f$8, against the Hermite-style approach of Fuhrländer and Schöps (a BOBYQA modification enforcing all available derivative conditions at every sampled point, solved as regression). Derivative availability was simulated by randomly drawing subsets $\partial^\alpha f$9 of variables with $0 \in A$0, with second derivatives available only when both indices lie in $0 \in A$1. Performance was measured in normalized oracle accesses counting each $0 \in A$2 query as a distinct unit.

Three findings stand out:

1. Consistency: the Birkhoff method improves steadily as $0 \in A$3 grows from 0.25 to 0.75 and exhibits considerably less variability across availability regimes than the Hermite baseline.
2. Hermite sensitivity: the Hermite method performs strongly at 75% availability but degrades substantially at 25%, attributable to its requirement of bundling all derivative conditions with each point.
3. Robustness mechanism: selective incorporation of derivative conditions appears to be the decisive factor in robustness, not merely the presence of derivative information.

The authors candidly note a confound: their implementation always builds fully determined quadratic models requiring $0 \in A$4 data points, whereas the Hermite implementation reduces to BOBYQA (underdetermined models) when no derivatives are available. Purely derivative-free and fully derivative-based baselines were therefore excluded as uninformative comparisons, and the authors acknowledge that underdetermined Birkhoff models could improve their method further.

Limitations and open questions

Several limitations are stated plainly. The theoretical guarantees assume Lipschitz continuous first and second derivatives, which may not hold for all CUTEst test problems used. The available set $0 \in A$5 must be known before optimization begins, and the current analysis covers only fully determined quadratic models with linear-model extensions left as "straightforward analogues" rather than worked out. Differential evaluation costs among available derivatives are acknowledged but unused. Finally, the numerical comparison is confined to synthetic availability patterns on small-dimensional problems; whether the consistency advantage over Hermite-style bundling persists on large-scale simulation-driven problems with structured availability (e.g., adjoint-derived blocks) remains untested.

Conclusion

This paper extends interpolation-based trust-region derivative-free optimization from Lagrange to Birkhoff interpolation, providing poisedness definitions, conditioning equivalences, greedy model-completion and improvement algorithms, fully quadratic accuracy bounds with explicit constants, and global convergence guarantees—all while permitting arbitrary mixtures of function values and partial derivatives at interpolation nodes. The experimental evidence indicates that how partial derivative information enters the interpolation conditions materially affects robustness, with selective Birkhoff-style incorporation outperforming Hermite-style bundling when derivative availability is sparse.