Kernel Exchange Algorithms (KEA)

• Kernel Exchange Algorithms (KEA) are methods that integrate a parameter‐free bootstrap approach in TDDFT with bidirectional center swaps in sparse surrogate modeling.
• In surrogate modeling, KEA refines an n-point interpolant by exchanging centers based on residual or power function criteria, reducing L∞ interpolation errors significantly.
• In TDDFT, KEA iteratively constructs the exchange-correlation kernel using a self-consistent bootstrap method, achieving accurate optical spectra with modest computational cost.

Kernel Exchange Algorithms (KEA) encompass two distinct algorithmic families developed independently for use in time-dependent density functional theory (TDDFT) and for sparse kernel-based interpolation in surrogate modeling. In TDDFT, KEA refers to the “bootstrap” parameter-free construction of the exchange-correlation kernel $f_{xc}$, vital for accurately capturing optical spectra. In high-dimensional kernel surrogate modeling, KEA designates local-search improvements that combine greedy insertion and removal, enabling fine-tuned, computationally efficient sparse representations without increasing model size. The union of these approaches highlights the broad applicability and underlying structural themes of kernel exchange methods in modern computational science.

1. KEA in Kernel Surrogate Modeling: Problem Statement and Algorithmic Structure

In the context of kernel-based surrogate modeling, the core problem is to construct an $n$-point interpolant,

$s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$

that minimizes the $L^\infty$ interpolation error on a compact domain $\Omega \subset \mathbb{R}^d$ given a large data set $X = \{x^{(1)}, \ldots, x^{(N)}\}$ with target values $Y = \{f(x^{(i)})\}_{i=1}^N$ and a strictly positive definite kernel $k: \Omega \times \Omega \to \mathbb{R}$. Selecting the optimal subset $X_n$ of size $n$ is a combinatorial task of size $n$0. Traditional approaches such as greedy "knot insertion" (e.g., $n$1–greedy or $n$2–greedy) and "knot removal" yield only locally optimal nested sets. KEA defines a post-processing phase in which centers are swapped: at each exchange step, one center is removed, and a new one added, in an attempt to further reduce the interpolation error, guided by surrogate error metrics such as residuals or power function increases (Wenzel et al., 2024).

2. Algorithmic Description and Computational Considerations

KEA is initialized from an $n$3-point greedy solution $n$4. Each of $n$5 exchange steps selects:

• a candidate for addition from $n$6 maximizing a prescribed criterion (e.g., residual or power function);
• a candidate for removal from $n$7 minimizing the degradation, for instance, as measured by leave-one-out residual or reduced power function.

The swap $n$8 is performed, and the interpolant is updated efficiently (via Newton basis updates and Rippa’s rule for leave-one-out error calculation). Each step costs $n$9, so $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$0 exchanges are computationally feasible when $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$1 and $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$2 (Wenzel et al., 2024). Early stopping can be triggered if the improvement plateaus.

3. Theoretical Properties: Convergence and Error Reduction

For many kernels, $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$3–greedy insertion satisfies

$s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$4

with $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$5 determined by kernel smoothness and the dimension, but the constant $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$6 for greedy selection incurs a suboptimal factor $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$7 compared to the optimal. KEA maintains the same rate $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$8 since $s_{n, X_n}(x) = \sum_{j=1}^n \alpha_j k(x, x_j),$9 is fixed, but empirically can reduce the prefactor $L^\infty$0 by correcting early suboptimal inclusions or removals. Theoretically rigorous analysis of global convergence and prefactor reduction for KEA is not yet available (Wenzel et al., 2024).

4. Empirical Performance in Interpolation and Surrogate Modeling

Numerical experiments on both low-dimensional and high-dimensional interpolation tasks demonstrate that KEA can substantially reduce the $L^\infty$1 interpolation error. For $L^\infty$2–greedy models in $L^\infty$3D, $L^\infty$4D, and $L^\infty$5D, followed by up to $L^\infty$6 exchange steps, observed improvements include:

• Best-case error reduction up to $L^\infty$7 (from $L^\infty$8);
• Average error reduction over $L^\infty$9 randomized trials of approximately $\Omega \subset \mathbb{R}^d$0 ($\Omega \subset \mathbb{R}^d$1) in low-dimensional settings;
• For higher dimensions, improvement ratios around $\Omega \subset \mathbb{R}^d$2 (i.e., $\Omega \subset \mathbb{R}^d$3 mean reduction).

Improvements are more pronounced for smoother (higher-order) Matérn kernels, with negligible effect for the roughest ($\Omega \subset \mathbb{R}^d$4), consistent with known theoretical arguments (Wenzel et al., 2024).

Domain	Best-case improvement	Average improvement ratio
2D, Franke-type	$\Omega \subset \mathbb{R}^d$5	$\Omega \subset \mathbb{R}^d$6
5D/6D, Gaussian-type	up to $\Omega \subset \mathbb{R}^d$7	$\Omega \subset \mathbb{R}^d$8

KEA allows direct budget control (fixed $\Omega \subset \mathbb{R}^d$9), can retroactively repair non-optimal greedy choices, and fits as a lightweight post-processing wrapper to insertion-removal pipelines.

5. Comparison with Pure Greedy Methods and Related Algorithms

KEA differs from classical greedy insertion or removal by decoupling the nested, single-direction step restriction and allowing bidirectional swaps. Whereas greedy methods create incrementally nested subsets, KEA can untangle suboptimal early choices by directly considering the effect of one-for-one exchanges. Unlike algorithms such as VKOGA (insertion) or ERBA (removal), KEA can wrap around any such method without requiring changes to the kernel or interpolant structure. Computationally, the final model remains of size $X = \{x^{(1)}, \ldots, x^{(N)}\}$0, so evaluation costs are unchanged. Limitations include unchanged asymptotic error rates and possible overfitting without early-stopping heuristics. The computational overhead is only practical for small $X = \{x^{(1)}, \ldots, x^{(N)}\}$1 relative to $X = \{x^{(1)}, \ldots, x^{(N)}\}$2 (Wenzel et al., 2024).

6. KEA in TDDFT: Bootstrap Kernel Construction for $X = \{x^{(1)}, \ldots, x^{(N)}\}$3

In time-dependent DFT, the "bootstrap" KEA provides a self-consistent, parameter-free approximation for the frequency-independent exchange-correlation kernel $X = \{x^{(1)}, \ldots, x^{(N)}\}$4, crucial for accurate optical response prediction. The approach iteratively closes the Dyson equation for density response $X = \{x^{(1)}, \ldots, x^{(N)}\}$5 with the ansatz

$X = \{x^{(1)}, \ldots, x^{(N)}\}$6

where $X = \{x^{(1)}, \ldots, x^{(N)}\}$7 is the Coulomb kernel, $X = \{x^{(1)}, \ldots, x^{(N)}\}$8 is the dielectric function built from the Kohn–Sham response, and $X = \{x^{(1)}, \ldots, x^{(N)}\}$9 is the inverse macroscopic dielectric. This kernel recovers the required $Y = \{f(x^{(i)})\}_{i=1}^N$0 asymptotics for excitonic binding and contains no empirical parameters. It is computed in an iterative loop:

1. Initialize $Y = \{f(x^{(i)})\}_{i=1}^N$1.
2. Iteratively solve the Dyson equation and update $Y = \{f(x^{(i)})\}_{i=1}^N$2 using the bootstrap closure until convergence.
3. Output the converged $Y = \{f(x^{(i)})\}_{i=1}^N$3 and $Y = \{f(x^{(i)})\}_{i=1}^N$4.

The computational cost is modest, primarily requiring small matrix inversions per $Y = \{f(x^{(i)})\}_{i=1}^N$5-point and frequency, and typically converges in fewer than 10 iterations (Sharma et al., 2011).

7. Applications, Strengths, and Limitations across Domains

In surrogate modeling, KEA routinely achieves $Y = \{f(x^{(i)})\}_{i=1}^N$6–$Y = \{f(x^{(i)})\}_{i=1}^N$7 mean error reduction (and up to $Y = \{f(x^{(i)})\}_{i=1}^N$8 best-case) at negligible additional final evaluation cost and with minimal conceptual overhead. The method is compatible with any kernel and selection criterion and is effective for a wide range of surrogate modeling tasks, particularly when $Y = \{f(x^{(i)})\}_{i=1}^N$9 (Wenzel et al., 2024).

In TDDFT, bootstrap KEA reproduces experimental optical spectra with high fidelity for both small-gap semiconductors (Ge, Si, GaAs, AlN, TiO$k: \Omega \times \Omega \to \mathbb{R}$0, SiC), large-gap/ionic/Frenkel excitonic systems (C, LiF, Ar, Ne), and magnetic insulators (NiO). Notable results include peak position errors within $k: \Omega \times \Omega \to \mathbb{R}$1 eV and intensity errors within $k: \Omega \times \Omega \to \mathbb{R}$2 of experiment, matching the performance of self-consistent GW+BSE at much lower computational overhead. It is parameter-free, has the correct excitonic long-range structure, and is broadly applicable. Limitations include lack of frequency-dependence in $k: \Omega \times \Omega \to \mathbb{R}$3, continued reliance on external gap corrections (scissor/GW/LDA+U), and untested extension to finite-$k: \Omega \times \Omega \to \mathbb{R}$4 or low-dimensional systems (Sharma et al., 2011).

KEA Domain	Strengths	Limitations
Surrogate Modeling	Error reduction (mean $k: \Omega \times \Omega \to \mathbb{R}$5), post-greedy repair, fixed model size	No asymptotic rate improvement, potential overfitting, computational overhead for large $k: \Omega \times \Omega \to \mathbb{R}$6
TDDFT Bootstrap	Parameter-free, universal, correct $k: \Omega \times \Omega \to \mathbb{R}$7 asymptotics, low cost	Frequency-independent, incomplete gap self-consistency, limited dimensional/tested scope

The KEA framework, in both computational physics and machine learning, thus provides a principled and adaptable methodology for improving kernel-based approximations, either by direct construction of exchange-correlation kernels in quantum response or by local search among sparse point selections in high-dimensional surrogate modeling.