Deep Δ-Interpolator

• Deep Δ-Interpolator is a framework that leverages delta representations to interpolate between parameters, outputs, and feature manifolds across neural tasks.
• It refines analytic baselines in generative models, motion in-betweening, and medical registration by smoothly integrating learned corrections.
• The approach establishes theoretical bounds on network complexity and interpolation accuracy, guiding efficient model design under varying data separations.

Deep Δ-Interpolator encompasses a family of neural network architectures and theoretical regimes where deep networks perform interpolation tasks by leveraging delta (residual) representations, parameter interpolation, or data-driven manifold interpolation. The unifying idea is to either interpolate between endpoints in parameter or feature space, or to estimate outputs as corrections (deltas) to well-posed baselines. This framework extends to domains such as conditional generative modeling, motion in-betweening, medical registration, output activations, and learning-theoretic limits.

1. Parameter Interpolation for Scalar Conditioning

Deep Δ-Interpolator (often abbreviated as DPI) includes methods where scalar conditioning is incorporated into neural networks by interpolating between two full parameter sets, rather than by explicit feature concatenation or modulation. In this approach, a base network with parameters $\theta\in\mathbb{R}^p$ and input $x\in\mathbb{R}^m$ is reparameterized by learning two sets of parameters, $\Theta^0$ and $\Theta^1$. For a conditioning scalar $s\in[0,1]$, the operational parameter set is determined as $$\Theta(s) = (1-\lambda(s))\Theta^0 + \lambda(s)\Theta^1$$, where $\lambda(s)$ is either a fixed linear map ($\lambda(s)=s$) or a monotonic learnable function via cumulative softmax.

This construction preserves the architectural topology and augments networks to smoothly "morph" across the range of $s$, such as denoising steps in diffusion models or time in flow matching. Training jointly optimizes $\Theta^0$, $\Theta^1$, and the parameters of $\lambda(s)$ (if learnable) via standard MSE objectives (Park et al., 26 Nov 2025). Empirical results demonstrate consistent denoising MSE reductions (e.g., DRUNet: $$1.987\times 10^{-2} \rightarrow 1.865\times 10^{-2}$$), FID improvements (DRUNet diffusion: $26.2 \rightarrow 12.2$; ADM: $10.1 \rightarrow 8.3$), negligible added runtime cost (<1% extra FLOPs), and modest memory overhead.

The DPI methodology is particularly impactful in generative modeling regimes where conditional dependence is continuous and the vector field varies smoothly. It circumvents the need for architectural modifications such as embedding condition vectors into activations or explicit feature-wise modulation, offering an agnostic solution that generalizes across convolutional, normalization, attention, and linear layers.

2. Delta-Mode Residual Interpolation in Sequence and Motion Tasks

An alternative instantiation of Deep Δ-Interpolator operates by expressing the interpolation problem as learning residuals (deltas) relative to analytically tractable baselines. In the context of human motion in-betweening, the input features and outputs are both defined in a local $\Delta$-space, measured relative to reference frames or baseline interpolators such as SLERP (spherical linear interpolation for quaternions) or LERP (linear interpolation for positions) (Oreshkin et al., 2022).

• The network receives inputs $\Delta X = X - X_\mathrm{ref}$, using local reference frames to eliminate sensitivity to global translations/rotations.
• The output is modeled as a residual: $\Delta Y = f_\theta(\Delta X)$. The final interpolated output is $\hat Y = \tilde Y + \Delta Y$ where $\tilde Y$ is the baseline (e.g., SLERP trajectory).

This strategy yields robustness to global domain shifts and constrains the network to focus on physically meaningful correction terms. Ablations confirm that operation in local $\Delta$-space considerably reduces test error and boosts generalization compared to global-space processing; for example, on LaFAN1, L2P@30 frames: SSMCT (global) $1.10$ vs. $\Delta$-Interpolator (local) $1.00$. State-of-the-art performance is achieved across motion benchmarks, with minimal architectural complexity.

3. Deep Δ-Interpolation for Geometric and Biomechanical Regression

In medical image registration, Deep Δ-Interpolator frameworks refine dense deformation fields using learnable delta corrections to geometric interpolations. Here, synthetic training data from biomechanical brain simulations is used to supervise a residual 3D U-Net, which operates on the tuple $(I_\mathrm{pre},\,\phi_\mathrm{init})$, with $\phi_\mathrm{init}$ arising from classical interpolators (thin-plate spline, RBF, or linear) applied to sparse keypoints (Assis et al., 19 Aug 2025).

• The network predicts a correction $\epsilon_\theta$, yielding the total displacement $$\phi_\mathrm{pred} = \phi_\mathrm{init} + \epsilon_\theta$$.
• A regularization loss penalizes non-invertible deformations via a Jacobian-determinant constraint.

Quantitatively, the approach halves whole-brain displacement MSE relative to geometric baselines (TPS: $6.5\pm2.6$ mm²; TPS + $\Delta$ + Jacobian: $3.4\pm1.6$ mm²), while introducing negligible inference cost. The delta regime enables the model to efficiently learn local, biomechanically plausible corrections to analytically smooth, but physically naive, interpolants.

4. Theoretical Interpolation Power in Deep Networks

From a learning-theoretic perspective, "Deep Δ-Interpolator" also refers to regimes where a deep network is constructed to interpolate prescribed values at $N$ inputs with minimum parameter count. The sharp lower bound in this regime (for ReLU or piecewise-linear activations) states that when the data points are separated by $\delta = e^{-\Theta(N)}$, any network that exactly interpolates $N$ labels requires $\Omega(N)$ parameters; further, $O(N)$ always suffices (Siegel, 2023).

• For polynomially separated data ($\delta \geq N^{-k}$), interpolation is possible with $o(N)$ parameters.
• The classic bit-extraction approach to VC-dimension lower bounds is inapplicable for arbitrary $\delta$-separated sets, as it requires exponentially finer grid resolution than is available in the exponential separation regime.
• Sobolev-space approximation at the embedding endpoint aligns with these results: the best possible error is $\Omega(P^{-1})$ for $P$ parameters, with no accelerated "super-rate."

These results delimit the regime where interpolation is efficient and reveal the conditions under which parameter counts must scale linearly with the number of interpolated observations.

5. Manifold-Aware Δ-Interpolating Output Activations

Another realization of the Deep Δ-Interpolator concept involves direct interpolation over neural feature manifolds as a final network activation. Instead of classical linear or softmax layers, the output is generated by a weighted nonlocal Laplacian (WNLL) harmonic extension, minimizing Dirichlet energy on a similarity graph of learned feature representations (Wang et al., 2018):

• For each test input, prediction is performed by graph-based harmonic extension from a small labeled template over the manifold structure of network features.
• Training alternates between standard linear head and WNLL activation; in backpropagation, the gradient from the linear surrogate head efficiently proxies for the implicit WNLL system.

This mechanism is particularly effective in small-sample regimes and for architectures of increasing depth where standard softmax-based generalization degrades. On benchmarks such as CIFAR-10 and SVHN, the WNLL-activated deep nets deliver consistent reductions in test error, e.g., PreActResNet56: $7.30\%\rightarrow 5.16\%$ on CIFAR-10.

6. Comparative Summary and Impact

Deep Δ-Interpolator encompasses a spectrum of interpolation approaches unified by delta-based, parameter-interpolated, or manifold-respecting representations. The distinguishing characteristics include:

Method/Class	Core Mechanism	Application Domain
Parameter Interpolation (Park et al., 26 Nov 2025)	$$\Theta(s) = (1-\lambda(s))\Theta^0 + \lambda(s)\Theta^1$$	Scalar-conditioned generative models
Residual Δ-mode (Oreshkin et al., 2022)	Output as correction to analytic baseline	Motion sequence in-betweening
Geometric Δ-refinement (Assis et al., 19 Aug 2025)	$$\phi_\mathrm{pred} = \phi_\mathrm{init} + \epsilon_{\theta}$$	Medical keypoint registration
VC/interpolation bounds (Siegel, 2023)	Lower bound on parameters vs data spacing	Theoretical function approximation
Manifold interpolation (Wang et al., 2018)	Output via WNLL on learned features	Deep net classification, low-data regime

These approaches demonstrate that delta-based interpolation—at the level of parameters, outputs, or feature spaces—can lead to state-of-the-art accuracy, robustness to mismatches, and improved theoretical understanding of neural interpolation. Distinguishing itself from strictly end-to-end black-box regression, each Δ-Interpolator method leverages compositionality: combining rough analytic initializations with learnable refinement, or parameterizing smooth variation by construction, or respecting manifold geometry in label propagation.

The formalization and deployment of Deep Δ-Interpolator strategies have influenced architectural best practices in conditional modeling, biophysically informed regression, resource-efficient interpolation, and manifold-regularized learning.