InterpoLated Learning (InterpoLL) Overview

• InterpoLated Learning (InterpoLL) is a framework that integrates interpolation as a structural learning mechanism across latent representation, weight consolidation, and optimization processes.
• It mitigates shortcut learning in NLP by blending majority and minority feature representations and enhances continual learning via interpolation of previous and current model weights.
• InterpoLL also informs adaptive optimization and instance-based learning, enabling methods like ALI-G and interpolated nearest neighbors to balance performance with model stability.

Searching arXiv for the cited papers to ground the article. Searching arXiv for "Continual Learning with Weight Interpolation" and related InterpoLL uses. InterpoLated Learning (InterpoLL) denotes a set of interpolation-centered learning formulations rather than a single universally fixed algorithm. In recent arXiv usage, the label has been applied to at least three prominent settings: representation-space interpolation for mitigating shortcut learning in NLP, weight interpolation for rehearsal-based continual learning, and broader interpolation-driven designs in optimization, nonparametric prediction, instance-based learning, and neural-network architecture (Korakakis et al., 7 Jul 2025, Kozal et al., 2024, Berrada et al., 2019, Xing et al., 2018, Lian, 2021, Park et al., 2024). A related but distinct acronym is INTERPOL, an identification framework for de-anonymizing model outputs in pairwise voting settings via interpolated preference learning; despite the similar name, it addresses a different problem class (Cho et al., 16 Mar 2026).

1. Nomenclature and conceptual scope

The common element across these formulations is that interpolation is treated as an active mechanism of learning rather than as a purely numerical post-processing device. Depending on the paper, the interpolation target may be latent representations, model weights, stochastic step-sizes derived from an interpolation regime, neighbor weights in nonparametric estimation, or trainable interpolation nodes embedded directly in a neural architecture (Korakakis et al., 7 Jul 2025, Kozal et al., 2024, Berrada et al., 2019, Xing et al., 2018, Park et al., 2024).

Formulation	Interpolation locus	Representative paper
Shortcut mitigation InterpoLL	Latent representations of majority and minority examples	(Korakakis et al., 7 Jul 2025)
CLeWI / InterpoLL in continual learning	Old and new model weights after permutation alignment	(Kozal et al., 2024)
ALI-G and interpolation property	Optimization regime where per-sample losses are driven to zero	(Berrada et al., 2019)
Interpolated-NN	Neighbor weights that diverge at zero distance	(Xing et al., 2018)
ADB interpolation	Coordinate-wise approximation-degree and sum-times-difference synthesis	(Lian, 2021)
INN	Trainable interpolation points and basis-function evaluations	(Park et al., 2024)

This multiplicity of usage is itself an important feature of the term. It suggests that InterpoLL functions as a broader paradigm in which interpolation is elevated to a structural prior for learning, but the operational meaning of interpolation depends on the application domain.

A recurrent misconception is that “interpolation” necessarily denotes naive averaging or uncontrolled overfitting. The papers associated with InterpoLL argue for more specialized constructions: intra-class latent interpolation with $\lambda \sim \mathrm{Uniform}(0,0.5)$ for shortcut mitigation, permutation-aligned weight interpolation to exploit mode connectivity in continual learning, and clipped stochastic Polyak-style updates when the interpolation property holds in deep optimization (Korakakis et al., 7 Jul 2025, Kozal et al., 2024, Berrada et al., 2019).

2. Representation-space InterpoLL for shortcut mitigation

In "Mitigating Shortcut Learning with InterpoLated Learning" (Korakakis et al., 7 Jul 2025), InterpoLL is introduced as a model-agnostic representation-space augmentation designed to weaken spurious correlations exploited under ERM. The setup decomposes the predictor into an encoder and classifier,

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$

with latent representation $z_i=f_{\mathrm{enc}}(x_i)$. Within each class $y$, the method distinguishes majority examples $g_{\mathrm{maj}}(y)$, whose spurious features align with $y$, from minority examples $g_{\mathrm{min}}(y)$, whose features contradict the shortcut. The split is inferred by an under-parameterized auxiliary model $f_\phi$: examples mis-predicted by $f_\phi$ are treated as minority, and correctly predicted examples as majority (Korakakis et al., 7 Jul 2025).

For each majority example $(x_i,y_i)\in g_{\mathrm{maj}}$, a same-class minority example $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$0 with $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$1 is sampled, and their representations are interpolated:

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$2

Minority examples use their normal representation. The resulting objective is

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$3

where $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$4. All model parameters are updated by back-propagating this loss (Korakakis et al., 7 Jul 2025).

Algorithmically, the method first uses $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$5 to partition the data into $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$6 and $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$7, then processes each minibatch by replacing the latent representation of each majority example with an interpolated representation and leaving minority examples unchanged. The forward pass and loss are then computed on the transformed latent batch (Korakakis et al., 7 Jul 2025).

The empirical scope is broad. The paper reports results on natural language inference tasks such as MNLI, FEVER, and QQP with OOD test sets and stress tests including HANS and PAWS; text-classification tasks including FDCL18, CivilComments-WILDS, and Amazon-WILDS; and domain generalization via GLUE-X. Architectures span encoder-only models such as BERT-base, BERT-large, RoBERTa-large, ELECTRA-large, and XLNet-large, encoder-decoder models such as T5-large and T5-3B, and decoder-only models such as GPT2-medium and GPT2-large (Korakakis et al., 7 Jul 2025).

A concise summary of reported benchmark deltas is as follows.

Setting	ERM	InterpoLL
NLI OOD / Stress	$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$8	$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$9
FDCL18 minority accuracy	$z_i=f_{\mathrm{enc}}(x_i)$0	$z_i=f_{\mathrm{enc}}(x_i)$1
CivilComments minority accuracy	$z_i=f_{\mathrm{enc}}(x_i)$2	$z_i=f_{\mathrm{enc}}(x_i)$3
GLUE-X average	$z_i=f_{\mathrm{enc}}(x_i)$4	$z_i=f_{\mathrm{enc}}(x_i)$5
BERT-large on NLI $z_i=f_{\mathrm{enc}}(x_i)$6 HANS	$z_i=f_{\mathrm{enc}}(x_i)$7	$z_i=f_{\mathrm{enc}}(x_i)$8

The paper also reports that Uniform$z_i=f_{\mathrm{enc}}(x_i)$9 is the best interpolation ratio, whereas Uniform$y$0 and $y$1 with $y$2 hurt either ID or OOD performance. Inter-class mixing and inverting interpolation underperform intra-class mixing. Minimum description length probing indicates that InterpoLL yields the lowest extractability of known shortcuts. With $y$3 synthetic label flips, the method degrades much less than ERM and other methods. On MNLI, runtime is reported as $y$4 h for ERM and $y$5 h for InterpoLL, while most other debiasing methods take at least $y$6 h and up to $y$7 h (Korakakis et al., 7 Jul 2025).

The paper is explicit about limitations. It notes a small drop in ID accuracy on some tasks, recommends interpolation in upper encoder layers, and warns that mixing may introduce unnatural representations if minority examples are too dissimilar. It also reports that a no-auxiliary variant, in which the learner self-identifies $y$8 and $y$9, still outperforms prior methods (Korakakis et al., 7 Jul 2025).

3. Weight interpolation in continual learning

In "Continual Learning with Weight Interpolation" (Kozal et al., 2024), the relevant InterpoLL formulation is called CLeWI. It is a post-task weight consolidation method for continual learning with rehearsal. After learning task $g_{\mathrm{maj}}(y)$0, let $g_{\mathrm{maj}}(y)$1 denote the saved network weights from task $g_{\mathrm{maj}}(y)$2, and let $g_{\mathrm{maj}}(y)$3 denote the weights after fine-tuning on task $g_{\mathrm{maj}}(y)$4. A permutation $g_{\mathrm{maj}}(y)$5 is first computed via the REPAIR procedure to align $g_{\mathrm{maj}}(y)$6 to $g_{\mathrm{maj}}(y)$7 in weight space, and the interpolated weights are then formed as

$g_{\mathrm{maj}}(y)$8

with $g_{\mathrm{maj}}(y)$9. In the formulation given in the paper, $y$0 controls plasticity and $y$1 controls stability (Kozal et al., 2024).

The role of $y$2 is directly quantified. On split-CIFAR10, the reported ablation gives:

• $y$3: average accuracy $y$4, last-task accuracy $y$5, forgetting $y$6;
• $y$7: $y$8, $y$9, $g_{\mathrm{min}}(y)$0;
• $g_{\mathrm{min}}(y)$1: $g_{\mathrm{min}}(y)$2, $g_{\mathrm{min}}(y)$3, $g_{\mathrm{min}}(y)$4;
• $g_{\mathrm{min}}(y)$5: $g_{\mathrm{min}}(y)$6, $g_{\mathrm{min}}(y)$7, $g_{\mathrm{min}}(y)$8;
• $g_{\mathrm{min}}(y)$9: $f_\phi$0, $f_\phi$1, $f_\phi$2 (Kozal et al., 2024).

This ablation confirms the stability–plasticity trade-off claimed by the method. A small $f_\phi$3 favors current-task adaptation but increases forgetting; a large $f_\phi$4 preserves earlier-task performance but restricts adaptation to new tasks. In practice, $f_\phi$5 is selected by lightweight grid search on a held-out stream or by inspecting interpolation plots of test accuracy versus $f_\phi$6 using the replay buffer (Kozal et al., 2024).

CLeWI is designed as a plug-in over rehearsal-based training. The pseudocode given in the paper applies standard replay-based SGD with a replay buffer $f_\phi$7, then, for tasks $f_\phi$8, computes a permutation $f_\phi$9, interpolates the weights, and recomputes BatchNorm statistics through update_batchnorm(θ,𝓜). The buffer is used both for feature-map correlation estimates in the permutation step and for re-estimating BatchNorm statistics to avoid variance collapse. calc_permutation solves a bipartite matching on feature-map correlations, with explicit reference to Jordan et al. 2022 (Kozal et al., 2024).

The theoretical intuition is framed via the cumulative test loss

$f_\phi$0

and the forgetting increment

$f_\phi$1

The paper argues that minimizing cumulative loss requires both low current-task loss and low forgetting, and that SGD solutions for consecutive tasks often lie in mode-connected, or near-convex, regions of the loss landscape once aligned by permutation. A linear interpolation in this aligned space is therefore expected to remain in a low-loss valley that balances old and new performance (Kozal et al., 2024).

Empirically, the method is evaluated on class-incremental splits of CIFAR-10 with $f_\phi$2 tasks, CIFAR-100 with $f_\phi$3 tasks, and Tiny-ImageNet with $f_\phi$4 tasks, using a replay buffer of $f_\phi$5 examples and ResNet-18 as the primary architecture, with WideResNet used in ablations (Kozal et al., 2024). The paper reports the following average accuracy and forgetting improvements:

Method pair	CIFAR-10	CIFAR-100	Tiny-ImageNet
ER $f_\phi$6 CLeWI+ER	$f_\phi$7, $f_\phi$8 FM	$f_\phi$9, $(x_i,y_i)\in g_{\mathrm{maj}}$0 FM	$(x_i,y_i)\in g_{\mathrm{maj}}$1, $(x_i,y_i)\in g_{\mathrm{maj}}$2 FM
MIR $(x_i,y_i)\in g_{\mathrm{maj}}$3 CLeWI+MIR	$(x_i,y_i)\in g_{\mathrm{maj}}$4, $(x_i,y_i)\in g_{\mathrm{maj}}$5 FM	$(x_i,y_i)\in g_{\mathrm{maj}}$6, $(x_i,y_i)\in g_{\mathrm{maj}}$7 FM	$(x_i,y_i)\in g_{\mathrm{maj}}$8, $(x_i,y_i)\in g_{\mathrm{maj}}$9 FM
DER++ $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$00 CLeWI+DER++	$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$01, $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$02 FM	$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$03, $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$04 FM	$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$05, $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$06 FM

The paper further states that the method can be plugged into ER, aGEM, ER-ACE, MIR, BIC, and DER++ with no further algorithmic changes, and that without any replay, pure interpolated SGD remains poor. That result is important because it circumscribes the method’s scope: the evidence supports CLeWI as a complement to rehearsal rather than as a replacement for it (Kozal et al., 2024).

4. Interpolation as an optimization regime: ALI-G

A different, earlier interpolation-centered usage appears in "Training Neural Networks for and by Interpolation" (Berrada et al., 2019). Here the central object is the interpolation property of supervised learning: for a parametric model $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$07, per-sample loss $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$08, and expected risk

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$09

the problem satisfies interpolation if there exists $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$10 such that

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$11

The paper then derives Adaptive Learning-rates for Interpolation with Gradients (ALI-G), a stochastic Polyak-style method that exploits this property (Berrada et al., 2019).

Starting from Polyak’s update

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$12

the interpolation setting with $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$13 motivates the stochastic raw stepsize

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$14

with a small $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$15 to avoid division by zero. ALI-G clips this value at a user-chosen maximum $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$16:

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$17

Clipping is presented as crucial in nonconvex settings; the paper gives the one-dimensional counterexample $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$18, which oscillates without clipping (Berrada et al., 2019).

The convergence guarantees are stratified by assumption class. For convex and $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$19-Lipschitz losses under interpolation, ALI-G attains an $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$20 rate. With additional $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$21-smoothness, the expected suboptimality improves to $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$22. In the strongly convex and smooth setting, the method achieves an exponential rate,

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$23

For certain nonconvex objectives satisfying the Restricted Secant Inequality,

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$24

the paper gives an exponential convergence statement when $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$25 and smoothness holds (Berrada et al., 2019).

The empirical evaluation covers a differentiable neural computer copy task, WRN-16-4 on SVHN, a Bi-LSTM on SNLI, WRN-40-4 and DenseNet-40-40 on CIFAR-10/100, and ResNet-18 on ImageNet without data augmentation (Berrada et al., 2019). Representative figures include:

• on SVHN after $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$26 epochs, ALI-G reaches $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$27 test accuracy, compared with $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$28 for hand-scheduled SGD;
• on SNLI, ALI-G reaches $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$29 for CE/SVM loss, matching SGD’s $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$30;
• on CIFAR-10 WRN, ALI-G achieves $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$31 versus SGD’s $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$32;
• on CIFAR-100 DenseNet, ALI-G achieves $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$33, equal to SGD’s $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$34;
• on ImageNet, with batch-size $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$35 over $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$36 GPUs and fixed $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$37, ALI-G reaches $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$38 top-5 train accuracy in $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$39 epochs (Berrada et al., 2019).

In this usage, InterpoLL does not denote interpolation between examples or weights. Instead, interpolation is a property of the data-fitting regime that enables a closed-form adaptive learning rate. This is a materially different interpretation of the term from the shortcut-mitigation and continual-learning formulations.

5. Nonparametric and instance-based interpolation

Two additional lines of work extend the InterpoLL idea into nonparametric estimation and instance-based learning. In "Statistical Optimality of Interpolated Nearest Neighbor Algorithms" (Xing et al., 2018), the predictor is an interpolated nearest-neighbor rule. For ordered Euclidean distances $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$40 from a query $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$41, neighborhood size $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$42, and a positive function $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$43 with $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$44, the estimate is

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$45

with weights

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$46

when $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$47, and $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$48 when the query coincides with a training point. The classification rule is $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$49 (Xing et al., 2018).

The paper proves that, under Hölder smoothness, support regularity, a compact-support density assumption, and a Tsybakov margin condition for classification, the method attains minimax-optimal rates in both regression and classification. For regression, choosing $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$50 yields

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$51

For classification, the excess-risk rate becomes

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$52

The analysis emphasizes the usual bias–variance decomposition: interpolated-NN reduces bias by heavily down-weighting distant neighbors but incurs a variance penalty through unequal weights. Empirically, the paper reports slightly lower excess risk than classical $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$53NN across simulated settings and consistently lower classification error on the HTRU2 pulsar dataset (Xing et al., 2018).

In "A New Interpolation Approach and Corresponding Instance-Based Learning" (Lian, 2021), the relevant formulation is ADB interpolation, based on the approximation-degree between numerical values. For a center $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$54 in an interval and approximation region $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$55, the approximation-degree is piecewise linear, equals $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$56 at $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$57, and vanishes on the region boundary. This is extended to a strict approximation region in $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$58 via an axis-aligned box (Lian, 2021).

The one-dimensional ADB interpolant reconstructs $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$59 locally from adjacent sample points and yields piecewise linear formulas such as

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$60

and

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$61

In higher dimensions, the paper introduces the sum-times-difference synthesis

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$62

with coordinate-wise partial interpolations $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$63 around a center value $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$64 (Lian, 2021).

The induced instance-based learner includes both regression and classification procedures. The ADB regressor first finds the closest training point in a Chebyshev sense, performs one-dimensional ADB interpolation along each coordinate axis, and returns the sum-times-difference estimate. The ADB classifier searches for a training point whose strict approximation region contains the query; if found, it returns that class label, otherwise it abstains or returns “unknown” (Lian, 2021).

The computational analysis states a nearest-neighbor lookup cost of $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$65 with a kd-tree or $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$66 with hash grids under assumptions, plus $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$67 for one-dimensional interpolation along each axis, for total cost $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$68. The paper reports empirical results including maximum absolute error on the order of $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$69 for $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$70 on $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$71, RMSE $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$72 versus approximately $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$73 for bilinear interpolation on $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$74, RMSE approximately $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$75 versus $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$76 for trilinear interpolation on $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$77, and $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$78 classification accuracy on test points inside strict regions for a synthetic two-class dataset (Lian, 2021).

These two works are conceptually related but methodologically distinct. Interpolated-NN is a statistical learning rule with explicit minimax analysis, whereas ADB is framed as an interpolation-first analogue-based learner with structured abstention in classification (Xing et al., 2018, Lian, 2021).

6. Interpolating neural networks and broader synthesis

A further extension of the InterpoLL idea appears in "Interpolating neural network: A novel unification of machine learning and interpolation theory" (Park et al., 2024). This paper defines InterpoLated Learning as the construction of end-to-end differentiable models by embedding classical interpolation theory into the architecture itself. Instead of dense unstructured weight matrices, the model introduces trainable interpolation points with coordinates $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$79 and associated values $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$80, and defines the forward map through interpolation basis functions (Park et al., 2024).

In one dimension, a segment network partitions the input interval into contiguous segments bounded by interpolation nodes, computes local interpolant basis functions $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$81, and outputs

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$82

In multiple dimensions, the paper avoids a full tensor of size $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$83 through tensor decomposition, writing

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$84

The corresponding forward pass for mode $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$85 uses

$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$86

The paper interprets this construction as a CP-style tensor factorization of the weight tensor (Park et al., 2024).

The claimed advantages are parameter efficiency, training speed, compact memory footprint, and interpretability of learned node values as values at physical interpolation points. Parameter growth is characterized as $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$87 for node values, with optional node coordinates and analytic-network weights typically negligible relative to that count (Park et al., 2024).

The main application is Laser Powder Bed Fusion heat conduction, with an $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$88-dimensional input consisting of $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$89 spatial coordinates, $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$90 temporal coordinate, and $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$91 uncertain parameters. The paper reports that with $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$92 modes and $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$93 nodes in each dimension, the model uses approximately $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$94 node values. Reported performance includes speed-up versus an FEM-plus-surrogate pipeline of approximately $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$95 faster inference for a $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$96 mm tool-path at $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$97m resolution, a model size of tens of megabytes rather than gigabytes, sub-$f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$98m temperature-field error over a $f_\theta(x)=f_{\mathrm{cls}}(f_{\mathrm{enc}}(x)),$99 mm scan length, calibration from only $z_i=f_{\mathrm{enc}}(x_i)$00 FEM-generated melt-pool profiles with $z_i=f_{\mathrm{enc}}(x_i)$01 error relative to experimental melt-pool temperatures, and real-time online control in which a comparable FFNN uses only $z_i=f_{\mathrm{enc}}(x_i)$02 of the parameters and trains $z_i=f_{\mathrm{enc}}(x_i)$03–$z_i=f_{\mathrm{enc}}(x_i)$04 faster to the same loss (Park et al., 2024).

The paper also states explicit limitations: selecting segment count $z_i=f_{\mathrm{enc}}(x_i)$05, modes $z_i=f_{\mathrm{enc}}(x_i)$06, and basis type still requires user guidance; mathematical proof of convergence and generalization bounds for InterpoLL remains open; and graph-structured data or non-rectangular domains may require specialized basis constructions (Park et al., 2024).

Taken together, these formulations suggest that InterpoLL is best understood as a research direction organized around where interpolation is inserted into the learning system. In some works it modifies latent representations to suppress shortcuts; in others it merges models across tasks, exploits the interpolation regime of overparameterized training, reweights nearest neighbors, encodes instance-based approximation rules, or replaces dense layers with interpolation-theoretic operators (Korakakis et al., 7 Jul 2025, Kozal et al., 2024, Berrada et al., 2019, Xing et al., 2018, Lian, 2021, Park et al., 2024). A separate but related direction, INTERPOL, uses interpolated preference data and a staged triplet-based curriculum to identify anonymous LLM outputs and quantify ranking-manipulation threats in Arena-style leaderboards, underscoring that interpolation has also become a design primitive in preference learning (Cho et al., 16 Mar 2026).