Insec-SPI: Batch-Invariant Spectral Intelligence

• Insec-SPI is a robust framework for insect species authentication via NIR spectroscopy, mitigating batch effects through adaptive preprocessing and adversarial invariance.
• It integrates a sparse attentive encoder and informed preprocessing initialized with Savitzky-Golay filtering to enhance cross-batch generalization.
• The framework offers explainability by combining integrated gradients and region-specific counterfactuals that correlate model predictions with known biochemical markers.

Searching arXiv for the specified paper and closely related terminology. Insec-SPI, short for Batch-Invariant Spectral Intelligence, denotes a framework for robust and explainable insect species authentication from near-infrared spectroscopy under realistic industrial conditions. In the formulation reported in "Batch-Invariant Spectral Intelligence for Robust and Explainable Insect Authentication" (Babor et al., 25 Jun 2026), Insec-SPI is realised by the Batch-Invariant Spectral Network (BISN), an end-to-end framework that combines a learnable preprocessing module, initialised with Savitzky-Golay filtering, with an entropy-regularised adversarial objective to suppress batch-specific spectral variation. The framework is developed for industrial insect processing involving Acheta domesticus, Hermetia illucens, and Tenebrio molitor, where reliable species authentication is required to control allergen exposure, prevent adulteration, and meet regulatory standards (Babor et al., 25 Jun 2026).

1. Problem formulation and industrial motivation

In industrial insect processing, rapid species authentication by NIR spectroscopy is appealing, but batch effects induce spectral shifts that degrade off-batch generalisation. The reported sources of batch effects include instrument drift, moisture, blanching, plasma-activated water, and ultrasound. A static preprocessing pipeline such as Savitzky-Golay, SNV, or MSC, chosen on source batches, often fails on new production runs (Babor et al., 25 Jun 2026).

The learning problem is posed on reflectance spectra $\mathbf{x}\in\mathbb{R}^d$, with $d=136$ wavelengths from 700–2050 nm. Each sample has a species label $y\in\{1,2,3\}$ and a production-batch label $b\in\{1,2,3\}$. The target is a learned mapping

$\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$

such that the transformed spectrum remains species-informative while being almost independent of batch:

$$P(\hat{\mathbf{x}}\mid y)\approx P(\hat{\mathbf{x}}\mid y,b).$$

This setup makes explicit that the central obstacle is not only multiclass classification but cross-batch robustness. A common misconception is that preprocessing is merely a fixed front-end choice; the Insec-SPI formulation instead treats preprocessing itself as an adaptive component of the invariance objective (Babor et al., 25 Jun 2026).

2. BISN architecture as the operational core of Insec-SPI

BISN jointly optimises four components: an informed preprocessing module, a sparse attentive encoder, a species classifier, and an entropy-regularised batch branch. Its defining distinction from Domain-Adversarial Neural Networks is that BISN suppresses batch-effects before species-specific features are learned, whereas DANN enforces domain adaptation only after feature extraction (Babor et al., 25 Jun 2026).

The informed preprocessing module uses a 1D convolution plus instance-normalisation to suppress low-level noise and baseline artifacts. The convolution kernel has width $w$ and is initialised to the analytic first-derivative Savitzky-Golay coefficients $\{a_j^{(1)}\}_{j=1}^w$, with transformed signal

$$\tilde{\mathbf{x}}_j = (W * \mathbf{x})_j = \sum_{k=1}^w W_k\,x_{j+k-\frac{w+1}{2}}.$$

The parameters $w$, the Savitzky-Golay polynomial degree $d=136$0, and derivative order $d=136$1 are selected by inner-fold cross-validation on source batches to minimize batch-separation, measured by logistic accuracy and MMD. During training, $d=136$2 is updated so that $d=136$3 learns to remove batch-specific structure while preserving species-discriminative features (Babor et al., 25 Jun 2026).

The sparse attentive encoder begins from $d=136$4. A feature transformer $d=136$5 produces a seed vector $d=136$6, which is projected back to wavelength space and processed by Ghost-Batch-Norm and sparsemax to form an attention mask

$d=136$7

Because sparsemax produces exact zeros, entries with $d=136$8 drop uninformative bands. The masked spectrum

$d=136$9

is passed to a second transformer $y\in\{1,2,3\}$0, which shares its first two GLU layers with $y\in\{1,2,3\}$1 and yields an 8-dimensional nonnegative embedding $y\in\{1,2,3\}$2 (Babor et al., 25 Jun 2026).

To prevent attention collapse onto a single wavelength, BISN includes the sparsity regulariser

$y\in\{1,2,3\}$3

This makes the encoder sparse but not degenerate. A plausible implication is that Insec-SPI treats spectral parsimony as an interpretability constraint rather than only as a dimensionality-reduction device.

3. Adversarial invariance objective and optimisation

Species prediction is performed by a single linear layer plus softmax on $y\in\{1,2,3\}$4,

$y\in\{1,2,3\}$5

with cross-entropy species loss

$y\in\{1,2,3\}$6

Batch invariance is enforced upstream. The preprocessed spectrum $y\in\{1,2,3\}$7 is sent through a Gradient-Reversal Layer to a small MLP with 128 hidden units and 2 outputs per leave-one-batch-out split, producing batch-probabilities $y\in\{1,2,3\}$8. Instead of training the discriminator to be accurate, the method maximises its prediction entropy, equivalently minimising the negative-entropy

$y\in\{1,2,3\}$9

Because the GRL flips the gradient, $b\in\{1,2,3\}$0 is updated to remove batch cues from $b\in\{1,2,3\}$1 (Babor et al., 25 Jun 2026).

The total loss in epoch $b\in\{1,2,3\}$2 is

$b\in\{1,2,3\}$3

where $b\in\{1,2,3\}$4 weights the mask-sparsity term and

$b\in\{1,2,3\}$5

ramps the adversarial weight from 0 to 1. The stated purpose of this schedule is to avoid early adversarial collapse. This optimisation design is central to Insec-SPI: the invariance pressure is applied at the spectral-preprocessing stage rather than only at latent feature level (Babor et al., 25 Jun 2026).

4. Training protocol and quantitative performance

The reported dataset comprises 2,700 spectra, corresponding to $b\in\{1,2,3\}$6 species $b\in\{1,2,3\}$7 $b\in\{1,2,3\}$8 batches $b\in\{1,2,3\}$9 $\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$0 treatments $\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$1 $\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$2 ultrasound conditions $\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$3 $\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$4 replicates. Evaluation uses a leave-one-batch-out protocol. For each held-out batch, the remaining two batches, comprising 1,800 spectra, are split 80/20 for training and internal validation, while the held-out batch of 900 spectra is used as the test set (Babor et al., 25 Jun 2026).

Hyperparameters, including Savitzky-Golay filter settings, encoder dimensions $\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$5, $\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$6, learning rate, and batch size, are tuned by five-fold stratified cross-validation on the two source batches, optimising species balanced accuracy. Preprocessing for all baselines, including Savitzky-Golay window, MSC/SNV, and detrend, is likewise optimised only on the source batches. The reported headline metrics are accuracy and macro-F1 averaged over the three LOBO folds (Babor et al., 25 Jun 2026).

The principal quantitative results are as follows.

Model or statistic	Reported value
Best classical/baseline after preprocessing: DANN accuracy	$\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$7
BISN accuracy on raw spectra with learnable preprocessing	$\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$8
BISN F1	$\hat{\mathbf{x}} = g(\mathbf{x};\theta_x)$9
Bootstrap improvement over strongest baseline	$$P(\hat{\mathbf{x}}\mid y)\approx P(\hat{\mathbf{x}}\mid y,b).$$0
95% confidence interval	$$P(\hat{\mathbf{x}}\mid y)\approx P(\hat{\mathbf{x}}\mid y,b).$$1
Bootstrap significance	$$P(\hat{\mathbf{x}}\mid y)\approx P(\hat{\mathbf{x}}\mid y,b).$$2

The bootstrap analysis uses $$P(\hat{\mathbf{x}}\mid y)\approx P(\hat{\mathbf{x}}\mid y,b).$$3 draws on the combined 2,700 test samples. The paper summarises the result as BISN outperforming the strongest baseline by four percent under LOBO evaluation (Babor et al., 25 Jun 2026). Within the stated experimental design, this supports the claim that upstream adversarial preprocessing yields stronger OOD robustness than post-hoc domain alignment.

5. Explainability, attribution, and biochemical grounding

Insec-SPI incorporates explainable AI at two levels: attribution and region-constrained counterfactual analysis. For each test sample, the work computes Integrated Gradients

$$P(\hat{\mathbf{x}}\mid y)\approx P(\hat{\mathbf{x}}\mid y,b).$$4

Attributions are aggregated into 11 interpretable spectral regions. The top median IG scores are reported for Lipid 2 (1190–1260 nm), Protein 1 (1000–1180 nm), Water 1 (910–1000 nm), Pigment (700–840 nm), and Mixed-Organics 1 (1260–1400 nm), with median values 0.59, 0.55, 0.44, 0.39, and 0.37 respectively (Babor et al., 25 Jun 2026).

These regions are connected to known C–H and N–H overtone bands that biochemically distinguish species, with lipid and protein contents reported as approximately 13–43% and 48–75%. The paper further states that model decisions consistently rely on the lipid and protein absorption regions across all folds, thereby connecting predictive performance to known insect biochemistry (Babor et al., 25 Jun 2026).

The counterfactual analysis targets misclassified spectra. For each case, the minimal $$P(\hat{\mathbf{x}}\mid y)\approx P(\hat{\mathbf{x}}\mid y,b).$$5 perturbation is optimised within a single region while keeping all other wavelengths fixed, with the objective of flipping the BISN prediction to the true class. The reported flip success rates and perturbation magnitudes indicate that Mixed-Organics 1, Protein 1, and Water 1 are both highly attributed and causally sufficient to correct errors, whereas Lipid 1 (840–910 nm) is highly attributed but often insufficient when batch treatments have perturbed that band (Babor et al., 25 Jun 2026).

This combination of saliency and causal sufficiency is important for interpretation. Attribution alone can identify correlates of prediction, whereas the region-constrained counterfactuals test whether a region can by itself correct a misclassification under the model. The paper therefore presents explainability not only as localisation of relevant wavelengths but as a consistency check between prediction, intervention, and biochemical plausibility.

6. Significance, scope, and prospective extensions

The framework is presented as addressing both cross-batch robustness and biochemical interpretability for automated insect species authentication under realistic industrial conditions. As summarised in the paper, Insec-SPI provides automated, real-time NIR-based authentication robust to batch-to-batch and process-induced variation; joint saliency maps that certify which bands drive each decision; and region-specific counterfactuals that validate causal sufficiency (Babor et al., 25 Jun 2026).

A recurring misconception in spectral classification is that robustness and explainability are orthogonal objectives. The Insec-SPI formulation argues against that separation by coupling batch-invariant learning with sparse attention and post hoc analyses that are explicitly tied to lipid and protein absorption regions. Another potential misconception is that stronger invariance necessarily requires suppressing discriminative signal; the reported architecture instead aims to suppress batch-specific structure while preserving species-discriminative features.

The framework also serves as a basis for extensions explicitly identified in the source: semi-supervised domain adaptation, new species, and online monitoring. These are stated as future extensions rather than demonstrated capabilities. A plausible implication is that the main contribution of Insec-SPI lies not only in its reported LOBO accuracy of $$P(\hat{\mathbf{x}}\mid y)\approx P(\hat{\mathbf{x}}\mid y,b).$$6, but in establishing a template for industrial spectral authentication in which learnable preprocessing, entropy-maximising adversarial objectives, sparse attention, and rigorous explainability analyses are jointly optimised (Babor et al., 25 Jun 2026).