Semi-Supervised Self-Normalizing Neural Network (SS-NN)

• The paper introduces SS-NN, a framework combining SELU self-normalization with staged semi-supervised learning to accurately model wavelength-dependent EDFA gain spectra using minimal labels.
• It employs an unsupervised denoising autoencoder pre-training step followed by supervised fine-tuning and one-shot transfer learning to predict gains across multiple EDFAs.
• Experimental results on 22 EDFAs demonstrate low mean absolute errors, outperforming benchmarks in both same-type and cross-type transfer scenarios.

Searching arXiv for the specified SS-NN and related SNN papers to ground the article in current records. The Semi-Supervised Self-Normalizing Neural Network (SS-NN) is a machine-learning framework for modeling the wavelength-dependent gain spectrum of Erbium-Doped Fiber Amplifiers (EDFAs) under minimal labeled-data conditions. In the EDFA literature, it denotes a fully connected feed-forward neural network that combines SELU-based self-normalization with staged semi-supervised learning: an unsupervised denoising autoencoder pre-training step on unlabeled telemetry followed by supervised fine-tuning on a small labeled set, and then one-shot transfer to a new amplifier using only one labeled measurement from the target device. The framework was introduced for EDFA gain-spectrum prediction across 22 EDFAs in the Open Ireland and COSMOS testbeds, including transfers between Booster and Pre-Amplifier types (Raj et al., 2023).

1. Conceptual definition and motivation

SS-NN was introduced to address a specific regime in optical networking: accurate modeling of EDFA wavelength-dependent gain when labeled gain spectra are expensive to collect in live networks, but unlabeled internal telemetry is easier to obtain. In this setting, the gain spectrum depends nonlinearly on channel loading, target gain setting, pump operation, inversion level, and operating mode, which makes purely physics-based fitting difficult to achieve with high accuracy in field conditions. The SS-NN framework therefore adopts a data-driven formulation that is explicitly designed for minimal-label operation and cross-device transfer (Raj et al., 21 Mar 2025).

Its defining combination has two components. The first is self-normalization through SELU activations, used in place of batch normalization. The stated motivation is that batch normalization can be unreliable with very small batch sizes or few observations, particularly below 32 observations, whereas SELU-based self-normalization aims to keep hidden activations close to zero mean and unit variance across layers. The second is semi-supervised staged training, in which denoising autoencoder pre-training on unlabeled inputs provides a weight initialization that captures structure in the input features, after which supervised fine-tuning is performed on a small labeled set. In the EDFA application, this staged procedure is used to support source-model training with 256 labeled samples and one-shot transfer to a new EDFA with only one labeled sample (Raj et al., 2023).

The framework is therefore not a generic semi-supervised neural network in the broad sense of consistency regularization, pseudo-labeling, adversarial alignment, or joint unsupervised-supervised optimization. The reported formulation is strictly staged: unsupervised pre-training first, supervised fine-tuning second, and transfer learning afterward. No additional semi-supervised loss terms, weighting schedules for combined objectives, or explicit domain-adaptation losses are reported in the original EDFA implementation (Raj et al., 2023).

2. Network architecture and self-normalizing mechanism

The reported SS-NN is a 5-layer fully connected network with hidden-layer sizes 200, 200, 100, and 100, followed by an output layer of 95 units that predicts the per-channel gain across 95 C-band channels. Hidden layers use SELU activations, and no batch normalization is used. No other normalization layers are reported. Dropout is not reported, and AlphaDropout, although commonly associated with SELU in the foundational SNN literature, is not mentioned in this implementation (Raj et al., 2023).

The hidden-layer nonlinearity is the scaled exponential linear unit,

$$\mathrm{selu}(x)=\lambda \begin{cases} x, & x>0 \ \alpha(e^x-1), & x\le 0 \end{cases}$$

with the original SNN paper giving $\alpha \approx 1.67326$ and $\lambda \approx 1.0507$, or equivalently $\alpha = 1.673$ and $\lambda = 1.050$ in the later EDFA presentation (Klambauer et al., 2017, Raj et al., 21 Mar 2025).

The self-normalizing principle derives from the SNN result that, under appropriate conditions, the layerwise propagation of activation mean and variance is governed by a mapping with a stable fixed point near $(0,1)$. In the notation used in the EDFA synthesis, if $z_l = W_l a_{l-1} + b_l$ and $a_l = \mathrm{selu}(z_l)$, then

$(m_l,v_l)=T(m_{l-1},v_{l-1}), \qquad (m^\*,v^\*)\approx (0,1).$

The foundational SNN paper formalizes this as a contraction toward zero mean and unit variance under SELU and suitable weight scaling, and proves that activation variances neither explode nor vanish in the specified domains (Klambauer et al., 2017).

Within the EDFA application, the practical role of this mechanism is narrower but crucial: because the transfer stage fine-tunes on extremely small labeled sets, including the one-sample case, the network avoids batch normalization and instead relies on SELU to stabilize hidden-layer statistics “implicitly.” The later EDFA paper further states that batch normalization performs poorly under limited data and that SELU-based self-normalization is key to stable training and generalization in the small-sample regime (Raj et al., 2023, Raj et al., 21 Mar 2025).

A notable implementation detail is that the 2023 EDFA paper reports neurons initialized with zero weights, while also noting that the original SNN theory recommends LeCun-normal-style initialization. This discrepancy is a property of the reported implementation rather than of SNN theory itself (Raj et al., 2023, Klambauer et al., 2017).

3. Inputs, outputs, and staged semi-supervised training

The output target is the wavelength-dependent gain spectrum across 95 ITU DWDM 50 GHz C-band channels, written in the 2025 formulation as

$$\overline{G}(\lambda_i)=[G(\lambda_1),\ldots,G(\lambda_{95})].$$

Noise figure is not modeled in this work (Raj et al., 2023, Raj et al., 21 Mar 2025).

The input feature set combines external spectrum measurements with internal amplifier telemetry. The reported inputs are the target gain setting, total EDFA input power and total output power, per-channel input power for 95 channels, a 95-element binary loading vector indicating whether each channel is present, and embedded VOA features: total VOA input power, total VOA output power, and VOA attenuation. The later paper states that pump power/current, ASE metrics, temperature, and explicit mode flags are not included as features beyond the fixed operating mode used during measurements (Raj et al., 2023, Raj et al., 21 Mar 2025).

Training is staged rather than joint. In the 2023 presentation, the unsupervised phase is described as a denoising autoencoder operating on the input layer, with 512 unlabeled measurements per target gain setting, input noise injection, MSE reconstruction loss, Adam with learning rate 0.001, and 1,800 epochs. After pre-training, encoder weights are fixed and used to initialize the source model, and the subsequent layers are trained in the supervised stage. The unsupervised objective is

$\alpha \approx 1.67326$0

The supervised phase then uses 256 labeled, randomly loaded gain-spectrum measurements, Adam with learning rate 0.001, MSE loss, and 1,200 epochs, with objective

$\alpha \approx 1.67326$1

No consistency loss, entropy minimization, MMD, adversarial alignment, or pseudo-labels are reported (Raj et al., 2023).

The 2025 paper presents the same framework in a more explicit layer-wise denoising formulation. It states that pre-training is greedy and layer-wise, with Gaussian noise injected into the inputs, 512 unlabeled input spectra per target gain setting, 1,800 epochs per layer, MSE loss, and learning rate $\alpha \approx 1.67326$2. Supervised fine-tuning uses a masked MSE that averages errors only over active channels,

$\alpha \approx 1.67326$3

with Adam, learning rate $\alpha \approx 1.67326$4, 1,200 epochs, and gradient clipping at 1.0. It also gives a conceptual overall formulation with optional $\alpha \approx 1.67326$5 regularization, while emphasizing that optimization remains sequential rather than joint (Raj et al., 21 Mar 2025).

4. One-shot transfer learning protocol

A central feature of SS-NN is one-shot transfer learning. In the reported EDFA setting, a base model is trained per amplifier type, specifically for Booster and Pre-Amplifier source models, and this source model is then transferred to a target EDFA using only one labeled measurement from the target device. The 2023 paper describes this as one-shot transfer using one labeled, randomly loaded measurement from the target EDFA, whereas the 2025 paper specifies a single fully loaded measurement per target gain setting (Raj et al., 2023, Raj et al., 21 Mar 2025).

The transfer model retains the same 5-layer architecture with SELU activations and pre-trained encoder weights. Fine-tuning is performed with Adam for 10,000 epochs and with differential learning rates across layers. The output layer uses learning rate $\alpha \approx 1.67326$6, and each preceding hidden layer uses a rate equal to 10% of the next layer’s learning rate, yielding a schedule described in the 2023 synthesis as approximately $\alpha \approx 1.67326$7 from input to output. No layers are frozen (Raj et al., 2023).

The stated rationale is that aggressive adaptation in the output layer captures target-specific characteristics quickly, while lower layers adapt slowly so as to avoid overfitting and to retain general features learned from the source EDFA. The 2025 paper formalizes the layer-specific transfer update as

$\alpha \approx 1.67326$8

where $\alpha \approx 1.67326$9 is the layer-specific learning rate and the output layer has the largest $\lambda \approx 1.0507$0 (Raj et al., 21 Mar 2025).

This transfer procedure is applied both to same-type transfers, such as Booster$\lambda \approx 1.0507$1Booster and PreAmp$\lambda \approx 1.0507$2PreAmp, and to cross-type transfers, such as Booster$\lambda \approx 1.0507$3PreAmp and PreAmp$\lambda \approx 1.0507$4Booster. The later paper emphasizes that all layers are fine-tuned and no layers are hard-frozen, which distinguishes the protocol from transfer-learning approaches that restrict adaptation to the final layer (Raj et al., 21 Mar 2025).

5. Dataset, experimental setting, and reported performance

The reported experiments span 22 EDFAs from commercial Lumentum ROADM-20 units. Open Ireland contributes 3 units, each with Booster and Pre-Amplifier, and COSMOS contributes 8 similar units, again each with Booster and Pre-Amplifier. The 2025 paper restates this as 22 EDFAs total, comprising 11 boosters and 11 pre-amps from 3 Lumentum ROADM-20 units in Open Ireland and 8 units in PAWR COSMOS, with pre-amplifiers operated in high-gain mode with 0 dB tilt (Raj et al., 2023, Raj et al., 21 Mar 2025).

Measurements are taken on a 95-channel, 50 GHz C-band grid. Channel loading modes include Random loading and Goalpost allocation. The reported scale is 3,168 gain measurements per EDFA for each target gain setting. Gain settings differ by testbed and amplifier type: Open Ireland uses 15/20/25 dB; COSMOS boosters use 15/18/21 dB; COSMOS pre-amplifiers use 15/18/21/24/27 dB in high-gain mode with 0 dB tilt (Raj et al., 2023).

For source-model evaluation, the 2023 paper reports 512 unlabeled measurements per gain setting for unsupervised pre-training, 256 labeled randomly loaded measurements for supervised fine-tuning, and testing on all Fixed Goalpost measurements, 270 samples, plus 20% of the Random Baseline, 220 samples. The 2025 paper additionally states that the main direct-training results use 256 labeled measurements total per EDFA, yielding 1,792 samples per EDFA when unlabeled pretraining data are included, and that updated robustness experiments increased the labeled set to 512 with similar or improved performance (Raj et al., 2023, Raj et al., 21 Mar 2025).

Evaluation uses mean absolute error in dB over the 95-channel gain spectrum,

$\lambda \approx 1.0507$5

The 2023 paper reports abstract-level overall transfer accuracy of MAE within 0.14 dB for same-type transfer and within 0.17 dB for cross-type transfer. Its detailed transfer discussion states that SS-NN outperforms a benchmark transfer-learning method denoted [Wang:23] on goalpost loading for same-type transfer and is comparable on random loading, while cross-type transfer shows significant improvement and consistency versus the benchmark, including a reported 3× reduction in MAE using the same external features. Adding internal VOA features further improves MAE, and for the random-loading subset with internal features, the heatmap shows MAE within 0.13 dB for same-type and within 0.11 dB for cross-type transfers across devices (Raj et al., 2023).

The 2025 paper adds direct-training performance figures: MAE of 0.07 dB for boosters and 0.08 dB for pre-amplifiers. In the detailed distributions, boosters show SS-NN MAE of approximately 0.07 dB for goalpost and 0.05 dB for random loading, with 95th percentile errors of 0.25/0.14 dB, compared with the benchmark’s 0.38/0.16 dB. For preamps, SS-NN gives approximately 0.08/0.05 dB using the same external features, and adding internal VOA features yields approximately 0.07/0.05 dB. For transfer learning with one fully loaded labeled measurement per target gain setting and internal features, the later paper reports per-EDFA MAE below 0.14 dB for same-type transfers and below 0.19 dB for cross-type transfers, while stating that error distributions are superior to the benchmark across both random and goalpost loading (Raj et al., 21 Mar 2025).

The same paper also notes an instrumentation bound: OCM resolution is 0.1 dB per channel, so errors near $\lambda \approx 1.0507$6 dB reflect quantization limits. This provides context for the direct-training error levels reported for random loading (Raj et al., 21 Mar 2025).

6. Relation to self-normalizing neural networks, limitations, and plausible extensions

SS-NN inherits its self-normalizing rationale from the broader SNN framework introduced by Klambauer et al. The foundational SNN paper proves that, with SELU activations and appropriate initialization, activation statistics are driven toward a stable fixed point near zero mean and unit variance, and that activation variance is bounded away from both explosion and collapse. It also recommends Gaussian weight initialization with variance $\lambda \approx 1.0507$7, zero biases, and AlphaDropout rather than standard dropout, while advising against batch, layer, or weight normalization in the SNN setting (Klambauer et al., 2017).

The EDFA-specific SS-NN uses only a subset of that broader prescription. It preserves the SELU/no-batch-normalization backbone and wide fully connected layers, but does not report AlphaDropout, does not report explicit input standardization beyond powers expressed in dB and Gaussian-noise corruption in denoising pretraining, and reports zero weight initialization in the 2023 implementation. This suggests that the term “self-normalizing” in the EDFA literature refers principally to reliance on SELU-induced activation stabilization under small-data fine-tuning, rather than to full adherence to the original SNN implementation recipe (Raj et al., 2023, Klambauer et al., 2017).

The empirical literature around SS-NN also contains several explicit limitations. The 2023 paper reports no head-to-head ablation against ReLU+BatchNorm or ReLU-only models, and no ablation on removing the unsupervised pre-training stage or varying the number of unlabeled samples. It does not model noise figure, does not impose physics-informed constraints, and does not report computational resources, training time, batch size, weight decay, early stopping, or statistical significance (Raj et al., 2023). The 2025 paper adds that transfer-learning tail errors are higher than in direct training, that severe distribution shift may degrade self-normalization statistics and increase errors, that sparse channel activations with unusual spectral shapes can reduce the effectiveness of the masked loss, and that cross-vendor and in-line amplifier transfer have not yet been studied (Raj et al., 21 Mar 2025).

A recurring empirical conclusion is the importance of internal VOA features. In direct training on the same EDFA, these features offer little advantage, but in transfer learning they substantially improve performance and robustness across devices and amplifier types, suggesting that they encode device-specific behavior beneficial for domain transfer. Likewise, the 2023 paper notes that transferring between testbeds may produce slightly higher errors because of configuration differences, and suggests that a small number of additional target measurements could further improve cross-testbed transfer, although no quantitative schedule is reported (Raj et al., 2023, Raj et al., 21 Mar 2025).

Future directions are stated cautiously. Reported possibilities include incorporating additional internal features such as pump telemetry, temperatures, and ASE estimates; studying transfer across vendors and in-line amplifiers; exploring multi-shot transfer strategies; and adding physics-informed constraints or hybrid modeling, for example smoothness across wavelengths or inversion-level priors. A plausible implication is that SS-NN occupies a middle ground between purely empirical black-box EDFA modeling and more structured hybrid models, with its present contribution centered on accuracy under extreme label scarcity rather than on explicit optical-device physics (Raj et al., 21 Mar 2025).