Error Level Noise Embedding

• Error Level Noise (ELN) embedding is a technique that quantifies instance-specific noise as continuous features to enhance model robustness.
• It is applied across domains such as dimensionality reduction, PET image denoising, and speech recognition, improving metrics like PSNR, SSIM, and WER.
• ELN integration employs methods like FiLM modulation and prefix tuning to condition neural architectures on noise, making uncertainty an actionable signal.

Error Level Noise (ELN) embedding refers to a family of techniques for explicitly quantifying, modeling, and leveraging local or instance-specific noise characteristics as continuous features or representations within machine learning systems. ELN embeddings facilitate noise-aware processing and robustness, where noise is not simply treated as a nuisance but is systematically incorporated in the model or downstream pipeline. The methodology has been applied in dimensionality reduction, image denoising, and sequence modeling, and exhibits distinct instantiations in each domain depending on the structure and source of the noise (Shao, 2022, Li et al., 2022, Rahmani et al., 19 Dec 2025).

1. Formal Definition and General Principles

In the context of ELN embeddings, the "error level" is a scalar or vectorial quantification of the noise affecting a data instance—this could be a vector, image patch, or sequence output. The ELN is computed either from explicit statistical models (e.g., Poisson statistics in PET imaging) or from empirical disagreement (e.g., among ASR hypotheses). The chief principle is to produce an embedding (real-valued, readily incorporated into neural architectures) that characterizes the degree or pattern of uncertainty/noise, which can then be fused or injected into the processing pipeline alongside conventional feature representations.

2. ELN Embedding in Dimensionality Reduction

The "Johnson–Lindenstrauss embeddings for noisy vectors" framework (Shao, 2022) demonstrates that for high-dimensional vectors $x \in \mathbb{R}^n$ observed as $y = x + \eta$ with $\eta \sim N(0, \sigma^2 I_n)$, additive Gaussian noise fundamentally alters the landscape for norm-preserving linear embeddings. Classical sparse Johnson–Lindenstrauss transforms exhibit a dependence on the signal's $\ell_\infty/\ell_2$ ratio; however, the presence of Gaussian noise drives this ratio to $O(\sqrt{\log n / n})$, effectively "uniformizing" the observed data. Sparse embeddings such as random subsampling or hashing then preserve Euclidean norms up to multiplicative $1 \pm \varepsilon$ distortion with the same optimal sample complexity $m = O(\varepsilon^{-2} \log(1/\delta))$ as dense Gaussian projections, independent of signal structure. Here, the indispensable property is that the error/noise "helps" rather than hinders embedding quality in high dimension. The ELN is thus an implicit byproduct of the observation model, regularizing the statistical geometry of the data.

Embedding Method	Target Dimension $m$	Dependence on Noise
Dense Gaussian projection	$O(\varepsilon^{-2} \log(1/\delta))$	None
Subsampling (with ELN)	$O(\varepsilon^{-2} \log(1/\delta))$	Exploits uniformization
CountSketch hashing (with ELN)	$O(\varepsilon^{-2} \log(1/\delta))$	Exploits uniformization

In this setting, the essential property is that ELN transforms (as realized through noise-corrupted observations) grant access to highly efficient, sparse dimensionality reduction mechanisms without requiring explicit calculation of a noise embedding per se.

3. Local ELN Embedding for Imaging: PET Denoising

The framework proposed in "A Noise-level-aware Framework for PET Image Denoising" (Li et al., 2022) presents a canonical instantiation of explicit ELN embedding in medical imaging. Here, ELN is defined per image patch as the coefficient of variation (COV) of the local Poisson distribution of PET photon counts:

$$\epsilon := \mathrm{COV}_{\Omega} = \frac{1}{\sqrt{m_{\Omega}}}$$

where $m_{\Omega}$ is the average photon count over a 3D patch $\Omega$.

This patchwise scalar ELN, reflecting relative local noise, is injected into every channel-attention block of a deep convolutional neural network (DCNN) via a two-layer embedding sub-network. For each block:

• Input: scalar $\epsilon$
• Project via two fully connected layers to produce channel-wise scale and shift parameters ($\gamma$, $\beta$)
• Apply as FiLM modulation on the channel-pooled features

This procedure conditions the denoiser's response on spatially varying noise, enabling adaptive feature processing. Incorporation of the ELN embedding yields statistically significant improvements in PSNR and SSIM compared to non-conditioned baselines.

4. ELN Embedding in Sequence Modeling and ASR

"Incorporating Error Level Noise Embedding for Improving LLM-Assisted Robustness in Persian Speech Recognition" (Rahmani et al., 19 Dec 2025) establishes a methodology for extracting and leveraging ELN in autoregressive sequence correction for noisy ASR hypotheses.

• For each noisy audio input, an ASR model (Whisper-large-fa-v1) produces $k$-best transcript hypotheses $H = \{h^{(1)}, ..., h^{(k)}\}$.
• Token-level ELN: For each position $i$, compute the mean pairwise disagreement between tokens across hypotheses (indicator or embedding distance), yielding $e_i$.
• Sentence-level ELN: Compute mean pairwise distance between full hypotheses (Levenshtein or embedding-based).
• Concatenate token-level and sentence-level scores: $$\mathbf{v}_{\mathrm{ELN}} = [E_s; e_1, ..., e_{L_{\max}}]$$.

These ELN features are then mapped to vectors matching the LLM's (LLM) hidden dimension via linear projection, and injected as:

• Prefix-tuning vectors at each transformer layer
• Additive or concatenated modification of token embeddings

Through prefix-tuning and LoRA adapters, the base LLaMA-2-7B model is conditioned on ELN without modifying core weights. ELN-integration achieves marked reductions in word error rate (WER), particularly under noise, outperforming both unconditioned and fine-tuned (text-only) baselines.

Method	Mixed Noise WER (%)	SNR=5dB WER (%)
Raw Whisper	31.10	42.70
Fine-tuned (no ELN)	30.79	39.76
Fine-tuned + ELN	24.84	32.34

Ablation studies indicate sentence- and token-level ELN provide complementary improvements.

5. Architectural and Fusion Strategies

In ELN embedding frameworks, architectural strategies for utilizing noise-level features are domain-specific but often exhibit several unifying patterns:

• PET denoising networks inject scalar ELN via FiLM-style modulation in attention blocks.
• LLM-based ASR correction injects ELN both as prefix key/value attention vectors (prefix tuning) and into token embeddings.
• In both architectures, ELN acts as a context or condition for adaptive processing, modulating network response according to local or instance-level uncertainty.

ELN embedding may be constructed as a scalar, vector, or concatenated (sentence- and token-level) feature, and is typically mapped into the model's hidden dimensionality via learnable linear projection before fusion.

6. Empirical Impact and Statistical Evaluation

Across applications, ELN embeddings produce consistent improvements in empirical measures of performance:

• In PET denoising, PSNR and SSIM gain ($\Delta$PSNR $\in [1.04, 2.26]$ dB for 1/8→full task) over strong baselines is statistically significant ($p < 0.01$), and paired t-tests confirm gains are robust (Li et al., 2022).
• In ASR, ELN-embedding models reduce WER by several percentage points relative to text-only fine-tuning and far outperform generic LLM baselines (Rahmani et al., 19 Dec 2025).
• In high-dimensional embedding, sparsity and efficiency are achieved with no penalty in distortion, highlighting ELN’s regularizing effect (Shao, 2022).

7. Design Considerations and Applications

Design of ELN embeddings involves choosing appropriate noise quantification (statistical, empirical, or disagreement-based), embedding dimensionality, injection location and modality (additive, concatenative, prefix). The approach generalizes to settings where noise is instance-specific and heterogeneously distributed, including but not limited to:

• Imaging modalities with variable count statistics (PET, SPECT)
• Sequence modeling under ambiguous/noisy generation (ASR, MT)
• Dimensionality reduction and efficient sketching in high dimension

In all cases, ELN embeddings turn inherent noise into an explicit, actionable signal for model conditioning, conferring marked gains in robustness and adaptation across statistical and deep learning pipelines.