FeatJND: Deep Feature Just Noticeable Difference

• FeatJND is a framework that defines the maximum feature perturbation tolerable before degrading downstream task performance.
• It employs chance-constrained optimization and differentiable surrogate losses to drive effective quantization, compression, and predictive quality assessment.
• Experimental results demonstrate substantial gains in accuracy and rate-distortion improvements, highlighting its effectiveness over traditional methods.

FeatJND models the concept of Just Noticeable Difference (JND) within deep visual feature spaces, providing a principled approach to determining the maximum magnitude of feature perturbation that preserves downstream task performance. This concept arises from the need to characterize, control, and optimize the perceptual or task-aligned quality of features generated by machine vision models, particularly in the context of compression, quantization, and rate-distortion optimization for both human and machine perception. The FeatJND paradigm is distinct from classical pixel-based JND, as it is formulated directly in feature space and aligned with the invariances and sensitivities of downstream tasks (Zhao et al., 29 Jan 2026, &&&1&&&).

1. Mathematical Formulation and Theoretical Basis

FeatJND defines the tolerance boundary for features as the maximum per-feature perturbation δ that induces an imperceptible or negligible drop in downstream task performance $P_t(h(f))$, where $f\in\mathbb{R}^{C\times H\times W}$ is the feature tensor at a particular model split point, and $h(\cdot)$ is the fixed (pre-trained) downstream head.

The ideal chance-constrained optimization is given by: $$\max_{\delta}\;\mathcal{M}(\delta), \quad \text{s.t.}\quad \Pr\left[P_t\bigl(h(f)\bigr) - P_t\bigl(h(f+\delta)\bigr) \le \varepsilon\right] \ge 1 - \rho$$ where $\mathcal{M}(\delta) = \|\delta\|_2$, $\varepsilon$ is the tolerable performance drop, and $\rho$ a small violation probability. This is typically relaxed and turned into a differentiable form via hinge losses or soft constraints: $$\mathcal{L} = \lambda_t D_t(f+G_{\theta}(f), f) - \mathcal{M}(G_{\theta}(f))$$ $D_t$ denotes a differentiable surrogate for the performance difference, and $G_{\theta}$ is a learnable estimator of the JND perturbation in feature space (Zhao et al., 29 Jan 2026).

FeatJND can be further specialized for perceptual encoding by enforcing that the reconstructed feature maps of generative models (e.g., autoencoders for image compression) remain within the feature-space “tube” defined by JND threshold images, ensuring that the reconstruction, under a reference perceptual network (e.g., VGG), matches the JND-limited input (Pakdaman et al., 2024).

2. Estimator Architecture and Training Protocols

The FeatJND estimation network $G_{\theta}$ typically has the following structure:

• Input: feature tensor $f$ at a standardized split (e.g., backbone output for classification, FPN outputs for detection/segmentation)
• Architecture: a sequence of $3\times 3$ convolutions with BatchNorm and ReLU, followed by several $1\times 1$ “residual” blocks.
• Output: perturbation map $\delta = G_{\theta}(f)$, same shape as $f$.
• Training: downstream model $h(\cdot)$ is frozen; only $G_{\theta}$ is optimized using Adam. For stability, outputs are clamped and gradients clipped.

Loss functions $D_t$ are task-dependent:

• Classification: softened KL divergence between logits of clean and perturbed input.
• Detection/Segmentation: sums of classification/regression consistency losses (KL, smooth-$\ell_1$, and mask MSE).

In learned image compression, FeatJND appears as the Feature-Wise JND Loss (FWL). During training, reconstructed images and JND-thresholded reference images are both encoded via a fixed perceptual network; the loss penalizes feature-wise distances, enforcing that outputs remain in the human-imperceptible region (Pakdaman et al., 2024).

3. Integration with Prediction and Rate-Distortion Frameworks

In video quality assessment, the FeatJND approach is integrated as a feature-driven predictor for the Satisfied User Ratio (SUR) curve and JND points (Wang et al., 2018):

• Input features comprise quality-drop histograms (using segment-wise VMAF scores) and masking-effect summaries (spatial/temporal statistics).
• SVR with RBF kernel regresses input features to estimated SUR curves.
• For image and feature compression, FeatJND is incorporated into the rate-distortion objective: $$L_{\mathrm{total}} = R(\hat{y}) + \lambda [w \, d(x_0, \hat{x}) + (1-w)\, d(F(x_j), F(\hat{x}))]$$ where $R(\hat{y})$ is the estimated bit rate, $d$ is MSE or feature-MSE, and $F$ denotes perceptual feature extraction (Pakdaman et al., 2024).

Quantization can be guided by FeatJND tolerance maps, allocating per-token (spatial) quantization step sizes proportional to their individual tolerance, optimizing bit allocation under a fixed noise budget (Zhao et al., 29 Jan 2026).

4. Experimental Validation and Empirical Findings

Under matched feature distortion (NRMSE), FeatJND perturbations consistently yield higher downstream accuracy than IID Gaussian noise:

• Classification: At NRMSE ≈ 0.4, FeatJND preserves 10%+ higher Top-1 accuracy versus Gaussian baseline (e.g., Swin-T: 75% vs. 65%).
• Detection/Segmentation: Substantial gains in mAP (~10–17 points at corresponding NRMSE), with better preservation of semantic structure.
• Dynamic quantization: FeatJND-guided step size assignment delivers significant performance improvements over uniform or randomly permuted schemes under equal noise budgets; for instance, Swin-T Top-1 rises from ~60% (uniform) to 77.5% (FeatJND-based) at fixed $\sigma$ (Zhao et al., 29 Jan 2026).

In neural compression, incorporating feature-wise JND loss yields:

• Rate-distortion improvements: 4–10% BD-rate savings over the baseline (Pakdaman et al., 2024).
• JND-level bitrate savings: 9–11% lower bpp at the subjective visibility threshold.
• Visual quality: Reduced edge artifacts and sharper detail at equal bitrates.

In video, FeatJND enables accurate prediction of SUR curves and JND points, achieving mean absolute SUR errors <0.05 and QP-level errors ≈1.2–1.6 across resolutions and JND stages (Wang et al., 2018).

5. Visualization, Attribution, and Interpretability

FeatJND perturbation maps, when visualized, show larger tolerances in background regions and smaller tolerances on semantically relevant objects, reflecting the alignment with task-critical content (Zhao et al., 29 Jan 2026). Attribution analysis (e.g., integrated gradients) confirms that applying FeatJND preserves or even sharpens attribution in key regions while suppressing contributions from non-critical background areas.

For neural codecs, qualitative inspection demonstrates that FeatJND-based training reduces visually distracting artifacts (such as edge ringing), ensuring that image differences remain below human perceptual thresholds (Pakdaman et al., 2024).

6. Applications and Practical Significance

FeatJND is applicable to:

• Deep feature compression: Enables adaptive quantization strategies and rate control, maximizing compression under task-aligned tolerance maps (Zhao et al., 29 Jan 2026).
• Learned image and video codecs: Provides objective, perceptually informed loss functions, raising coding efficiency without runtime overhead (Pakdaman et al., 2024).
• Predictive quality assessment: Supports rapid bitrate ladder selection for streaming and encoding workflows by accurately predicting user satisfaction thresholds from a compact set of features (Wang et al., 2018).

The unified rationale is that FeatJND formalizes a task- and perception-aligned region of imperceptibility in feature or pixel space, yielding gains in efficiency, robustness, and subjective or downstream task quality.

7. Distinctions from Classical JND and Future Directions

FeatJND generalizes JND concepts from the perceptual pixel domain to flexible, task-calibrated feature spaces, addressing the increased prominence of deep representations as transmission, storage, and control interfaces in modern vision systems (Zhao et al., 29 Jan 2026, Pakdaman et al., 2024). By aligning tolerance boundaries with downstream performance, FeatJND avoids overconstraining non-critical features, allowing more aggressive compression or quantization without semantic loss.

Ongoing research directions include extending FeatJND to multimodal representations, further automating the calibration of tolerances under shifting resource constraints, and integrating FeatJND-guided control into end-to-end trainable video and feature pipelines.