Degeneration-aware Adaptive Sensor Fusion

• The paper surveys degeneration-aware fusion methods that detect sensor degradation using observability metrics, uncertainty estimation, and deep classifiers.
• It details adaptive strategies such as dynamic reweighting, attention gating, and selective measurement injection to maintain robust state estimation.
• Empirical findings demonstrate significant gains in pose accuracy and object detection reliability across adverse conditions like poor geometry, weather, and sensor failures.

Degeneration-aware Adaptive Sensor Fusion addresses performance degradation in multi-sensor state estimation by explicitly detecting and adapting to loss of geometric observability or information in one or more sensing modalities. Such degeneration arises in geometry-poor environments, adverse weather, sensor occlusion, failure, or dynamic scenes. Modern approaches combine statistical metrics, discriminative learning, uncertainty estimation, and control-theoretic adaptation to enable robust SLAM, odometry, object detection, and data fusion across a spectrum of application domains. This article surveys the principal methodologies and empirical results associated with degeneration-aware adaptive sensor fusion, emphasizing technical mechanisms, architectural designs, and algorithmic frameworks as reported in recent arXiv literature.

1. Problem Formulation and Principles of Degeneration-awareness

Degeneration in sensor fusion refers to the loss (partial or total) of geometric constraint or information in one or more sensor streams, such that fusing all measurements indiscriminately provides limited additional information or even deteriorates state estimation. Classic examples include:

• Planar/linear scenes in LiDAR SLAM, resulting in rank-deficient scan-matching Jacobians (Li et al., 17 Feb 2025, Liao et al., 2024, Nissov et al., 2024, Han et al., 2023).
• Severe weather, low lighting, or occlusion degrading visual sensing (Lai et al., 28 Feb 2025, Paek et al., 10 Mar 2025, Zhang et al., 5 Sep 2025).
• Hardware failure leading to sensor drop-out (Paek et al., 10 Mar 2025).

Degeneration-aware adaptive fusion encompasses two tightly linked components:

1. Degeneracy Detection: Quantification (either by analytic observability, neural classifiers, uncertainty heuristics, or entropy metrics) whether and where a sensor’s contribution is weak or untrustworthy.
2. Adaptive Fusion: Dynamic reweighting, selection, correction, or gating of sensor measurements or feature representations such that the fused estimate relies during degeneration only on the best-informed sensors or axes, and falls back on auxiliary information where needed.

The central goal is to maximize reliability and accuracy, minimizing drift or error spikes despite environmental or hardware adversity.

2. Core Algorithms for Degeneracy Detection

Several approaches to degeneracy detection have emerged, often tailored to sensor type and fusion architecture:

2.1 Analytic Observability/Condition Number Methods

• Hessian Analysis: For LiDAR-based estimation, the block components of the Gauss–Newton Hessian are decomposed and their eigenvalues (or ratios thereof) used as “degeneracy factors.” For example, $S_R = \frac{\lambda_{r1}}{\lambda_{r3}}$, $S_t = \frac{\lambda_{t1}}{\lambda_{t3}}$, where $\lambda_{r*}$, $\lambda_{t*}$ are eigenvalues of rotation/translation Hessians (Liao et al., 2024, Xu et al., 2024). Large ratios indicate under-constrained axes.
• Covariance-based Methods: The inverse information matrix after linearizing point-plane/edge constraints is examined; large eigenvalues in the covariance matrix signal poor observability along specific state directions (Xu et al., 2024).

2.2 Feature Statistics and Residuals

• Feature Count Thresholding: Drop in geometric (e.g., edge/plane) feature counts or an increase in residuals signals degeneration; e.g., $N_t^{\text{feat}} < \tau_N$, $\epsilon_t^{\text{align}} > \tau_\epsilon$ (Zhang et al., 11 Jul 2025, Qian et al., 24 Jul 2025).
• Visual Keypoint Statistics: Track the number of inlier features per frame; suspend fusion when below a threshold (Zhang et al., 11 Jul 2025).

2.3 Learned/Deep Classifiers

• ResNet/Transformer-based Models: For particle-filter SLAM, 2D histograms of particle distributions are classified as degenerate or non-degenerate by a ResNet+Transformer discriminator, outputting a degeneracy confidence $c$ used downstream (Li et al., 17 Feb 2025). Image-domain classifiers can outperform rule-based methods.

2.4 Clustering and Outlier Detection

• DBSCAN on Degeneracy Factors: Outlier clustering in short time-windows of degeneracy metrics (e.g., via DBSCAN over [time, factor] pairs) is used to trigger robust, threshold-free detection (Liao et al., 2024).

2.5 Attention or Entropy-based Implicit Estimation

• Latent Attention Gating: Neural pipelines estimate degradation implicitly via normalized attention scores; degraded frames/channels are softly or discretely masked during fusion (Lai et al., 28 Feb 2025).
• Entropy Reduction: In particle-filter multitarget trackers, the expected entropy reduction (information gain) per sensor candidate is computed and used as a policy for sensor selection, thereby adapting to the most informative available modality (Starodubov et al., 9 Mar 2026).

3. Adaptive Fusion and Mitigation Strategies

Upon detection of degeneracy, the adaptive response can take multiple forms.

3.1 Selective Measurement Injection

• Selective Kalman Filter (SKF): Only inject measurements of auxiliary modalities (e.g., vision) along the degenerate axes as identified from the principal eigenvectors of the covariance; remaining axes are left to propagate by the high-confidence sensor (e.g., LiDAR-inertial) (Xu et al., 2024).
• Modular EKF/iEKF Pipelines: Secondary odometry or pose increments are triggered and injected into the state update only in degenerate frames, e.g. via joint state–residual stacking in iEKF (DAMS-LIO) (Han et al., 2023).

3.2 Dynamic Weighting or Gating

• Condition-number-weighted Blending: Fusion weights between odometry/IMU and LiDAR are set in inverse proportion to the degeneracy factor; as reliability on one source drops, trust shifts smoothly to the backup (Liao et al., 2024).
• Attention Masking: Neural attention (via Gumbel-Softmax or channel masks) gates out entire frames or individual feature channels from degraded sensors (Lai et al., 28 Feb 2025).
• Availability-aware Cross-attention: Unified canonical embedding spaces and cross-modal attention (as in ASF) learn to shift the focus to available, discriminative sensors, maintaining performance and robustness as any combination of modalities degrades or fails (Paek et al., 10 Mar 2025).

3.3 Enhanced Resampling and Diversity

• Enriched Particle Resampling: Resampling steps in PF-SLAM are modified to drive diversity and perturbation when degeneracy is detected, such as pairing nearest neighbors and introducing Gaussian noise inversely weighted by particle confidence (Li et al., 17 Feb 2025).

3.4 Sensor Switching or Subsystem Fallback

• Subsystem-level Switching: Frameworks such as Ground-Fusion++ maintain multiple tightly/loosely coupled subsystems (e.g., LIO and VIO) and switch to the best-available state estimate according to degeneracy flags, blending poses via robust SE(3) alignment (Zhang et al., 11 Jul 2025).
• Sensor-/Modality-wise Scheduling: Entropy/gain-based policies in adaptive particle filters select the single best or a subset of sensors for fusion in each cycle, yielding resilience and efficiency (Starodubov et al., 9 Mar 2026).

3.5 Partial-information Factors

• Degenerate-axis Projection: Graph-based fusion schemes reduce or remove information content (i.e., set the information matrix to project to the non-degenerate axis) for partially degenerate features, ensuring only reliable constraints enter the estimator (Nissov et al., 2024).

3.6 Multi-domain and Local-global Feature Remapping

• Dual-domain Adaptive Filtering: In vision/image fusion, modality-specific (e.g., DCT, Retinex) decompositions allied to spatial transformer or cross-attention modules filter, enhance, and fuse features, with prompt guidance or cross-modal interaction to bypass degraded portions (Zhang et al., 15 Apr 2025, Zhang et al., 5 Sep 2025).

4. Experimental Methodologies and Empirical Outcomes

Quantitative and qualitative demonstrations span diverse modalities, robotic platforms, and challenge scenarios.

4.1 Metrics and Evaluation

• Absolute and Relative Pose Error: Trajectory-level RMSE, ATE, RPE are commonly reported in both simulated and real-world datasets (Lai et al., 28 Feb 2025, Zhang et al., 11 Jul 2025, Liao et al., 2024).
• Object Detection AP: In multi-modal object detection, detection AP BEV and AP 3D are used to compare under sensor degradation/failure and across weather challenges (Paek et al., 10 Mar 2025).
• Degeneracy Detection Accuracy: F1 and accuracy evaluated against held-out/test splits or against baseline degeneracy detection algorithms (Li et al., 17 Feb 2025).
• Uncertainty and Fusion Credibility: Credible intervals, ellipsoid width, and innovation statistics for online adaptation of Kalman filter noise models (Truong-Ba et al., 2 Jun 2025, Kolar et al., 2020).

4.2 Representative Empirical Findings

• Large Reductions in Drift/ATE: In geometry-poor or weather-degraded environments, adaptive fusion improves ATE/APE by up to 84% relative to non-adaptive baselines (Li et al., 17 Feb 2025, Liao et al., 2024).
• Attention/Gating Outperforms Hard Thresholds: Neural attention methods (ResNet+Transformer/TempNet/Noisy-Or) robustly handle both seen and unseen degradations, outperforming classical fusion and exceeding strong baselines by ≥ 11–28% mean IoU in semantic segmentation (Tian et al., 2019, Lai et al., 28 Feb 2025).
• Availability-aware Fusion Is Near-Optimal: ASF yields state-of-the-art detection accuracy in all sensor availability/failure combinations with negligible computational penalty, demonstrating up to 20.1% AP_3D gains under sensor loss (Paek et al., 10 Mar 2025).
• Selective Fusion Is Efficient: SKF and DAMS-LIO achieve lower update times and maintain or exceed SLAM accuracy, thanks to not processing visual features or auxiliary measurements in normal (non-degenerate) conditions (Xu et al., 2024, Han et al., 2023).

A summary table of representative empirical results:

Paper	Application	Core Degeneration Mode	Adaptive Fusion Method	Relative Gain (Key Metric)
(Li et al., 17 Feb 2025)	PF-SLAM (LiDAR)	Feature-less geometry	ResNet+Transformer + resampling fusion	ATE ↓ up to 84% vs GMapping
(Zhang et al., 11 Jul 2025)	Ground robot SLAM	LiDAR, vision, odom	Subsystem switching + pose blending	RMSE ↓ 70–85% (corridor)
(Liao et al., 2024)	LiDAR-IMU SLAM	Structural degeneracy	DBSCAN-based compensation (IMU blend)	RMSE ↓ up to 80% at event
(Lai et al., 28 Feb 2025)	Multi-modal odometry	Weather/visual/LiDAR	Temporal+spatial attention gating	t_rel ↓ 1.36% (KITTI)
(Paek et al., 10 Mar 2025)	AD object detection	Weather/failure	UCP + per-patch cross-attn (ASF)	AP_3D ↑ +20% vs L4DR
(Xu et al., 2024)	SLAM	Subspace underconstraint	SKF selective update via eigen-analysis	10× VIO speedup, best RMSE

5. Applications and Domain-specific Instantiations

Degeneration-aware adaptive sensor fusion has been applied across modalities and robotics/automation domains:

• SLAM and Odometry: Humanoid, ground, aerial, and maritime platforms; all-weather, underground, long-corridor, tunnel, and GPS-denied navigation (Qian et al., 24 Jul 2025, Liao et al., 2024, Han et al., 2023, Truong-Ba et al., 2 Jun 2025, Starodubov et al., 9 Mar 2026).
• Multi-sensor Object Detection: Automotive 3D object detection in adverse conditions, leveraging LiDAR, 4D radar, and camera, including full “smooth fallback” to available sensors (Paek et al., 10 Mar 2025).
• Medical, Infrastructure, and Non-robotic Sensing: Adaptive Kalman fusion for track geometry with noisy on-board railway sensors (Truong-Ba et al., 2 Jun 2025), multiplicatively-corrected time-series via unsupervised monotonic learning (Kolar et al., 2020).
• Image Fusion and Enhancement: Robust IR–visible image fusion via explicit degradation filtering and deep adaptive local-global cross-attention (Zhang et al., 15 Apr 2025, Zhang et al., 5 Sep 2025).

6. Limitations, Challenges, and Future Directions

Despite substantial progress, real-world deployments highlight critical challenges and open problems:

• Dynamic/Multi-actor Scenes: Many schemes are tuned to static scenes; online learning or policy adaptation is needed for dynamic environments (Li et al., 17 Feb 2025).
• Open-world Uncertainty: Unanticipated (out-of-distribution) degradations are not always detected by analytic metrics; ensembles and uncertainty-aware fusions (e.g., TempNet+Noisy-Or) offer better reliability but require more computation (Tian et al., 2019).
• Hyperparameter Sensitivity: Threshold-dependent methods require environment- and sensor-specific tuning, while deep approaches depend on the training distribution (Liao et al., 2024, Nissov et al., 2024).
• Computation/Mobility Tradeoffs: High-dimensional neural or graph-based fusion is resource-intensive; emergent research focuses on lightweight attention or discrete gating (Lai et al., 28 Feb 2025, Paek et al., 10 Mar 2025).
• Partial Observability: While partial-information factor graphs and SKF architectures efficiently use available information, full observability under all degenerate scenarios may require auxiliary sensors or collaborative, multi-agent fusion (Pritzl et al., 23 Oct 2025).

Potential research avenues include integrative learning policies for adaptive sensor trust, unsupervised or adversarial domain adaptation, generalized sensor-degeneracy representations (beyond geometric or photometric), and full SLAM-level joint optimization resilient to cascade failures.

7. Cross-Cutting Insights and Theoretical Foundations

Degeneration-aware adaptive fusion builds on several theoretical pillars:

• Observability and Fisher Information: Analytic structure-grounding of degeneracy, especially via CRLB and information matrix eigen/singular value analysis (Han et al., 2023).
• Uncertainty Quantification: Use of predictive entropy, epistemic/aleatoric uncertainty, and deviation-ratio to gauge reliability and guide fusion weights (Tian et al., 2019, Lai et al., 28 Feb 2025).
• Deep Representation Learning: Discriminative and generative models (ResNet, Transformer, TempNet, CLIP-guided) learn to abstract and adapt to complex, cross-modal degradation patterns (Li et al., 17 Feb 2025, Zhang et al., 5 Sep 2025, Paek et al., 10 Mar 2025).
• Policy-driven Modal Switching: Sensor selection is formalized as an information-gain optimization, reducing resource use and maximizing estimation confidence (Starodubov et al., 9 Mar 2026).

These strategies collectively enable resilient, real-time estimation across sensing modalities and degradation regimes, merging statistical rigor, deep learning, and real-world operational constraints.