Set-Based Transformer for Atmospheric Compensation in Standoff LWIR Hyperspectral Imaging

Abstract: Passive long-wave infrared (LWIR) hyperspectral imaging under a standoff geometry depends on atmospheric absorption and emission, as well as reflected radiance, thus making atmospheric compensation essential to get knowledge of a target of interest. Despite its importance, this compensation has been largely overlooked due to its practical and modeling difficulty. In this paper, we present a lightweight set-based deep learning framework that takes multiple radiance measurements, collected at different standoff ranges, as input and jointly estimates transmittance, atmospheric path radiance, and a shared downwelling spectrum. We analyze the learned representation with a sparse autoencoder and observe that several latent features do activate on geographically coherent subsets of the test data despite the absence of location supervision. Experiments on a MODTRAN generated standoff LWIR dataset demonstrate low spectral distortion across all estimated products. The dataset and code is publicly available at: https://factral.co/SAE-LWIR/

• The paper proposes a set-based transformer that jointly estimates transmittance, upwelling, and downwelling radiance from range-diverse LWIR hyperspectral measurements.
• It demonstrates that incorporating multiple standoff ranges significantly reduces spectral distortions as measured by metrics like Spectral Angle Mapper and normalized RMSE.
• The framework employs a sparse autoencoder to reveal emergent geographic clustering in latent features, offering insights into deep model interpretability in remote sensing.

Set-Based Transformer Framework for Atmospheric Compensation in Standoff LWIR Hyperspectral Imaging

Introduction

Atmospheric compensation is essential for accurate interpretation of standoff long-wave infrared (LWIR) hyperspectral measurements due to range-dependent absorption, emission, and reflection effects governed by the radiative transfer equation (RTE). LWIR sensors operate independently of solar illumination and are widely deployed in applications such as security, autonomous driving, wildfire detection, and remote sensing. However, existing literature largely focuses on airborne or satellite platforms in near-nadir geometries, failing to address the range-diverse, near-horizontal paths typical in standoff configurations. This paper introduces a lightweight set-based transformer architecture capable of jointly estimating atmospheric transmittance, upwelling path radiance, and a shared downwelling spectrum from multiple radiance measurements acquired at different standoff ranges.

Methodology

Set-Based Neural Architecture

The proposed framework processes a set of radiance measurements collected at varying standoff distances. Each input is a matrix $X \in \mathbb{R}^{N \times B}$, where each row corresponds to a spectrum over $B$ spectral bands at a specific range. The architecture utilizes a Set-Transformer encoder, composed of stacked Induced Set Attention Blocks (ISAB), to extract permutation-equivariant latent representations from unordered sets of measurements. The encoder outputs are routed to three task-specific decoders:

• Transmittance and Upwelling Decoders: Token-wise FFNs applied independently to each latent vector, enforcing permutation equivariance.
• Downwelling Decoder: Pooling by Multihead Attention (PMA), followed by an MLP, produces a permutation-invariant estimate of the shared downwelling radiance.

This arrangement ensures that the output predictions maintain the appropriate symmetry properties dictated by the physics of the measurements. The model is trained to minimize mean squared error across all three products.

Sparse Autoencoder for Latent Space Analysis

To probe the interpretability and physical alignment of learned representations, a sparse autoencoder (SAE) is employed post training. Each encoder token is reconstructed via a Top-K gated, overcomplete dictionary, facilitating analysis of activation patterns across the test set. Top-activating analysis reveals whether latent features encode meaningful atmospheric variations, potentially corresponding to geographic regions or physical atmospheric conditions.

Dataset Generation

A simulated standoff LWIR dataset is constructed using MODTRAN5, based on profiles from the clear-sky atmospheric profile database (CSP). The dataset comprises 36,547 atmospheric profiles, each evaluated at seven standoff ranges and seven target temperatures, totaling 255,829 radiance samples. Spectral bands and instrument response are modeled after the DARPA LWIR hyperspectral sensor. Strict filtering on cloud cover and humidity ensures inclusion of only clear-sky cases. This MODTRAN-generated dataset is made publicly available, marking the first open resource dedicated to atmospheric compensation in standoff LWIR scenarios.

Results

Test evaluations demonstrate significant gains in accuracy when transitioning from single-measurement ($N=1$) to multi-range set inputs ($N=7$), with notable reduction in spectral distortion (as measured by Spectral Angle Mapper and normalized RMSE) for all estimated atmospheric products. The transmittance estimation exhibits the lowest error rates, indicating robust constraint provided by range-diverse measurements.

Qualitative inspection of predicted spectra for transmittance, upwelling, and downwelling products shows strong agreement with ground truth, preserving fine-grained spectral structure across the LWIR window. SAE-based analysis of encoder activations reveals emergent clustering of high-activation samples corresponding to similar geographic locations, despite absence of explicit location supervision. This observation suggests the model leverages physical atmospheric invariants in its latent space organization.

Implications and Future Directions

Practically, the presented framework addresses an underexplored challenge in passive standoff LWIR sensing by enabling accurate, scalable, and computationally efficient atmospheric compensation from range-diverse measurements. The utilization of set-based transformers circumvents ill-posedness inherent in single-observation inference and opens possibilities for robust AC over extended operational environments.

Theoretically, the emergent organization of latent features offers insight into deep network interpretability for physical remote sensing tasks. The SAE methodology provides a promising avenue for analyzing and validating representations in terms of atmospheric state variables.

Future developments may include modeling angle-dependent downwelling, incorporating variable emissivity spectra, and investigating sensitivity to measurement configuration and standoff range selection. Alternative sparse autoencoder variants and expanded model capacities may further enhance interpretability and performance.

Conclusion

This study introduces and validates a set-based transformer approach for standoff LWIR atmospheric compensation, jointly estimating transmittance, upwelling, and downwelling radiance products from multiple range-diverse measurements. The approach demonstrates strong numerical performance, interpretable latent representations, and practical scalability, enabled by the first open standoff LWIR AC dataset. These results provide a foundation for further research in physically grounded neural models and robust atmospheric correction in remote hyperspectral sensing applications.