CuMoLoS-MAE: Ensemble Masked Autoencoder

• CuMoLoS-MAE is a curriculum-guided Monte Carlo ensemble masked autoencoder designed to inpaint remote-sensing atmospheric fields from degraded Doppler-lidar data.
• It employs a ViT-based encoder with a curriculum over mask ratios and stochastic masking at inference to restore fine-scale features and provide pixel-level uncertainty.
• The method achieves superior spectral fidelity and faster convergence compared to traditional gap filling, preserving key atmospheric structures like updrafts, shear lines, and vortices.

CuMoLoS-MAE is a Curriculum-Guided Monte Carlo Stochastic Ensemble Masked Autoencoder for reconstructing remote-sensing atmospheric fields, introduced for Doppler-lidar vertical-velocity data that are frequently degraded by low-SNR gates, range folding, and spurious discontinuities (Naskar et al., 20 Aug 2025). The model is designed to restore fine-scale features such as updraft and downdraft cores, shear lines, and small vortices; to learn a data-driven prior over atmospheric fields; and to provide pixel-wise uncertainty estimates through Monte Carlo aggregation. In the formulation reported for the ARM Southern Great Plains Doppler-lidar setting, CuMoLoS-MAE combines a ViT-based masked autoencoder with a curriculum over mask ratio during training and repeated stochastic masking at inference to approximate a posterior predictive mean reconstruction and a finely resolved uncertainty map.

1. Problem setting and conceptual scope

The method addresses a recurring limitation in atmospheric remote sensing: accurate atmospheric profiles from instruments such as Doppler Lidar, Radar, and radiometers are often corrupted by missing or unreliable observations. The motivating claim is twofold. Traditional gap filling blurs fine-scale structures, while deep models commonly lack confidence estimates (Naskar et al., 20 Aug 2025). CuMoLoS-MAE is positioned precisely at that intersection: it seeks both high-fidelity inpainting and calibrated spatially resolved uncertainty.

In the reported implementation, the target variable is Doppler-lidar vertical velocity represented as $64\times 64$ time-height slices. These slices are treated as images, but their semantics are geophysical rather than photographic: coherent structures correspond to atmospheric motions, including updraft and downdraft cores, shear lines, and small vortices. The model therefore operates in a regime where preserving mesoscale and storm-scale structure is more important than local smoothing. This is the context for the paper’s emphasis on spectral fidelity and on the restoration of coherent anisotropic features rather than merely low pointwise error.

A central conceptual feature is that the model does not reconstruct from fully observed context alone. Instead, it is trained to inpaint from increasingly sparse observations. This forces the network to internalize non-local correlations in the vertical-velocity field. A plausible implication is that the latent representation functions as an atmospheric prior learned from the training set, rather than as a purely local denoiser.

2. Architecture and representation

The architecture begins from $64\times 64$ input images formed from time-height slices of Doppler-lidar vertical-velocity fields. Each image is split into $2\times 2$ micro-patches, producing

$(64/2)^2 = 32\times 32 = 1{,}024$

tokens. The use of $2\times 2$ micro-patches is integral to the reported fine-scale recovery, because it preserves relatively high spatial granularity while still enabling transformer-based token processing (Naskar et al., 20 Aug 2025).

The encoder is a standard Vision Transformer with 12 transformer-encoder layers. Positional embeddings are added to the visible, unmasked tokens. The decoder is a lightweight ViT-style decoder with 4 transformer layers that upsamples and merges token features back to full $64\times 64$ resolution. It processes the visible tokens together with a set of learnable mask tokens inserted at masked positions. In textual pipeline form, the architecture is:

$$[64\times 64\ \text{input}] \rightarrow \text{micro-patch tokenizer} \rightarrow \{\text{visible tokens, masked positions}\} \rightarrow +\text{positional encodings} \rightarrow 12\times \text{ViT encoder} \rightarrow 4\times \text{ViT decoder} \rightarrow 64\times 64\ \text{reconstruction}.$$

This design places CuMoLoS-MAE within the masked-autoencoding family, but with a remote-sensing-specific emphasis on geophysical field reconstruction and uncertainty quantification. The decoder remains comparatively lightweight, while the encoder carries most of the contextual modeling burden. Because the decoder sees mask tokens explicitly, reconstruction is framed as structured inpainting rather than direct denoising of a complete image.

3. Curriculum-guided masking and training objective

Let $M$ denote the total number of micro-patch tokens, with $M=1{,}024$. At training epoch $t$, a random binary mask $64\times 64$0 is sampled, and the mask ratio is defined as

$64\times 64$1

The reported curriculum is piecewise: a fixed mask ratio of $64\times 64$2 for the first 5 epochs, a cosine ramp from $64\times 64$3 to $64\times 64$4 between epochs 5 and 30, and a fixed ratio of $64\times 64$5 thereafter (Naskar et al., 20 Aug 2025). In explicit form,

$64\times 64$6

The training loss is a masked mean-squared error computed only on hidden pixels: $64\times 64$7 where $64\times 64$8 is the ground-truth field, $64\times 64$9 the reconstruction, and $2\times 2$0 the set of masked pixel indices. This restriction of the loss to masked pixels is a defining property of the method: the network is not rewarded for copying visible context, only for inferring the missing field from surrounding information.

The curriculum has a specific empirical role. The reported ablation shows that fixed masking at $2\times 2$1 reaches PSNR $2\times 2$2, MSE $2\times 2$3, SSIM $2\times 2$4, and spectral fidelity $2\times 2$5 in 333 epochs, whereas curriculum masking converges approximately 10% faster, in 286 epochs, with comparable final scores: PSNR $2\times 2$6, MSE $2\times 2$7, SSIM $2\times 2$8, and spectral fidelity $2\times 2$9 (Naskar et al., 20 Aug 2025). This directly counters a possible misconception that the curriculum is presented primarily as a final-accuracy improvement; in the reported results, its principal advantage is faster convergence with comparable terminal performance.

4. Monte Carlo ensemble inference and uncertainty quantification

At inference time, CuMoLoS-MAE uses repeated stochastic masking to approximate a posterior predictive distribution under the masking procedure. For each unseen $(64/2)^2 = 32\times 32 = 1{,}024$0 patch $(64/2)^2 = 32\times 32 = 1{,}024$1, the model draws $(64/2)^2 = 32\times 32 = 1{,}024$2 independent random masks $(64/2)^2 = 32\times 32 = 1{,}024$3 using the final mask ratio $(64/2)^2 = 32\times 32 = 1{,}024$4, computes

$(64/2)^2 = 32\times 32 = 1{,}024$5

and aggregates the resulting ensemble $(64/2)^2 = 32\times 32 = 1{,}024$6 (Naskar et al., 20 Aug 2025). Here, $(64/2)^2 = 32\times 32 = 1{,}024$7 denotes masking out pixels or tokens, either by setting them to zero or replacing them with mask tokens.

The posterior predictive mean reconstruction is

$(64/2)^2 = 32\times 32 = 1{,}024$8

and the per-pixel uncertainty is represented by the variance

$(64/2)^2 = 32\times 32 = 1{,}024$9

The standard deviation $2\times 2$0 is visualized as a per-pixel uncertainty map. In practice, $2\times 2$1 is used to balance stability against compute cost; the ablation further reports that smaller $2\times 2$2 yields noisier uncertainty maps.

The calibration statistics are unusually strong for a reconstruction model in this domain. Across all pixels in the 1,028 test patches, the Pearson correlation between $2\times 2$3 and absolute error $2\times 2$4 is reported as mean per-patch $2\times 2$5, with global Pearson $2\times 2$6 and Spearman $2\times 2$7 (Naskar et al., 20 Aug 2025). When pixels are binned by increasing $2\times 2$8 into deciles, MAE rises monotonically from $2\times 2$9 in the lowest decile to $64\times 64$0 in the highest, a 35-fold span. Likewise, the top 1%, 5%, 10%, and 20% of pixels ranked by $64\times 64$1 capture 10.1%, 30.6%, 43.4%, and 59.4% of total absolute error, respectively. These results indicate that the uncertainty map is not merely a by-product of ensembling; it tracks reconstruction risk in a strongly ordered way.

5. Dataset, preprocessing, and quantitative results

The reported dataset is Doppler-lidar vertical velocity from the ARM Southern Great Plains site. The original measurement grid contains 320 range gates at 30 m spacing, and the experiments focus on the lowest 64 gates, corresponding to 1.92 km. Preprocessing consists of an SNR filter that keeps only gates with intensity $64\times 64$2, velocity clamping to $64\times 64$3, and extraction of non-overlapping $64\times 64$4 patches in time $64\times 64$5 height. The train split covers June 1–9, 2011; the test split is the held-out day June 15, 2011; and the total number of test patches is 1,028 (Naskar et al., 20 Aug 2025).

On those 1,028 test patches, the reported benchmark is:

Method	Reconstruction metrics	Distribution / spectral metrics
8×8 mean-filter	PSNR 23.41, SSIM 0.4950, MSE 0.5186	FID 5.13, Spectral Fidelity 91.67%
CVAE	PSNR 26.70, SSIM 0.4190, MSE 0.4036	FID 3.28, Spectral Fidelity 80.21%
DnCNN (Noise2Void)	PSNR 23.09, SSIM 0.6466, MSE 0.6232	FID 0.12, Spectral Fidelity 36.46%
U-Net (Noise2Void)	PSNR 27.70, SSIM 0.7016, MSE 0.2581	FID 0.44, Spectral Fidelity 49.48%
CuMoLoS-MAE	PSNR 29.45, SSIM 0.7857, MSE 0.1854	FID 1.87, Spectral Fidelity 93.75%

Spectral fidelity is defined on low frequencies $64\times 64$6 by comparing log-PSDs via

$64\times 64$7

and measuring the fraction of bins satisfying $64\times 64$8. In this benchmark, CuMoLoS-MAE yields the highest PSNR, SSIM, and spectral fidelity, and the lowest MSE among the listed methods. The FID values show that no single baseline dominates every metric; this is relevant because the paper’s emphasis is not purely photographic realism but preservation of atmospheric structure and spectra.

A further ablation on window size reports that the best PSNR, SSIM, and spectral fidelity occur at $64\times 64$9, while larger windows degrade spectral fidelity sharply. This indicates that window geometry is not incidental but tied to the scale at which coherent atmospheric motions are represented.

6. Atmospheric features, applications, and interpretive boundaries

The feature-restoration claim is explicit. By reconstructing from up to 70% missing context and operating on $$[64\times 64\ \text{input}] \rightarrow \text{micro-patch tokenizer} \rightarrow \{\text{visible tokens, masked positions}\} \rightarrow +\text{positional encodings} \rightarrow 12\times \text{ViT encoder} \rightarrow 4\times \text{ViT decoder} \rightarrow 64\times 64\ \text{reconstruction}.$$0 micro-patch tokens, CuMoLoS-MAE learns a data-driven prior over vertical-velocity fields and faithfully restores updraft and downdraft cores, shear lines, and small vortices and turbulent eddies (Naskar et al., 20 Aug 2025). The comparison with traditional gap filling is equally explicit: sliding $$[64\times 64\ \text{input}] \rightarrow \text{micro-patch tokenizer} \rightarrow \{\text{visible tokens, masked positions}\} \rightarrow +\text{positional encodings} \rightarrow 12\times \text{ViT encoder} \rightarrow 4\times \text{ViT decoder} \rightarrow 64\times 64\ \text{reconstruction}.$$1 mean filters smear or blur such structures because they average locally and cannot adapt to anisotropic features, whereas CuMoLoS-MAE preserves sharp edges and coherent structures by learning non-local correlations in the training data.

The reported applications follow directly from the combination of mean reconstruction and uncertainty. For real-time data assimilation, the high-fidelity mean reconstructions $$[64\times 64\ \text{input}] \rightarrow \text{micro-patch tokenizer} \rightarrow \{\text{visible tokens, masked positions}\} \rightarrow +\text{positional encodings} \rightarrow 12\times \text{ViT encoder} \rightarrow 4\times \text{ViT decoder} \rightarrow 64\times 64\ \text{reconstruction}.$$2 can be fed into NWP data-assimilation schemes without introducing spurious small-scale noise, while the per-pixel $$[64\times 64\ \text{input}] \rightarrow \text{micro-patch tokenizer} \rightarrow \{\text{visible tokens, masked positions}\} \rightarrow +\text{positional encodings} \rightarrow 12\times \text{ViT encoder} \rightarrow 4\times \text{ViT decoder} \rightarrow 64\times 64\ \text{reconstruction}.$$3 permits down-weighting low-confidence regions. For convection diagnostics, accurate recovery of updraft and downdraft cores and shear lines enhances detection and tracking of storm structures in real time. For long-term climate reanalysis, scaling training over months or years of ARM data is described as a means to provide continuous, uncertainty-quantified vertical profiles, improving estimates of convective transitions under climate change and enabling error-aware reanalysis of historical datasets.

Several interpretive boundaries are also clear from the reported formulation. First, the uncertainty estimate is derived from Monte Carlo over random mask realizations and is described as approximating the posterior predictive distribution under the masking procedure; it is therefore not presented as a direct physical error model tied to instrument noise alone. Second, the main empirical demonstration is on Doppler-lidar vertical velocity from a specific ARM site and split. Third, the curriculum does not simply yield uniformly superior final metrics; its documented benefit is faster convergence with comparable final scores. These points are important for situating the method accurately within masked autoencoding, remote-sensing reconstruction, and uncertainty-aware atmospheric data processing.