GLCM-MAE: Texture-Preserving Pre-training

• The paper introduces a texture-aware GLCM reconstruction loss that replaces pure MSE to capture diagnostically salient texture cues in medical images.
• It leverages a differentiable kernel density estimation to compute soft co-occurrence matrices, preserving intensity and spatial relationships within patches.
• Combined with SSIM and MSE in a two-phase training schedule, the method enhances accuracy across various tasks like ultrasound, X-ray, and CT classification.

Searching arXiv for the cited paper and closely related work on differentiable histogram/GLCM methods. GLCM-MAE is a masked autoencoder pre-training framework for medical image classification that replaces a purely pixel-wise reconstruction criterion with a texture-aware objective derived from the Gray Level Co-occurrence Matrix (GLCM). It is introduced in "Focus on Texture: Rethinking Pre-training in Masked Autoencoders for Medical Image Classification" (Madan et al., 15 Jul 2025). The central observation is that standard MAE pre-training uses a pixel-wise mean squared error (MSE) between original and reconstructed values, which encourages blurred reconstruction and can suppress the fine textural and morphological cues that are often diagnostically salient in medical imaging. GLCM-MAE addresses this by matching co-occurrence statistics between reconstructed and original patches through a differentiable GLCM-based loss, with the stated aim of preserving intensity and spatial relationships relevant to downstream classification (Madan et al., 15 Jul 2025).

1. Motivation and problem setting

Masked Autoencoders (MAEs) are described as a dominant strategy for self-supervised representation learning in natural images, where pre-training reconstructs masked patches using pixel-wise MSE. In the formulation given for standard MAEs, the loss is

$$L_{MSE} = \frac{1}{N} \sum_{i=1}^N \left\|x_m^{(i)} - \hat x_m^{(i)}\right\|_2^2.$$

The framework identifies a limitation of this objective in medical imaging: minimizing MSE tends to remove high-frequency components and produce blurred outputs, while preserving dominant edges. The paper states that this can be adequate for natural images, but that it fails when texture cues are more important for classification of a visual abnormality, as in ultrasound, X-ray, or CT imaging (Madan et al., 15 Jul 2025).

The motivating claim is that many diagnostic tasks rely on subtle morphological cues such as heterogeneity, granularity, and coarseness, which reside in second-order statistics of gray levels rather than in edges alone. GLCM, a classical radiomics feature, is presented as a suitable mechanism for encoding such information because it counts how often pairs of intensities occur at a fixed spatial offset. The framework therefore treats pre-training not merely as masked pixel recovery but as preservation of medically salient texture statistics.

A common misunderstanding would be to interpret GLCM-MAE as a rejection of pixel reconstruction altogether. The reported design does not do so. Instead, the method combines MSE, GLCM, and SSIM in a staged training schedule, and the ablation study shows that the joint objective performs better than any isolated component (Madan et al., 15 Jul 2025).

2. Gray Level Co-occurrence Matrix formulation

The Gray Level Co-occurrence Matrix is defined for a grayscale patch with $G$ intensity levels. Let $I$ be a $G$-level grayscale patch, and for a displacement $(d,\theta)$ define an offset vector $\Delta = (\Delta x,\Delta y)$. The co-occurrence matrix entry is

$$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$

where $\delta[\cdot]$ is the Kronecker delta.

The normalized form is

$$p(v,w;d,\theta) = \frac{G(v,w;d,\theta)}{\sum_{v',w'} G(v',w';d,\theta)},$$

with $\sum_{v,w} p(v,w)=1$.

Within the framework, GLCM is described as encoding textural measures such as contrast, correlation, energy (angular second moment), and homogeneity. The paper also notes that one often averages $G$0 over multiple directions $G$1 or uses only horizontal and vertical offsets for simplicity. In the actual implementation of GLCM-MAE, the offsets are horizontal $G$2 and vertical $G$3 (Madan et al., 15 Jul 2025).

This formulation situates GLCM-MAE in a radiomics-informed lineage. A plausible implication is that the method attempts to transfer a handcrafted textural prior from classical medical image analysis into self-supervised representation learning, but the explicit claim made is narrower: matching GLCM helps preserve morphological features because GLCM captures intensity and spatial relationships in an image (Madan et al., 15 Jul 2025).

3. Differentiable GLCM matching loss

A central technical issue is that the count-based GLCM definition is non-differentiable. GLCM-MAE therefore uses a differentiable kernel density estimation approach, cited in the paper as Avi-Aharon et al. TPAMI 2023, to construct a soft co-occurrence representation suitable for backpropagation (Madan et al., 15 Jul 2025).

For a single image $G$4 with $G$5 pixels, the soft density estimate is

$G$6

with

$G$7

and $G$8 the sigmoid.

To construct a joint histogram for horizontally adjacent pairs, the method forms $G$9 by dropping the last column of $I$0 and $I$1 by dropping the first column. For each bin center $I$2 with $I$3 and bin width $I$4, the soft assignments are

$I$5

$I$6

The differentiable joint histogram $I$7 is then

$I$8

The description further states that, as $I$9, this converges to the classical GLCM. A vertical co-occurrence matrix $G$0 can be computed analogously with an offset along rows, and one may sum or concatenate $G$1 and $G$2 to capture both directions (Madan et al., 15 Jul 2025).

At the patch level, if $G$3 and $G$4 denote the differentiable GLCMs of the predicted and ground-truth patches, the GLCM loss is

$G$5

The stated role of this loss is to force the model to preserve local texture, in contrast to MSE, which primarily enforces per-pixel fidelity.

4. Integration into the masked autoencoder architecture

GLCM-MAE is integrated into a masked autoencoder backbone with a specific architecture and training regime (Madan et al., 15 Jul 2025). The patch size is $G$6, and the masking ratio is $G$7. The encoder is ViT-B with 12 layers, embedding dimension 768, and 12 heads; only visible patches are fed into it. The decoder is a shallow ViT with 8 layers, embedding dimension 512, and 16 heads, and receives both encoded visible tokens and mask tokens with positional embeddings. The reconstruction head projects decoder outputs back to patch-pixel space, using RGB for natural MAE and grayscale for medical images.

The pre-training schedule has two phases:

1. Warm-up, epochs 1–200: only $G$8 is used, with $G$9, $(d,\theta)$0.
2. Full texture-aware training, epochs 201–400: the objective becomes

$(d,\theta)$1

with $(d,\theta)$2.

The description attributes distinct functions to the three terms. MSE stabilizes reconstruction gradients, SSIM encourages global consistency of structure, and GLCM preserves local texture. The ablation results are consistent with this decomposition: GLCM+SSIM is reported as insufficient for global structure relative to the full objective, while the combination of GLCM+MSE+SSIM gives the best result on the GBCU dataset (Madan et al., 15 Jul 2025).

During downstream fine-tuning, the decoder and reconstruction head are discarded. Only the encoder is retained, and a lightweight classification head is appended. This makes GLCM-MAE an encoder pre-training method rather than a change to the downstream classifier itself.

5. Hyperparameters, optimization, and fine-tuning protocol

The implementation details reported for GLCM-MAE are specific and fixed across the described experiments (Madan et al., 15 Jul 2025).

Component	Setting
Patch size	$(d,\theta)$3
Mask ratio	75% (uniform random)
GLCM offsets	horizontal $(d,\theta)$4 and vertical $(d,\theta)$5
Number of intensity bins	$(d,\theta)$6 (for 8-bit images)
Bin width	$(d,\theta)$7
KDE bandwidth	$(d,\theta)$8; $(d,\theta)$9 gave best results
Warm-up loss weights	$\Delta = (\Delta x,\Delta y)$0
Full loss weights	$\Delta = (\Delta x,\Delta y)$1

For optimization, the paper specifies AdamW; pre-training learning rate $\Delta = (\Delta x,\Delta y)$2, weight decay $\Delta = (\Delta x,\Delta y)$3, batch size $\Delta = (\Delta x,\Delta y)$4, and total epochs $\Delta = (\Delta x,\Delta y)$5. Fine-tuning uses a learning rate that starts at $\Delta = (\Delta x,\Delta y)$6 decay, weight decay $\Delta = (\Delta x,\Delta y)$7, batch size $\Delta = (\Delta x,\Delta y)$8, and epochs $\Delta = (\Delta x,\Delta y)$9 (Madan et al., 15 Jul 2025).

The downstream classification procedure is also explicit. The decoder and patch-reconstruction head are dropped, the ViT encoder is retained, and a linear or MLP classification head is appended with output dimension matching the number of classes. Fine-tuning is performed end-to-end on labeled medical images using cross-entropy or AUROC-based loss, and the selected checkpoint is the one with highest validation accuracy or AUROC (Madan et al., 15 Jul 2025).

These details show that GLCM-MAE is not presented as a change to the masking strategy or to the ViT encoder topology. Its novelty resides in the reconstruction criterion and its scheduling.

6. Empirical results, ablations, and interpretive scope

The reported empirical claim is that unsupervised pre-training on medical images with the proposed GLCM loss improves representations for downstream tasks and that GLCM-MAE outperforms the current state-of-the-art across four tasks (Madan et al., 15 Jul 2025).

The task-level results are as follows:

Task	Baseline and GLCM-MAE result
Gallbladder cancer detection (GBCU ultrasound, 5-fold CV)	MAE accuracy $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$0, GLCM-MAE $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$1; sensitivity improved from $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$2 to $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$3
Breast cancer detection (BUSI ultrasound)	MAE $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$4, GLCM-MAE $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$5
Pneumonia detection (Chest X-ray)	MAE AUROC $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$6, GLCM-MAE $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$7
COVID-19 detection (CT scans)	MAE AUROC $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$8, GLCM-MAE $$G(v,w;d,\theta) = \sum_{x,y} \delta[I(x,y)=v]\cdot\delta[I(x+\Delta x,y+\Delta y)=w],$$9

The corresponding improvements stated in the abstract are $\delta[\cdot]$0 for gallbladder cancer detection from ultrasound images, $\delta[\cdot]$1 for breast cancer detection from ultrasound, $\delta[\cdot]$2 for pneumonia detection from X-rays, and $\delta[\cdot]$3 for COVID detection from CT. For the CT task, the paper also reports a $\delta[\cdot]$4 improvement versus CropMixPaste. All improvements are stated to be statistically significant with $\delta[\cdot]$5 (Madan et al., 15 Jul 2025).

The ablation study on the GBCU dataset clarifies several design choices. For bandwidth, $\delta[\cdot]$6 performed best, while smaller $\delta[\cdot]$7 under-smoothed the histogram. For loss composition, the reported accuracies are:

• MSE alone $\delta[\cdot]$8 acc
• GLCM alone $\delta[\cdot]$9 acc
• GLCM+MSE $$p(v,w;d,\theta) = \frac{G(v,w;d,\theta)}{\sum_{v',w'} G(v',w';d,\theta)},$$0 acc
• GLCM+SSIM $$p(v,w;d,\theta) = \frac{G(v,w;d,\theta)}{\sum_{v',w'} G(v',w';d,\theta)},$$1 acc
• GLCM+MSE+SSIM $$p(v,w;d,\theta) = \frac{G(v,w;d,\theta)}{\sum_{v',w'} G(v',w';d,\theta)},$$2 acc

The mask ratio of $$p(v,w;d,\theta) = \frac{G(v,w;d,\theta)}{\sum_{v',w'} G(v',w';d,\theta)},$$3 was optimal, ViT-B outperformed smaller variants, and a decoder depth of 8 layers was best; a decoder that was too shallow underfit, while one that was too deep wasted capacity (Madan et al., 15 Jul 2025).

These results constrain the interpretation of the method. The evidence does not support the claim that texture-only reconstruction is sufficient in isolation; rather, the reported best-performing configuration is a hybrid objective in which GLCM is the key addition. This suggests that the contribution of GLCM-MAE lies in reweighting the pre-training target toward texture preservation without abandoning global structural consistency or stable reconstruction dynamics.

7. Position within medical self-supervision and radiomics

GLCM-MAE explicitly draws inspiration from radiomics, where GLCM is a classical feature family for characterizing texture. The framework transfers this notion into MAE pre-training by replacing hard co-occurrence counting with a differentiable approximation and using it as a reconstruction loss (Madan et al., 15 Jul 2025). In that sense, it connects self-supervised learning with second-order gray-level statistics traditionally associated with handcrafted medical image analysis.

The paper’s summary is that GLCM-MAE augments the standard masked autoencoder with a texture-aware reconstruction loss and that, by softly matching co-occurrence matrices between original and reconstructed patches, it forces a medical image model to retain fine textural cues underlying many diagnostic tasks. Source code and pre-trained models are reported as available at the project repository linked by the authors (Madan et al., 15 Jul 2025).

A plausible implication is that the framework is most relevant when downstream labels depend on textural morphology rather than only on large-scale contours or dominant edges. That interpretation is consistent with the paper’s stated motivation, its radiomics grounding, and its empirical emphasis on ultrasound, X-ray, and CT classification tasks.