Semantic-Aware Masking Progressive Learning (SAMPL)

• SAMPL is a training paradigm that uses semantic-aware masking combined with progressive curriculum schedules to gradually challenge models.
• It employs structured occlusions derived from semantic segmentation to enhance robustness in tasks like human–object interaction detection and masked autoencoding.
• Empirical results show that SAMPL improves both clean accuracy and performance under corruption, outperforming random or perturbation-based masking methods.

Semantic-Aware Masking-based Progressive Learning (SAMPL) is a training paradigm that combines semantic segmentation with progressive, curriculum-inspired masking schedules to optimize model robustness, representation quality, and interpretability in self-supervised and supervised learning systems. Originating from contexts such as masked autoencoding in computer vision and robust human-object interaction (HOI) detection, SAMPL leverages structural priors—frequently derived from models such as SAM (Segment Anything Model) or learned semantic parts—as the basis for controlled, curriculum-driven occlusions during training. This methodology enforces the learning of both global and local cues and enables adaptation to domain complexity and corruption.

1. Principle and Conceptual Foundations

Semantic-Aware Masking-based Progressive Learning is defined by two joint components: (i) semantic-aware masking, in which training data are selectively occluded using masks derived from semantic segmentations or community/part assignments; and (ii) progressive learning, in which the masking schedule follows a curriculum that transitions systematically from easy tasks to harder ones as training advances. Unlike purely random masking, SAMPL guarantees that occlusions respect meaningful object, part, or community boundaries in the data and exposes models to increasingly challenging partial information regimes.

In human-object interaction detection, SAMPL employs instance-level masks from SAM to selectively occlude regions crucial to the HOI signal, and dynamically varies occlusion severity based on validation metrics (Wen et al., 12 Jul 2025). For masked autoencoder pre-training, semantic parts are extracted through self-supervised attention mechanisms, and a curriculum interpolates between intra-part (patch-level) masking and inter-part (whole semantic part) masking (Li et al., 2022).

2. Semantic-Aware Masking Mechanisms

The core of SAMPL lies in its mask generation mechanisms, which draw on semantic or structural segmentations. For each training instance, relevant semantic regions—such as object instances in HOI, or learned semantic parts in images—are identified:

• HOI Detection: Given human/object bounding boxes $\{b\}$, SAM yields binary instance masks $\mathbf{M}_b\in\{0,1\}^{H\times W}$. These are morphologically dilated and convex-hull approximated to yield $\widehat{\mathbf{M}_b}$. For each severity $\omega$, a coverage ratio $[r^w_\omega, r^h_\omega]$ is used to define relative occlusion, and the resulting mask is applied as $I_\omega = I \odot (1-\mathbf{M}^b_\omega)$ (Wen et al., 12 Jul 2025).
• Semantic Part Masking for MAE: Input images are decomposed into $N$ semantic parts using a self-supervised refinement of ViT attention maps. A class token feature $F_c$ and patch tokens $F$ undergo channel-wise projections to produce part-to-patch attention maps $M\in\mathbb{R}^{N\times HW}$. Hard assignment $\mathbf{M}_b\in\{0,1\}^{H\times W}$0 yields part labels for each patch (Li et al., 2022).

By basing masking on these structural assignments, models are exposed during training to occlusions that correspond to meaningful or functionally relevant subsets of the data.

3. Progressive Curriculum and Masking Scheduling

Masking in SAMPL is not fixed but evolves over the course of training, following a progressive or curriculum-based heuristic that modulates masking "difficulty."

• HOI Detection (RoHOI): The curriculum comprises four discrete masking severities $\mathbf{M}_b\in\{0,1\}^{H\times W}$1 (clean to heavily occluded). Scheduling is governed dynamically: at each epoch, a validation metric (e.g., AP$\mathbf{M}_b\in\{0,1\}^{H\times W}$2) is monitored, and masking severity is escalated only when further progress at the current level saturates. The selection between the "clean" and current severity is implemented via a score $\mathbf{M}_b\in\{0,1\}^{H\times W}$3, favoring under-explored and high-performing severities (Wen et al., 12 Jul 2025).
• Masked Autoencoders (SemMAE): A continuous interpolation parameter $\mathbf{M}_b\in\{0,1\}^{H\times W}$4 transitions the masking from intra-part patch-level to full-part occlusion as training proceeds, where $\mathbf{M}_b\in\{0,1\}^{H\times W}$5 controls schedule aggressiveness. For each part $\mathbf{M}_b\in\{0,1\}^{H\times W}$6, the masked patch count is $\mathbf{M}_b\in\{0,1\}^{H\times W}$7, balancing within-part and whole-part occlusion (Li et al., 2022).

This progressive schedule ensures that at early training stages, the model receives more complete or redundant information (facilitating stable feature extraction), while at later stages, the occlusions become more challenging, fostering discrimination and robustness.

4. Empirical Impact and Benchmarks

SAMPL has demonstrated effectiveness across robust detection, classification, and transfer learning settings.

• Human-Object Interaction Detection: On the RoHOI benchmark, SAMPL-trained RLIPv2 with Swin-T backbone yields improved robustness, raising V-COCO MRI from 45.05 to 48.83 (+3.78 points) and CRI from 0.24 to 0.27 (+12.5%), while maintaining clean-set accuracy (AP$\mathbf{M}_b\in\{0,1\}^{H\times W}$8 from 70.76 to 71.13) (Wen et al., 12 Jul 2025). Ablations confirm that SAM-guided semantic masking (SAMSM) outperforms adaptive Gaussian noise and random pixelation for MRI/CRI tradeoff.
• Semantic-Guided MAE Pre-Training: On ImageNet-1k, SemMAE, which instantiates SAMPL with self-supervised part learning, achieves 84.5% fine-tuning accuracy (vs. 83.6% for MAE). In semantic segmentation (ADE20K), mIoU increases to 46.3 (vs. 46.1 for MAE). Masking curriculum ablation establishes that a smooth interpolation $\mathbf{M}_b\in\{0,1\}^{H\times W}$9 outperforms random, intra-part, or inter-part masking alone (Li et al., 2022).

These results indicate that semantic-aware, curriculum-based masking enables models to generalize better under both clean and corrupted input regimes.

5. Comparative Analysis with Alternative Masking Strategies

SAMPL’s gains are attributable to its semantic structure awareness and progressive scheduling, which distinguish it from random, block-wise, or attention-guided masking:

Method	Clean Acc. (V-COCO)	Robustness MRI	CRI
RLIPv2 baseline	70.76	45.05	0.24
RLIPv2 + Adaptive Gaussian	70.51	44.74	0.24
RLIPv2 + Pixelation	69.64	47.59	0.25
SAMPL (SAMSM)	70.24	48.20	0.27

This empirical comparison demonstrates that occlusions guided by structure-aware segmentations outperform both random and simple perturbation-based augmentations, especially under challenging corruption.

A plausible implication is that models trained with SAMPL are better equipped to exploit both holistic and partial cues, making them resilient to input degradation while retaining strong discriminative performance.

6. Limitations and Open Challenges

SAMPL’s principal limitations are linked to the scope of its robustness evaluation and structural prior quality:

• The RoHOI benchmark focuses exclusively on synthetic corruptions; real-world domain shifts (such as video compression artifacts and adversarial attacks) are not directly assessed (Wen et al., 12 Jul 2025).
• For severe or atypical corruptions (e.g., moiré, extreme blur), even SAMPL-trained models degrade, indicating that further extensions—such as multi-modal data integration or explicit occlusion reasoning—are necessary.
• In the sampled contexts, mask schedules and ratios require fine-tuning; overly aggressive occlusion can degrade both clean and robust accuracy.
• Issues of bias and fairness under structural masking and dataset shift remain largely unexamined.

Future work is expected to address these challenges through improved data modalities, adaptive curriculum design, and domain adaptation approaches. This suggests that while SAMPL effectively bridges clean-data accuracy and robustness, additional architectural and procedural advances are needed for comprehensive real-world applicability.

7. Contextualization within Broader Literature

SAMPL’s central thesis—that leveraging semantic partitions and adapting training schedules increases robustness and generalization—finds support in related approaches such as semantic-guided masking for masked autoencoders (Li et al., 2022), progressive community-aware masking in brain network analysis (Chen et al., 29 Jun 2026), and structure-aware perturbation strategies in robust detection (Wen et al., 12 Jul 2025). The consistent improvements achieved through this paradigm reinforce the value of semantic priors and curriculum-based learning in both self-supervised and supervised regimes across diverse domains.