OASIC: Occlusion-Agnostic and Severity-Informed Classification

Abstract: Severe occlusions of objects pose a major challenge for computer vision. We show that two root causes are (1) the loss of visible information and (2) the distracting patterns caused by the occluders. Our approach addresses both causes at the same time. First, the distracting patterns are removed at test-time, via masking of the occluding patterns. This masking is independent of the type of occlusion, by handling the occlusion through the lens of visual anomalies w.r.t. the object of interest. Second, to deal with less visual details, we follow standard practice by masking random parts of the object during training, for various degrees of occlusions. We discover that (a) it is possible to estimate the degree of the occlusion (i.e. severity) at test-time, and (b) that a model optimized for a specific degree of occlusion also performs best on a similar degree during test-time. Combining these two insights brings us to a severity-informed classification model called OASIC: Occlusion Agnostic Severity Informed Classification. We estimate the severity of occlusion for a test image, mask the occluder, and select the model that is optimized for the degree of occlusion. This strategy performs better than any single model optimized for any smaller or broader range of occlusion severities. Experiments show that combining gray masking with adaptive model selection improves $\text{AUC}_\text{occ}$ by +18.5 over standard training on occluded images and +23.7 over finetuning on unoccluded images.

• The paper introduces an occlusion-agnostic method combining unsupervised pixel-wise segmentation with adaptive selection from severity-specific models.
• The paper demonstrates significant gains, achieving AUCocc improvements of +18.5 and +23.7 compared to baseline approaches with gray masking and targeted training.
• The paper offers practical insights for deploying robust vision systems in real-world scenarios like autonomous driving, surveillance, and medical imaging.

OASIC: An Occlusion-Agnostic and Severity-Informed Framework for Robust Visual Recognition

Problem Formulation and Motivation

Occlusion remains a dominant challenge in computer vision, particularly in fine-grained recognition scenarios where discriminative details are frequently lost or obscured. Two fundamental obstacles are identified: (1) the systematic loss of salient object information and (2) interference from visually distracting patterns introduced by occluders. For robust deployment in real-world settings—ranging from autonomous driving to surveillance—classification models must address both issues, especially under severe occlusion where less than 10% of an object is visible.

Methodological Framework

The OASIC methodology consists of three integral components: (i) occlusion-agnostic pixel-wise segmentation, (ii) test-time masking to neutralize distracting artifacts, and (iii) adaptive selection from a pool of classification models fine-tuned for varying occlusion severities. All components synergize to enhance recognition robustness under diverse forms and intensities of occlusion.

Occlusion-Agnostic Segmentation

OASIC employs visual anomaly detection, instantiated via AnomalyDINO, to generate per-pixel occlusion likelihood maps without explicit supervision or prior knowledge of the occluder's appearance. This stands in contrast to prompt-driven segmentation approaches (e.g., OVSeg), which are constrained by their dependence on occluder-specific meta-information. Thresholding, performed adaptively using Otsu's method for maximizing inter-class variance, yields a binary occlusion mask that is leveraged for downstream classification.

Masking Distractive Textures

To mitigate the adverse effects of structured occluders (e.g., vegetation, rubble, smoke), occluded regions delineated by the anomaly map are replaced with a uniform gray tone. This effectively suppresses spurious high-frequency structure that typically diverts model attention away from informative object parts. Gray masking is applied both during training (as data augmentation at various severity levels) and at inference, ensuring domain consistency and minimizing distribution shift.

Severity-Informed Model Selection

OASIC introduces the estimation of current occlusion severity per test instance as an explicit signal, calculated as the mean of the anomaly map. A pool of models, each fine-tuned on images with occlusion sampled from a pre-specified range of severities, is maintained. At inference, the estimated severity is used to select the model whose training distribution most closely matches the observed occlusion level, optimizing classification accuracy for each scenario. Empirically, models specialized for a narrow severity range outperform any single model fine-tuned over broader distributions.

Experimental Results

Experiments are conducted on the Stanford Cars fine-grained dataset augmented with synthetic occlusions using realistic textures derived via SAM and Perlin noise masks. Quantitative results substantiate the following key findings:

• OASIC outperforms both standard models and state-of-the-art occlusion augmentation approaches. It achieves an AUCocc improvement of +18.5 over models trained directly on vegetation-occluded samples and +23.7 over those fine-tuned on unoccluded data.
• Gray masking alone yields significant gains by neutralizing occluder distraction, but the integration of severity-informed model selection offers additional, consistent improvement.
• Severe, textured occlusions (vegetation, rubble) are demonstrated to be markedly more disruptive than uniform gray occlusions, underscoring the necessity of distraction suppression.
• Saliency analyses using EigenGrad-CAM indicate that textured occluders severely degrade attention allocation to the object of interest. OASIC's masking and adaptive selection considerably mitigate this phenomenon.
• The unsupervised, anomaly-based segmentation achieves high AUROC and Average Precision across diverse occlusion types, surpassing prompt-dependent baselines without any occluder-specific priors.

Implications and Future Work

OASIC's dual-pronged strategy—treating occlusion as a visual anomaly and dynamically adapting model selection based on quantitative severity estimation—directly addresses the multi-faceted impact of occlusion. The framework's reliance on unsupervised anomaly maps for both segmentation and severity assessment represents a shift from handcrafted occluder priors towards generalizable, scalable solutions.

Practical implications extend beyond fine-grained recognition. Any vision system where partial visibility or unpredictable occluders occur—medical imaging, autonomous agents, visual surveillance—can directly benefit from OASIC’s occlusion-agnostic, severity-informed paradigm.

Potential extensions include the unification of segmentation, severity estimation, and classification within an end-to-end architecture, where pixel-level occlusion likelihoods serve as auxiliary supervision. This could promote a form of learned visibility reasoning, enhancing not just occlusion robustness, but also interpretability and general domain shift resilience.

Conclusion

OASIC provides an integrated response to the dual threats posed by occlusions: information loss and the confusion from distractive patterns. By detecting arbitrary occluders as visual anomalies, masking them at the pixel level, and dynamically selecting classifiers based on estimated severity, OASIC attains substantial and robust improvements over conventional approaches. This severity-informed, agnostic mechanism constitutes a reproducible, theoretically motivated advance for practical vision systems operating in imperfect, real-world conditions.

For detailed methodology and empirical validation, see "OASIC: Occlusion-Agnostic and Severity-Informed Classification" (2604.04012).