Task-dependent optimal aggregation of segmentation uncertainty for OoD detection

Determine which specific aggregation strategy for pixel-wise uncertainty maps is most suitable to decide whether a segmentation sample is in-distribution or out-of-distribution for a given dataset and task setting in image segmentation.

Background

The paper studies aggregation strategies that reduce pixel-wise uncertainty maps in image segmentation to scalar, image-level scores for downstream tasks such as out-of-distribution (OoD) detection and failure detection. While global averaging is commonly used, the authors show it can ignore critical spatial structures and that no single aggregation strategy consistently dominates across datasets or tasks.

To address variability, the authors introduce spatially-aware aggregators and a meta-aggregator (GMM-based) that combines multiple strategies. Despite these contributions, they explicitly note that, for a given deployment setting, it remains unresolved which single aggregation strategy is most suitable to decide whether a sample is in-distribution or OoD.

References

When deploying the same trained model on a new, potentially out-of-distribution sample our experimental results in Fig.~\ref{fig:ood_results} and Fig.~\ref{fig:fd_results} reveal a key challenge: it remains unclear which specific AggS is most suitable for determining whether a given sample is \iid{} or OoD for any particular task.

— Better than Average: Spatially-Aware Aggregation of Segmentation Uncertainty Improves Downstream Performance  (2603.29941 - Guarino et al., 31 Mar 2026) in Supplementary Material, Section "Details on Meta-Aggregation via GMM", Subsection "Gaussian Mixture Models"