Efficient KernelSHAP Explanations for Patch-based 3D Medical Image Segmentation

Abstract: Perturbation-based explainability methods such as KernelSHAP provide model-agnostic attributions but are typically impractical for patch-based 3D medical image segmentation due to the large number of coalition evaluations and the high cost of sliding-window inference. We present an efficient KernelSHAP framework for volumetric CT segmentation that restricts computation to a user-defined region of interest and its receptive-field support, and accelerates inference via patch logit caching, reusing baseline predictions for unaffected patches while preserving nnU-Net's fusion scheme. To enable clinically meaningful attributions, we compare three automatically generated feature abstractions within the receptive-field crop: whole-organ units, regular FCC supervoxels, and hybrid organ-aware supervoxels, and we study multiple aggregation/value functions targeting stabilizing evidence (TP/Dice/Soft Dice) or false-positive behavior. Experiments on whole-body CT segmentations show that caching substantially reduces redundant computation (with computational savings ranging from 15% to 30%) and that faithfulness and interpretability exhibit clear trade-offs: regular supervoxels often maximize perturbation-based metrics but lack anatomical alignment, whereas organ-aware units yield more clinically interpretable explanations and are particularly effective for highlighting false-positive drivers under normalized metrics.

• The paper introduces a localized, patch-based adaptation of KernelSHAP tailored for efficient 3D segmentation in medical imaging using nnU-Net.
• The paper evaluates three spatial partitioning methods—full organs, regular FCC supervoxels, and hybrid organ-aware FCC—highlighting trade-offs between interpretability and granularity.
• The paper demonstrates that patch logit caching achieves 15%-30% computational savings while preserving high attribution faithfulness for identifying false-positive activations.

Efficient KernelSHAP-based Explainability for 3D Medical Image Segmentation

Introduction

This paper introduces an efficient adaptation of KernelSHAP for patch-based 3D segmentation, specifically targeting dense volumetric tasks encountered in medical imaging with nnU-Net. The motivation stems from the need for faithful, model-agnostic explanations of 3D segmentations, where traditional perturbation-based XAI methods induce prohibitive computational costs and often lack anatomical interpretability due to geometric, not semantic, supervoxel tessellations. The framework focuses on localizing the computation to a user-defined region of interest (ROI) and restricting perturbation to its minimal receptive field, thus reducing both computational complexity and irrelevant attribution. The study thoroughly investigates the influence of unit partitioning—full organs, regular supervoxels (FCC), and hybrid organ-aware FCC supervoxels—on attribution quality and faithfulness, and examines the impact of various aggregation (value) functions used for SHAP scoring.

Methodology

The framework adapts KernelSHAP to 3D patch-wise architectures by integrating several crucial design choices:

• Localized Coalition Evaluation: Only the ROI and its receptive field (RF) are perturbed and recomputed, ensuring both faithfulness and computational efficiency.
• Unit Partitioning: Three spatial decompositions are evaluated:
  • Full Organs: Based on semantic masks from TotalSegmentator, optimizing clinical interpretability at the cost of granularity.
  • Regular Supervoxels: FCC tessellation maximizes spatial regularity and granular control, but units ignore anatomy.
  • Hybrid Organ-Aware Supervoxels: Cells from the FCC grid are subdivided using organ labels, preserving anatomical boundaries and intra-organ granularity.

After partitioning the input, binary coalitions are sampled, and corresponding units are replaced with a consistent physically plausible intensity baseline ($b=-1024$ HU). Several aggregation functions are considered for the SHAP value calculation: true positive logit support, false positive penalization, Dice, and soft Dice surrogates. To avoid redundant patch inference, the pipeline caches baseline patch logits, reusing them in coalitions where the patch is unperturbed, leading to nontrivial computational savings.

Figure 1: Coronal slices display the difference between regular FCC supervoxels (left) and hybrid organ-aware FCC supervoxels (right), highlighting improved anatomical alignment in the hybrid scheme.

Experiments

Data and Setup

The evaluation uses a clinical whole-body CT dataset (40 patients, 8 for testing) addressing the segmentation of lymph nodes and spleen for TMLI planning. SHAP explanations are generated for nnU-Net, disabling test-time augmentation, and the patch caching mechanism is benchmarked across the unit definitions and aggregation functions. For reliable KernelSHAP estimates, a sampling budget of 1000–2000 coalitions is used, adjusted based on partition size.

Evaluation Metrics

Faithfulness is quantified with perturbation-based metrics (MoRF and LeRF curves), focusing on area over the perturbation curve (AOPC) and area between perturbation curves (ABPC), using both absolute and normalized forms. The approach controls for variations in score range and feature granularity across configurations. Furthermore, computational efficiency is analyzed via cache hit ratios and inference runtime statistics.

Results

Qualitative Analysis

Attribution maps reveal that:

• Full Organ Partitioning yields highly interpretable regions aligned with organ boundaries but lacks fine spatial differentiation within organs.
• Regular FCC Supervoxels provide higher faithfulness in perturbation metrics but result in attributions with weak anatomical correspondence, increasing potential clinical ambiguity.
• Hybrid FCC Supervoxels synergize anatomical alignment and spatial resolution, enabling identification of false-positive subregions within organs.

Figure 2: Qualitative attribution maps for full-organ units (volume 7) across aggregation functions, evidencing high interpretability.

Figure 3: Attribution maps for regular FCC supervoxels show increased local granularity but diminished semantic alignment.

Figure 4: Hybrid organ-aware FCC attribution maps elucidate intra-organ evidence, especially for false-positive behaviors.

Quantitative Assessment

• Faithfulness: Regular supervoxels attain maximal (n)AOPC/(n)ABPC in TP, Dice, and soft Dice aggregations, attributed to both their granularity and frequent overlap with the target. However, such high metrics are partially an artifact of spatial partitioning and target inclusion, not solely SHAP ranking quality. Hybrid units surpass regular and full-organ supervoxels in normalized FP-based faithfulness, indicating their superior localization of substructures responsible for spurious activations.
• Computational Efficiency: Patch caching achieves savings of 15%–30%, most effective for organ-constrained or hybrid partitions where perturbations are spatially limited, validating the intended trade-off between runtime and granularity.

Figure 5: MoRF and LeRF curves for full organs demonstrate limited separation and gradual score degradation due to minimal unit count.

Figure 6: Regular FCC supervoxels show steep score drop in MoRF confirming high faithfulness—dominated by direct perturbation of target voxels.

Figure 7: Hybrid organ-aware FCC units enable the most precise identification and isolation of false-positive drivers.

Discussion

The framework's principal innovation lies in combining anatomical semantics into efficient SHAP-based explanation for 3D segmentation. The results corroborate the trade-off between segmentation faithfulness and clinical interpretability: purely geometric partitioning (FCC) favors quantitative perturbation metrics but lacks clinical actionability, while anatomical units offer stable explanations that are less sensitive but more interpretable. The hybrid method offers an effective compromise, particularly in the context of false-positive analysis.

The influence of the aggregation function is nontrivial—TP, Dice, and soft Dice highlight stabilizing evidence within the target, while FP aggregation uncovers destabilizing contextual features or drivers of spurious model output. The choice of aggregation therefore needs to be dynamically selected based on whether the aim is to validate segmentation confidence, debug failure cases, or interrogate specific error modalities.

Limitations are acknowledged, notably the relatively small test set, KernelSHAP's dependency on the linear surrogate and masking artifacts, and the absence of expert-in-the-loop or real usability validation. The findings should be interpreted with these caveats.

Implications and Future Directions

The study provides a practical method for generating localized, clinically interpretable SHAP attributions for large-scale 3D segmentation, suggesting multiple avenues for future work:

• Integration of adaptive, data-driven 3D supervoxels (e.g., SLIC, SEEDS) to further improve anatomical alignment and cost-regularity balance.
• Development of inpainting-based perturbations replacing hard masking, potentially yielding more realistic out-of-distribution behavior and even greater fidelity.
• Exploration of enhanced caching mechanisms (e.g., coalition-level cache hits) and scaling to larger datasets and more complex multi-structure tasks.
• Comprehensive clinical validation to link perturbation-based faithfulness to real-world trust and diagnostic improvement.

Conclusion

This work demonstrates that localized, anatomy-aligned SHAP explanations for 3D segmentation models are practically feasible and computationally efficient via receptive-field restriction and patch logit caching. The hybrid organ-aware FCC approach is highlighted as offering superior balance between interpretability and quantitative faithfulness, especially for assessing the sources of false positives. Together, these innovations provide a foundation for more actionable, reliable, and scalable explainability in clinical AI workflows.

Reference: "Efficient KernelSHAP Explanations for Patch-based 3D Medical Image Segmentation" (2604.11775)