Characterize the high-level features captured by VGG-based segmentation loss

Determine which high-level features of image segmentations are measured by the pre-trained VGG network used to compare predicted and ground-truth segmentations, and which such features the network ignores.

Background

The paper reviews a method that uses differences in intermediate activations of a pre-trained VGG network as an auxiliary loss for comparing predicted and ground-truth segmentations. Although that loss was shown empirically to be sensitive to certain topological changes, its precise representational scope is unresolved.

Identifying the features captured or neglected by the VGG-based loss would clarify what global shape and topology information the method actually incorporates and would help assess its reliability as a topology-aware segmentation objective.

References

However, it is unclear which kinds of high-level features of the segmentations this VGG network measures, and which will be ignored.

— A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology  (1910.01877 - Clough et al., 2019) in Section II-A, “Shape constraints in CNN segmentation”