Achieving Fair Skin Lesion Detection through Skin Tone Normalization and Channel Pruning

Abstract: Recent works have shown that deep learning based skin lesion image classification models trained on unbalanced dataset can exhibit bias toward protected demographic attributes such as race, age,and gender. Current bias mitigation methods usually either achieve high level of fairness with the degradation of accuracy, or only improve the model fairness on a single attribute. Additionally usually most bias mitigation strategies are either pre hoc through data processing or post hoc through fairness evaluation, instead of being integrated into the model learning itself. To solve these existing drawbacks, we propose a new Individual Typology Angle (ITA) Loss-based skin tone normalization and data augmentation method that directly feeds into an adaptable meta learning-based joint channel pruning framework. In skin tone normalization, ITA is used to estimate skin tone type and adjust automatically to target tones for dataset balancing. In the joint channel pruning framework, two nested optimization loops are used to find critical channels.The inner optimization loop finds and prunes the local critical channels by weighted soft nearest neighbor loss, and the outer optimization loop updates the weight of each attribute using group wise variance loss on meta-set. Experiments conducted in the ISIC2019 dataset validate the effectiveness of our method in simultaneously improving the fairness of the model on multiple sensitive attributes without significant degradation of accuracy. Finally, although the pruning mechanism adds some computational cost during training phase, usually training is done off line. More importantly,