Saliency-Based diversity and fairness Metric and FaceKeepOriginalAugment: A Novel Approach for Enhancing Fairness and Diversity (2411.00831v1)
Abstract: Data augmentation has become a pivotal tool in enhancing the performance of computer vision tasks, with the KeepOriginalAugment method emerging as a standout technique for its intelligent incorporation of salient regions within less prominent areas, enabling augmentation in both regions. Despite its success in image classification, its potential in addressing biases remains unexplored. In this study, we introduce an extension of the KeepOriginalAugment method, termed FaceKeepOriginalAugment, which explores various debiasing aspects-geographical, gender, and stereotypical biases-in computer vision models. By maintaining a delicate balance between data diversity and information preservation, our approach empowers models to exploit both diverse salient and non-salient regions, thereby fostering increased diversity and debiasing effects. We investigate multiple strategies for determining the placement of the salient region and swapping perspectives to decide which part undergoes augmentation. Leveraging the Image Similarity Score (ISS), we quantify dataset diversity across a range of datasets, including Flickr Faces HQ (FFHQ), WIKI, IMDB, Labelled Faces in the Wild (LFW), UTK Faces, and Diverse Dataset. We evaluate the effectiveness of FaceKeepOriginalAugment in mitigating gender bias across CEO, Engineer, Nurse, and School Teacher datasets, utilizing the Image-Image Association Score (IIAS) in convolutional neural networks (CNNs) and vision transformers (ViTs). Our findings shows the efficacy of FaceKeepOriginalAugment in promoting fairness and inclusivity within computer vision models, demonstrated by reduced gender bias and enhanced overall fairness. Additionally, we introduce a novel metric, Saliency-Based Diversity and Fairness Metric, which quantifies both diversity and fairness while handling data imbalance across various datasets.
- Random data augmentation based enhancement: a generalized enhancement approach for medical datasets, in: 24th Irish Machine Vision and Image Processing (IMVIP) Conference.
- Class specific autoencoders enhance sample diversity. Journal of Broadcast Engineering 26.
- Multimodal datasets: misogyny, pornography, and malignant stereotypes. ArXiv Preprint ArXiv:2110.01963 .
- Gender shades: Intersectional accuracy disparities in commercial gender classification, in: Conference On Fairness, Accountability And Transparency, pp. 77–91.
- Precise single-stage detector. arxiv 2022. arXiv preprint arXiv:2210.04252 .
- Audd: audio urdu digits dataset for automatic audio urdu digit recognition. Applied Sciences 11, 8842.
- Gridmask data augmentation. ArXiv Preprint ArXiv:2001.04086 .
- Salfmix: a novel single image-based data augmentation technique using a saliency map. Sensors 21.
- Randaugment: Practical automated data augmentation with a reduced search space, in: Proceedings Of The IEEE/CVF Conference On Computer Vision And Pattern Recognition Workshops, pp. 702–703.
- Towards a holistic view of bias in machine learning: bridging algorithmic fairness and imbalanced learning. Discover Data 2.
- Improved regularization of convolutional neural networks with cutout. ArXiv Preprint ArXiv:1708.04552 .
- Keepaugment: A simple information-preserving data augmentation approach, in: Proceedings Of The IEEE/CVF Conference On Computer Vision And Pattern Recognition, pp. 1055–1064.
- Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation, in: Proceedings Of The IEEE/CVF Winter Conference On Applications Of Computer Vision, pp. 1548–1558.
- Nvlabs/ffhq-dataset. URL: https://github.com/NVlabs/ffhq-dataset.
- Sql and nosql database software architecture performance analysis and assessments—a systematic literature review. Big Data and Cognitive Computing 7, 97.
- Introducing urdu digits dataset with demonstration of an efficient and robust noisy decoder-based pseudo example generator. Symmetry 14, 1976.
- Biaswap: Removing dataset bias with bias-tailored swapping augmentation, in: Proceedings Of The IEEE/CVF International Conference On Computer Vision, pp. 14992–15001.
- Navigating complexity: A tailored question-answering approach for pdfs in finance, bio-medicine, and science .
- Image data augmentation approaches: A comprehensive survey and future directions. IEEE Access .
- Keeporiginalaugment: Single image-based better information-preserving data augmentation approach, in: 20th International Conference on Artificial Intelligence Applications and Innovations.
- Keeporiginalaugment: Single image-based better information-preserving data augmentation approach. URL: https://api.semanticscholar.org/CorpusID:269741075.
- Advanced data augmentation approaches: A comprehensive survey and future directions. arXiv preprint arXiv:2301.02830 .
- Intra-class random erasing (icre) augmentation for audio classification, in: Proceedings Of The Korean Society Of Broadcast Engineers Conference, The Korean Institute of Broadcast and Media Engineers. pp. 244–247.
- Audrandaug: Random image augmentations for audio classification. arXiv preprint arXiv:2309.04762 .
- Forged character detection datasets: Passports, driving licences and visa stickers. International Journal of Artificial Intelligence & Applications .
- Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51, 181–207.
- Labeled faces in the wild: A survey, in: Advances In Face Detection And Facial Image Analysis, pp. 189–248.
- Learning debiased representation via disentangled feature augmentation, in: Advances In Neural Information Processing Systems, pp. 25123–25133.
- Dataset diversity: measuring and mitigating geographical bias in image search and retrieval, in: Proceedings Of The 1st International Workshop On Trustworthy AI For Multimedia Computing, pp. 19–25.
- Biased attention: Do vision transformers amplify gender bias more than convolutional neural networks? ArXiv Preprint ArXiv:2309.08760 .
- Multimodal composite association score: Measuring gender bias in generative multimodal models. ArXiv Preprint ArXiv:2304.13855 .
- Gender bias in multimodal models: A transnational feminist approach considering geographical region and culture. ArXiv Preprint ArXiv:2309.04997 .
- Human detection using a mobile platform and novel features derived from a visual saliency mechanism. Image And Vision Computing 28, 391–402.
- Search for optimal data augmentation policy for environmental sound classification with deep neural networks. Journal Of Broadcast Engineering 25, 854–860.
- Oxml challenge 2023: Carcinoma classification using data augmentation, in: IET Conference Proceedings CP887, The Institution of Engineering and Technology Stevenage, UK. pp. 303–306.
- Understanding eeg signals for subject-wise definition of armoni activities. arXiv preprint arXiv:2301.00948 .
- Me-ccnn: Multi-encoded images and a cascade convolutional neural network for breast tumor segmentation and recognition. Artificial Intelligence Review 56, 10099–10136.
- Dex: Deep expectation of apparent age from a single image, in: Proceedings Of The IEEE International Conference On Computer Vision Workshops, pp. 10–15.
- A computer vision-based object localization model for endangered wildlife detection. Ecological Economics, Forthcoming .
- Wildect-yolo: An efficient and robust computer vision-based accurate object localization model for automated endangered wildlife detection. Ecological Informatics 75, 101919.
- Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 .
- Deep learning-based cost-effective and responsive robot for autism treatment. Drones 7, 81.
- Efficient paddy grain quality assessment approach utilizing affordable sensors. Artificial Intelligence 5, 686–703.
- Hide-and-seek: A data augmentation technique for weakly-supervised localization and beyond. ArXiv Preprint ArXiv:1811.02545 .
- Understanding and mitigating bias in imaging artificial intelligence. RadioGraphics 44, e230067.
- Investigating multi-feature selection and ensembling for audio classification. arxiv 2022. arXiv preprint arXiv:2206.07511 .
- Investigating multi-feature selection and ensembling for audio classification. International Journal of Artificial Intelligence & Applications .
- Saliencymix: A saliency guided data augmentation strategy for better regularization. ArXiv Preprint ArXiv:2006.01791 .
- Cardiacnet: A neural networks based heartbeat classifications using ecg signals. Studies in Medical and Health Sciences 1, 1–17.
- Towards accuracy-fairness paradox: Adversarial example-based data augmentation for visual debiasing, in: Proceedings Of The 28th ACM International Conference On Multimedia, pp. 4346–4354.
- Age progression/regression by conditional adversarial autoencoder, in: Proceedings Of The IEEE Conference On Computer Vision And Pattern Recognition, pp. 5810–5818.
- Position: Measure dataset diversity, don’t just claim it. ArXiv Preprint ArXiv:2407.08188 .
- Random erasing data augmentation, in: Proceedings Of The AAAI Conference On Artificial Intelligence, pp. 13001–13008.
Collections
Sign up for free to add this paper to one or more collections.
