A Replication Study: Machine Learning Models Are Capable of Predicting Sexual Orientation From Facial Images (1902.10739v1)

Abstract: Recent research used machine learning methods to predict a person's sexual orientation from their photograph (Wang and Kosinski, 2017). To verify this result, two of these models are replicated, one based on a deep neural network (DNN) and one on facial morphology (FM). Using a new dataset of 20,910 photographs from dating websites, the ability to predict sexual orientation is confirmed (DNN accuracy male 68%, female 77%, FM male 62%, female 72%). To investigate whether facial features such as brightness or predominant colours are predictive of sexual orientation, a new model based on highly blurred facial images was created. This model was also able to predict sexual orientation (male 63%, female 72%). The tested models are invariant to intentional changes to a subject's makeup, eyewear, facial hair and head pose (angle that the photograph is taken at). It is shown that the head pose is not correlated with sexual orientation. While demonstrating that dating profile images carry rich information about sexual orientation these results leave open the question of how much is determined by facial morphology and how much by differences in grooming, presentation and lifestyle. The advent of new technology that is able to detect sexual orientation in this way may have serious implications for the privacy and safety of gay men and women.

• The paper replicates ML models originally proposed by Wang and Kosinski to predict sexual orientation from facial images.
• It compares a Deep Neural Network, a facial morphology model, and a blurred image model, noting significant accuracy improvements with multiple images.
• The study underscores the models' robustness to superficial variations while highlighting ethical concerns regarding privacy and sensitive data use.

Analysis of Machine Learning Models for Predicting Sexual Orientation From Facial Images: Insights From a Replication Study

The paper "A Replication Study: Machine Learning Models Are Capable of Predicting Sexual Orientation From Facial Images," conducted by John Leuner, offers a comprehensive examination of ML models in discerning sexual orientation from facial images. This paper is motivated by the contentious findings of Wang and Kosinski, who demonstrated the effectiveness of deep learning in this domain.

Overview of Methodology

The paper comprehensively replicates two ML models originally presented by Wang and Kosinski: a Deep Neural Network (DNN) model and a Facial Morphology (FM)-based model. Additionally, the author introduces a novel model utilizing highly blurred facial images to explore the predictive power of general color characteristics. A dataset comprising 20,910 facial images sourced from dating websites serves as the foundation for validation, providing a diverse array of photographic data that excludes race and geographical constraints that were considered in the original paper.

Core Findings and Analysis

1. Model Performance:

• DNN Model: This model achieved a ROC-AUC of 0.68 for males and 0.77 for females with one facial image. Notably, accuracy improved with additional images per subject, reaching AUC scores of 0.78 for males and 0.88 for females with three images.
• Facial Morphology Model: Results indicated AUC scores of 0.62 for males and 0.72 for females for single images, with performance enhancing to AUCs of 0.68 for males and 0.81 for females with three images.
• Blurred Image Model: Despite significant data blurring, this model yielded AUC scores of 0.63 for males and 0.72 for females, underscoring the informativeness of predominant color features in facial images.

2. Robustness Checks:

• Altering subjects' presentation via superficial features like facial hair, eyewear, makeup, and head pose did not significantly affect model predictions, suggesting the models capture deeper underlying patterns instead of superficial variances alone.
• Further analyses confirmed the models' predictive capabilities were unaffected by the controlled presence of facial hair or eyewear, emphasizing the models' resilience to specific visual cues.

3. Head Pose:

• No correlation was found between head pose angles and sexual orientation, dismissing notions that these pose metrics could skew predictive outcomes.

Implications and Future Research Directions

This replication paper corroborates the predictive power of ML models in identifying sexual orientation from facial images, albeit with some variance in gender-specific accuracy compared to initial results by Wang and Kosinski. The paper raises significant concerns about privacy, as the capability to infer sensitive personal data from publicly available media could pose ethical and societal challenges. It highlights the necessity for ethical guidelines and policies in the deployment of such technologies.

Future research should further dissect whether these models primarily tap into biological facial traits, or if presentation and lifestyle factors drive predictive disparities. Particularly intriguing is the performance of the blurred image model; identifying precisely what information remains in low-resolution images that associates with sexual orientation could illuminate new directions for both feature research and broader implications of privacy risks associated with AI-driven insights. Further scrutiny of the nuanced role of head pose and potential biases in data collection methods (e.g., affectations of race, age, and geography) will be essential.

In sum, while this replication paper validates prior works and introduces novel elements to the discourse, it amplifies ethical concerns and calls for deeper investigation into the underlying factors that enable such predictions beyond the immediate technical scope.