Can Transformers Smell Like Humans? (2411.03038v1)

Abstract: The human brain encodes stimuli from the environment into representations that form a sensory perception of the world. Despite recent advances in understanding visual and auditory perception, olfactory perception remains an under-explored topic in the machine learning community due to the lack of large-scale datasets annotated with labels of human olfactory perception. In this work, we ask the question of whether pre-trained transformer models of chemical structures encode representations that are aligned with human olfactory perception, i.e., can transformers smell like humans? We demonstrate that representations encoded from transformers pre-trained on general chemical structures are highly aligned with human olfactory perception. We use multiple datasets and different types of perceptual representations to show that the representations encoded by transformer models are able to predict: (i) labels associated with odorants provided by experts; (ii) continuous ratings provided by human participants with respect to pre-defined descriptors; and (iii) similarity ratings between odorants provided by human participants. Finally, we evaluate the extent to which this alignment is associated with physicochemical features of odorants known to be relevant for olfactory decoding.

• The paper demonstrates that MoLFormer, a pre-trained transformer model, effectively aligns with human olfactory perception through unsupervised prediction tasks.
• It employs diverse datasets to assess label, continuous rating, and similarity score predictions, revealing that deeper layers capture higher-level olfactory features.
• The research provides promising implications for fragrance development and cross-disciplinary studies, bridging computational chemistry and neuroscience.

Evaluating the Alignment of Transformer-Based Models with Human Olfactory Perception

The paper entitled "Can Transformers Smell Like Humans?" investigates the extent to which representations of odorant chemical structures extracted from pre-trained transformer models align with human olfactory perception. This paper is particularly relevant given the under-explored nature of olfactory perception in the machine learning community, primarily due to the lack of large-scale labeled datasets that capture the complexity of human olfactory experience.

Methodological Overview

The work focuses on the MoLFormer model, a transformer-based architecture pre-trained on general chemical structures, and evaluates its capacity to predict human olfactory perception without explicit supervision for this task. The authors employed several datasets, notably Leffingwell-Goodscent (GS-LF), Sagar, Keller, Ravia, and Snitz, to assess the representational alignment of MoLFormer against three tasks:

1. Label Prediction: The model's ability to predict expert-assigned labels for various odorants.
2. Continuous Rating Prediction: The prediction of continuous perceptual ratings provided by human participants.
3. Similarity Score Prediction: The model's capability in predicting similarity ratings between odorants, derived from human responses.

These tasks were complemented by a representational similarity analysis, aiming to directly compare the encoded similarities with human judgments.

Key Findings

The empirical results demonstrate that MoLFormer successfully aligns with human olfactory perception in several key areas:

• Label Prediction: MoLFormer surpassed the performance of the Distance Angle Model (DAM) in predicting expert-assigned labels, although it fell short of the Open-POM model, which benefited from direct supervision with the same data.
• Similarity Scores: The model showed a relatively high Pearson correlation in predicting similarity ratings between odorants (R = 0.64 for Snitz and R = 0.66 for Ravia), indicating its unsupervised odorant representations could infer the perceived similarity between odorants effectively.
• Continuous Ratings: MoLFormer displayed a competitive performance in predicting continuous ratings, albeit slightly underperforming compared to Open-POM. Its performance was dataset dependent, as illustrated by its varying success across Sagar and Keller datasets.

A crucial observation was that the alignment of physicochemical descriptors and perception improved at deeper layers of the transformer model. This suggests a hierarchical learning structure, with deeper layers encoding higher-level perceptual features, analogous to vision models.

Theoretical and Practical Implications

The findings imply that transformers, though not directly trained for olfactory tasks, capture underlying structures in chemical data that may be related to human perception. This could lead to the development of novel tools, allowing chemists and neuroscientists to better understand the molecular basis of olfactory perception. Additionally, such models may enable new applications in fragrance development, flavor design, and the synthesis of novel compounds with specific sensory properties, while reducing dependency on expert annotations.

Future Directions

Future work may focus on integrating intensity and concentration data to enhance representation accuracy. Expanding the model's application to encompass other sensory modalities, cross-modal architectures, or additional self-supervised learning frameworks would also be of interest. Moreover, exploring the neuroscientific basis of MoLFormer’s alignment with fMRI data from olfactory brain regions could elucidate the underlying mechanisms governing olfactory perception.

Closing Remarks

This research bridges the gaps between chemistry, machine learning, and neuroscience by showing how pre-trained transformer models can derive meaningful, perceptual information from chemical structures. While further studies are needed to enhance understanding and application, this work provides a robust foundation for future explorations into computational olfactory perception.