LLMvsSmall Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model

Abstract: Personality detection aims to detect one's personality traits underlying in social media posts. One challenge of this task is the scarcity of ground-truth personality traits which are collected from self-report questionnaires. Most existing methods learn post features directly by fine-tuning the pre-trained LLMs under the supervision of limited personality labels. This leads to inferior quality of post features and consequently affects the performance. In addition, they treat personality traits as one-hot classification labels, overlooking the semantic information within them. In this paper, we propose a LLM based text augmentation enhanced personality detection model, which distills the LLM's knowledge to enhance the small model for personality detection, even when the LLM fails in this task. Specifically, we enable LLM to generate post analyses (augmentations) from the aspects of semantic, sentiment, and linguistic, which are critical for personality detection. By using contrastive learning to pull them together in the embedding space, the post encoder can better capture the psycho-linguistic information within the post representations, thus improving personality detection. Furthermore, we utilize the LLM to enrich the information of personality labels for enhancing the detection performance. Experimental results on the benchmark datasets demonstrate that our model outperforms the state-of-the-art methods on personality detection.

• The paper presents an innovative method that uses LLMs to generate supplementary text analyses, overcoming data scarcity in personality detection.
• The approach enriches personality labels with graded, detailed explanations, moving beyond traditional one-hot encoding for improved generalization.
• Performance evaluations demonstrate significant gains in macro-F1 scores on benchmark datasets, validating the effectiveness of LLM-driven augmentation.

LLM Based Text Augmentation Enhanced Personality Detection Model

Personality detection from social media posts is the task of inferring personality traits based on written content. This task faces significant challenges, primarily due to the scarcity of labeled data and the nuanced nature of personality traits. The paper presents an innovative approach by leveraging LLMs to enhance the personality detection capabilities of smaller models through text augmentation.

Introduction to Personality Detection Challenges

Personality detection seeks to identify individual personality traits, such as those defined by the Myers-Briggs Type Indicator (MBTI), from social media texts. Traditional methods have relied heavily on manually crafted features derived from psychological models like LIWC. However, these methods are constrained by the limited availability of self-reported personality data and the oversimplification introduced by treating personality traits as one-hot vectors, ignoring their semantic richness.

Current approaches attempt to utilize pre-trained LLMs fine-tuned on the available data, but these tend to generate suboptimal post representations. The proposed model addresses these limitations by using LLMs not as direct predictors, but as knowledge bases to generate insightful text augmentations, enriching both data and labels in the personality detection task.

LLM-Driven Text Augmentation

The model takes advantage of LLMs to generate analyses that supplement the original social media posts. These analyses focus on key aspects: semantic content, sentiment dynamics, and linguistic style, all of which are known to correlate with personality traits.

The LLM is prompted to produce additional commentary on each post, creating a richer, multi-dimensional dataset that provides more nuanced signals for personality detection. This augmented dataset is used during training to enhance the embedding space, enabling more robust psycho-linguistic profiling. Notably, the LLM-derived augmentations do not incur additional computational cost at inference time.

Figure 1: An example of personality detection using the LLM: The result provided by the LLM is ISFJ, while the actual ground truth is ENFP.

Enriching Personality Label Information

The paper also leverages LLMs to go beyond one-hot personality labels, which typically fail to capture the full complexity of personality dimensions. By generating detailed explanations for each of the MBTI traits from various psychological dimensions, the LLM provides enriched label information.

This approach results in a set of soft labels that offer a graded understanding of personality traits, addressing issues such as measurement error and improving model generalization. These enriched labels facilitate better alignment of user representations with personality traits during training.

Performance Evaluation

Experiments demonstrate that the proposed model, which the paper refers to as Text Augmentation Enhanced (TAE) personality detection, significantly outperforms baseline models on two benchmark datasets. The model's superior performance is attributed to the inclusion of LLM-generated semantic, sentiment, and linguistic text analyses, as well as the enriched personality labels.

Figure 2: An overview of our TAE.

Ablation studies confirm the importance of each component, showing that each type of augmentation contributes to the final performance. Utilizing LLMs for both data augmentation and label information results in substantial improvements in macro-F1 scores across dimensions of the personality spectrum.

Conclusion

The paper introduces a robust framework that capitalizes on LLM capabilities for enhancing personality trait detection from text. By generating knowledgeable augmentations and enriching label semantics, the approach bridges the gap between limited datasets and the complex nature of personality traits. The results underscore the value of integrating LLMs in data augmentation for personality profiling tasks, paving the way for future research to explore other psychological traits and improve model explainability.

The findings suggest numerous paths for further development, such as using larger LLMs or different LLMs, including domain-specific knowledge to improve text augmentations, and extending this framework to cover additional personality dimensions and psychological attributes.