PersonalityScanner: MBTI Detection on Twitter

• PersonalityScanner is a computational framework that automatically detects MBTI personality types from Twitter data using advanced feature engineering and robust supervised classifiers.
• It integrates diverse linguistic, semantic, and profile metadata features, including tweet normalization, readability metrics, and embedding-based representations, to differentiate among Analysts, Diplomats, Sentinels, and Explorers.
• The system is trained on a large, curated dataset of over 56,000 users and achieves an F1 score of 45.7%, highlighting both its methodological advances and challenges such as label noise and sampling bias.

PersonalityScanner is a computational framework for automatic detection and analysis of MBTI ("Myers-Briggs Type Indicator") personality types from large-scale social media data, specifically Twitter. The system is designed to predict one of four MBTI functional classes (Analysts, Diplomats, Sentinels, Explorers) based on a user's public Twitter footprint, integrating multiple feature engineering strategies and robust supervised classifiers. The development of PersonalityScanner marks a significant advance in the scale and rigor of personality detection methodology, leveraging the largest automatically curated MBTI Twitter dataset to date (Datta et al., 2023).

1. Dataset Construction and Curation

PersonalityScanner is built on a dataset comprising 56,130 Twitter users with associated MBTI labels, totaling over 152 million tweets. MBTI type labels are harvested automatically using two distinct linkage patterns to 16personalities.com: profile-specific URLs and four-letter-type URLs embedded in users' tweets. After identifying candidate users via the Twitter API, Selenium is used to accurately scrape the MBTI label from the linked 16personalities profile.

Strict filtering steps ensure label accuracy and data quality:

• Only users with at least 100 English-language tweets (detected using fastText) are included.
• Users reporting conflicting MBTI types are excluded.
• The dataset covers all four MBTI "functional" classes, with each user's timeline capped at the most recent 3,200 tweets for consistency.

This protocol produces a highly structured, large-scale dataset, supporting comprehensive model training and evaluation.

2. Feature Engineering and Representation

Feature engineering in PersonalityScanner encompasses a broad spectrum of linguistic, semantic, and user-profile features, organized as follows:

A. Textual Processing

• All tweets are normalized using TweetBERT’s normalizer.
• Tweets are stripped of hashtags, URLs, and mentions, which are collected separately for further modeling.

B. Readability and Empath Features

• Eight readability metrics are computed on the cleaned tweet text, including Flesch, Flesch–Kincaid, Dale–Chall, Automated Readability Index (ARI), Coleman–Liau, Gunning Fog, Linsear-Write, and SPACHE.
• 194 Empath categories (word usage themes) are extracted using the Empath tool, providing high-dimensional semantic representation.

C. Embedding-Based Features

• Four alternatives for generating tweet and profile bio embeddings: fastText (700-dimensional average), BERT-base (CLS pooled), TweetBERT, and RoBERTa (all similarly pooled).
• Hashtag, URL, and mention embeddings are built at the user level as tf–idf vectors (excluding very rare terms) and projected to 64 dimensions via a 3-layer dense neural network trained to predict MBTI class; the network's penultimate layer serves as the embedding.

D. Profile Metadata

• Raw user-level counts (e.g., followers, friends, listing count, statuses, favorites) are included to capture social graph characteristics distinct for different MBTI types.

E. Feature Vector Structure

The final model input for each user concatenates:

• Tweet embedding
• Bio embedding
• 194D Empath vector
• 8D readability metrics
• 6D profile counts
• 64D hashtag, URL, and mention embeddings

3. Classification Models and Training Protocols

PersonalityScanner employs ensemble tree-based classifiers:

• Random Forest Classifier (RFC)
• XGBoost Classifier (XGB)

The objective function is standard multi-class cross-entropy over the four MBTI classes: $$\mathcal{L} = -\sum_{u=1}^M \sum_{c=1}^4 y_{u,c}\,\log\hat y_{u,c}$$ where $y_{u,c}$ is the true indicator label, and $\hat y_{u,c}$ is the predicted probability.

Training utilizes a class-balanced design (~4,000 users per class for training; ~1,000 per class for held-out test), default classifier hyperparameters (per scikit-learn and fastText), and standard data splits.

4. Performance Evaluation and Feature Importance

Evaluation metrics include per-class and micro-averaged accuracy and F1-score. The best overall configuration—fastText embedding, Random Forest, all features—yields an F1 of 45.7% and accuracy of 46.0% (4-way classification).

Ablation studies reveal:

Feature Group	F1 (w/ all)	F1 (removed)	ΔF1
Hashtag embeddings	45.7	43.5	–2.2
URL embeddings	45.7	45.1	–0.6
Mention embeddings	45.7	44.1	–1.6
Readability metrics	45.7	45.6	–0.1
Empath features	45.7	44.8	–0.9
Profile counts	45.7	45.3	–0.4

Hashtag-based embeddings provide the largest single feature gain outside tweet text itself, while readability metrics contribute the least as individual groups.

Empath categories and profile metadata exhibit meaningful MBTI-aligned patterns: analysts produce more complex tweets; explorers exhibit higher usage of terms like "dance," "music," and "appearance"; and professional terms in bios correlate with MBTI functional classes.

5. System Design, Deployment, and Challenges

The PersonalityScanner system is deployable as either a dashboard or API capable of returning MBTI predictions with associated confidence for any public Twitter user. The canonical pipeline is:

1. Ingestion: Pull user tweets and profile data via Twitter API.
2. Preprocessing: Normalize text, extract and segregate entities.
3. Feature Extraction: Compute embeddings, semantic features, and metadata statistics.
4. Inference: Apply RFC or XGB to produce predicted MBTI label.
5. Front-End: Render accessible visualizations or API endpoints.

Notable challenges:

• Label noise: Reliance on self-reported MBTI results introduces the risk of incorrect or non-representative labels.
• Sampling bias: The Twitter population is not a representative sample of the general population; demographic skew and social platform effects may influence patterns.
• Temporal drift: Users’ language and self-presentation may evolve, impacting model generalizability across time.
• Privacy and ethics: Data handling requires explicit attention to consent, anonymization, and compliance with data policies.

6. Analysis, Interpretation, and Recommendations

Qualitative and quantitative analyses confirm that multiple layers of extracted features correlate with known MBTI functional distinctions, validating both the feature engineering approach and the MBTI label assignment.

Ablation studies underscore the central importance of entity embeddings (especially hashtags), slight but consistent gains from semantic features (Empath), and minimal impact from readability metrics when all other features are present.

Potential pathways for future development include:

• Incorporating temporal modeling (e.g., LSTMs or timestamp-aware transformers) to capture the evolution of personality expression.
• Extending analysis to multimodal signals (e.g., images, videos, audio), providing a richer multimodal context for classification.
• Adding interpretability layers, such as feature-importance visualizations, for enhanced end-user transparency.
• Fine-tuning deep encoders end-to-end on the large-scale MBTI dataset now available.

7. Significance and Applications

PersonalityScanner offers state-of-the-art MBTI personality predictions from large-scale, real-world social media data and sets a new benchmark for both dataset size and methodological transparency in this domain. Its modularity and rigorous feature engineering make it suitable for extensions to additional platforms, personality assessment frameworks, and applications in fields such as candidate selection, marketing, and digital mental health screening (Datta et al., 2023).