Longitudinal Workplace Emotion Dataset

• Longitudinal workplace emotion dataset is a comprehensive multimodal collection recording affective states, physiological signals, and behavioral indicators in real-world work settings.
• The dataset enables robust analysis of emotional trajectories and predictive modeling of outcomes like turnover and stress using advanced machine learning techniques.
• Privacy-preserving measures, continuous sampling, and rich contextual metadata make this resource vital for emotion recognition research and digital phenotyping applications.

The longitudinal workplace emotion dataset encapsulates comprehensive, multimodal records of affective states, physiological signals, and behavioral indicators gathered in authentic work environments over extended periods. These resources are foundational to the rigorous paper of emotion recognition, affective dynamics, and emotional responses to both day-to-day activities and major societal events within workplace contexts. Longitudinal datasets are distinguished by continuous or repeated sampling, multimodal integration (e.g., facial expressions, wearable sensor data, audio, surveys), and contextual metadata (such as job roles or event markers). Their scale and naturalistic collection protocols enable robust modeling of emotional trajectories, interpersonal contagion, turnover prediction, and the design of affect-aware systems for real-world organizations.

1. Methodologies for Data Collection

Longitudinal workplace emotion datasets employ diverse methods for acquiring affective and contextual data streams in situ. The TILES-2018 dataset (Mundnich et al., 2020) studied 212 hospital workers over a 10-week interval, utilizing a suite of off-the-shelf wearable sensors (Fitbit Charge 2 for PPG-based heart rate, sleep, and step count; OMsignal smart garment for ECG and breathing during work shifts; Unihertz Jelly Pro smartphone worn as an audio badge), coupled with stationary environmental sensors (Owl-in-One Bluetooth hubs, Minew sensors for proximity, light, temperature, humidity, and door movement). Ecological Momentary Assessments (EMAs) and comprehensive surveys captured subjective states such as job performance, affect, stress, and psychological capital at varying frequencies. Data collection protocols enforced constant or shift-based sensor usage, while environmental data persisted 24/7.

WELD (Sun, 17 Oct 2025) acquired 733,651 facial expression records over 30.5 months (Nov. 2021–May 2024) from 38 employees in a Chinese software company, leveraging ceiling-mounted RGB security cameras (1920×1080, 25 fps) and a deep learning API (ResNet-50 backbone with attention, Asian facial tuning) to estimate probabilities for seven basic emotions from detected faces at 10-second intervals during work hours. No specialized hardware was required beyond the company’s existing security system.

"Inferring User Facial Affect in Work-like Settings" (Ilyas et al., 2021) devised controlled laboratory protocols simulating work-like settings—office (N-back cognitive task), assembly-line (Operations Game), and teleworking (Webcall)—with manipulable difficulty and stressors. Twelve subjects contributed multimodal data (facial video via multiple cameras, audio via Jabra microphones, physiological signals from Empatica/Muse, self-report affect via SAM questionnaires), yielding granular insights into emotional responses under baseline, easy, hard, and stress-enhanced conditions.

2. Structure and Content of Datasets

These datasets are characterized by meticulous multimodal organization and extensive metadata. TILES-2018 data are structured into sensor-specific folders: Fitbit (daily summary, minute-level time series, sleep staging), OMsignal (features, raw ECG at 250 Hz and 15-second windows), Owl-in-One Bluetooth RSSI for proximity, Minew environmental readings, and survey data (raw item responses, scored scales, demographic bins). Audio features (extracted using openSMILE, challenge feature sets like ComParE/emobase) are bundled with foreground speech prediction files (NumPy .npy).

WELD encompasses seven emotion probabilities per record (neutral, happy, sad, surprised, fear, disgusted, angry), extensive metadata (job role, employment outcomes, personality), and 32 advanced emotional metrics such as valence, arousal, volatility, inertia, predictability, and emotional contagion strength computed using affective science methods. Critical formulas include:

• Valence: $$v(t) = p_\text{happy}(t) - p_\text{sad}(t) - 0.5\,p_\text{angry}(t) - 0.5\,p_\text{fear}(t)$$
• Arousal: $$a(t) = p_\text{surprised}(t) + p_\text{fear}(t) + p_\text{angry}(t) - p_\text{sad}(t)$$

"Inferring User Facial Affect in Work-like Settings" integrates synchronized video, audio, physiological streams, and condition-coded affect labels with valence/arousal mapped to SAM scores ($x < 5$: negative; $x = 5$: neutral; $x > 5$: positive).

Dataset	Modalities	Scale / Duration	Core Features
TILES-2018	Physio (Fitbit, OMsignal), audio, EMA, env	212 workers, 10 weeks	Multimodal sensor + subjective surveys
WELD	Facial expresssion, emotion metrics, meta	38 employees, 30.5 months	733,651 records, 32 emotional metrics
WECARE paper	Video, audio, physio, questionnaires	12 subjects, multi-task	Lab-based, work-like affect annotation

3. Analytical and Modeling Techniques

State-of-the-art analysis on these datasets leverages both statistical and machine learning approaches. Survey reliability is vetted via Cronbach’s alpha (often >0.75 in TILES-2018), while temporal affective metrics in WELD (volatility: rolling SD; inertia: lag-1 autocorrelation; predictability: AR($R^2$)) capture long-term emotional stability and fluctuation.

Emotion recognition utilizes deep neural architectures: WELD’s recognition API is based on ResNet-50 with attention, and (Ilyas et al., 2021) employs a pre-trained AffectNet ResNet-18 as baseline, fine-tuned (F-Res) on work-contextual data, and extended to segment-level modeling (S-Res) exploiting spectral representations for frame-sequence aggregation. Segment-level features demonstrably outperform frame-level prediction, increasing RMSE, CCC, PCC in arousal estimation.

Baseline classification experiments (WELD) compared Random Forest (89.3% accuracy), SVM (85.7%), and LSTM (91.2%), showing the benefit of temporal context from sequential emotion records. Valence regression using LSTM achieved $R^2 = 0.84$ (RMSE = 0.149). Turnover prediction, using Random Forests on WELD’s emotional metrics, yielded an AUC = 1.0, establishing perfect predictive validity.

4. Applications and Key Findings

These datasets support a diverse array of research applications:

• Temporal affect modeling: WELD substantiates the weekend effect (+192% valence, $p<0.001$) and diurnal rhythm in emotion, and quantifies responses to societal events (e.g., Shanghai lockdown $\downarrow$11.9% valence; reopening $\uparrow$60.0% valence).
• Predictive analytics: Robust turnover prediction (WELD, AUC=1.0) leverages emotional volatility, high fear/anger ratios, and extended metrics as precursors of disengagement.
• Digital phenotyping: TILES-2018’s multimodal sensor data enables unobtrusive tracking of stress, psychological flexibility, and day-to-day changes within hospital staff, while indoor localization and proximity patterns illuminate social interactions.
• Emotion-aware system design: Findings motivate adaptive interfaces (real-time emotional state adjustment), early warning systems for attrition, and personalized well-being interventions.
• Contextual affect analysis: (Ilyas et al., 2021) demonstrates facial affect varies significantly between work-like and non-working settings, necessitating dataset collection within ecologically valid task environments and temporal modeling.

5. Privacy, Security, and Ethical Considerations

Comprehensive privacy and ethical strategies govern dataset use:

• De-identification and data stewardship: TILES-2018 enforces hash-based participant IDs, demographic binning, and text anonymization. Audio is restricted to features/predictions; raw signals and reconstructive filterbanks (MFCC, PLP) are not released in vulnerable forms.
• Consent and oversight: Institutional Review Board (IRB) protocols and informed consent underpin TILES-2018, with data access contingent on Data Usage Agreements forbidding re-identification or sensitive content extraction. WELD similarly controls for participant privacy in facial expression and personality metadata.
• Responsible use: Studies explicitly prohibit any re-identification attempts and prioritize privacy-preserving machine learning, suggesting this as an avenue for further methodological innovation.

6. Future Directions and Research Opportunities

The trajectory of longitudinal workplace emotion research centers on scaling up, broadening context, and refining analytical tools:

• Expansion and diversity: (Ilyas et al., 2021) targets larger multi-site datasets for generalizable facial affect modeling in diverse European work settings.
• Multimodal fusion: The aggregation of physiological, facial, audio, and contextual data (TILES-2018, WECARE) supports richer affect inference.
• Longitudinal tracking: Investigating inertia, volatility, and adaptive changes in affect over months to years (WELD) informs organizational climate, resilience, and recovery.
• Privacy-aware machine learning: Further algorithmic work is needed to balance predictive capacity with privacy, leveraging federated learning or differential privacy within multimodal contexts.
• Hypothesis testing and psychometric refinement: These datasets enable longitudinal tests regarding psychological flexibility, mood-performance linkage, and social contagion in workplaces, as well as new approaches to handling missing or artifacted time series.

A plausible implication is that continuous and fine-grained emotion monitoring, adequately anonymized, can act as a leading indicator for workforce well-being and institutional resilience, provided privacy concerns are systematically addressed.

7. Significance and Outlook

Longitudinal workplace emotion datasets represent a definitive advance in affective computing, organizational psychology, and digital health. They afford unprecedented scale, temporal depth, and context for modeling emotional dynamics under authentic workplace conditions. By documenting both routine and event-driven affective fluctuations, validating technical procedures, and connecting emotional states to practical outcomes (e.g., turnover, teamwork, adaptation), these resources underpin innovative research and application in emotion recognition, behavioral science, and adaptive system design. Continued methodological progress, larger and more diverse cohorts, and privacy-preserving frameworks are anticipated to further unlock their potential for academic and applied domains.