Large-Scale Mobile Short-Video Datasets

• Large-scale mobile short-video datasets are structured corpora that capture diverse modalities and interaction logs, enabling detailed multimedia analysis.
• They incorporate multimodal feature extraction techniques such as CNN embeddings, ASR transcripts, and metadata for robust model evaluation.
• These datasets drive research in user behavior modeling, recommendation systems, cross-platform influence analysis, and AI agent training.

Large-scale mobile short-video datasets are structured corpora capturing diverse, multimodal, and temporally indexed short-form video content, interaction traces, device/user attributes, and often richly curated metadata. These resources underpin empirical research across computational social science, multimedia learning, user behavior modeling, and mobile AI agent development. The section below synthesizes key datasets, methodologies, and research implications from recently published datasets and benchmarks, with an emphasis on those providing open, scalable, and heterogeneous mobile video corpora.

1. Dataset Landscape: Scope, Scale, and Modalities

Large-scale mobile short-video datasets vary in scope: from interaction-centric logs to audio-visual content, cross-platform propagation, semantic annotation, and knowledge graph structures.

• Interaction-Centric Datasets: The dataset of (Shang et al., 9 Feb 2025) documents 10,000 voluntary users (age ≥20), capturing every interaction with 153,561 videos (first-week logs: 1,019,568 records) from a real mobile short-video platform. Modality coverage includes interactions (implicit—view, explicit—like, comment, follow, forward, collect, hate), user attributes (demographics, geography, device model/price), and comprehensive video content (ResNet+ViT 8×256D embeddings per video, bilingual ASR transcripts via SenseVoice-Small and LLaMA3-8B translation).
• Propagation and Influence Graphs: XS-Video (Xue et al., 31 Mar 2025) targets cross-platform propagation with 117,720 unique videos and 381,926 temporal samples across five leading Chinese platforms. Content is annotated with interaction metrics (views, likes, shares, collects, comments, fans), platform/topic/author metadata, and propagation influence ratings ($y\in\{0\dots9\}$) derived from aligned, cumulative metrics.
• GUI/Agent Task Datasets: MONDAY (Jang et al., 19 May 2025) introduces 313,000 annotated frames from 20,320 YouTube instructional mobile navigation videos (iOS/Android), annotated for scene, UI layout, and multi-step action sequences, with automated pipeline support.
• Semantic and Affect-Driven Datasets: eMotions (Wu et al., 2023) provides 27,996 "hot" short videos from Douyin, Kuaishou, and TikTok (2019–2023), categorically labeled for emotion (six-way Plutchik taxonomy).
• Multilingual/Multimodal Collections: 3MASSIV (Gupta et al., 2022) provides 50K annotated, 100K unlabelled short videos (mean 20s, 11 languages), labeled for concept themes (34), affective states (11), audio/video type, and language.
• Knowledge-Structured Repositories: Kuaipedia (Pan et al., 2022) systematizes over 200M short videos as a massive multi-modal knowledge graph linking 26M items, 2.5M aspects, and millions of item–aspect–video triplets mined from Kuaishou uploads.
• Clip Repurposing Benchmarks: Repurpose-10K (Wu et al., 2024) encompasses 11,210 long-form YouTube videos and 120,925 repurposed short clips, targeting the long-to-short user-edited transformation benchmark.

Notably, most datasets include split-by-user or chronological stratification for robust train/val/test benchmarking, and several provide open licenses with public download (e.g., (Shang et al., 9 Feb 2025, Wu et al., 2024)).

2. Schema Design and Annotation Protocols

Data schemas are multilevel, typically organized around the following core files and modalities:

Dataset	Users	Videos	Interactions / Tasks	Content Modalities	Annotation Protocol
(Shang et al., 9 Feb 2025)	10,000	153,561	{view, like, comment...}	Visual frames, ASR (Chinese+English)	Volunteer logs + proxy, k-means + manual
(Xue et al., 31 Mar 2025)	N/A	117,720	views, likes, shares, etc.	Video, title, description, comments	Crawling, cross-platform temporal sampling
(Jang et al., 19 May 2025)	N/A	20,320 vid	navigation, UI actions	RGB screens, narration transcripts	OCR/ML-based pipeline, LLM-guided actions
(Pan et al., 2022)	N/A	>200M	item-aspect-video triplets	Caption, OCR/ASR, cover image	BERT+ResNet multi-modal classification
(Wu et al., 2024)	N/A	11,210	user repurposed clip points	Video, audio, caption	SaaS UGC annotation, LLM segmenting

Annotation protocols employ a mix of machine-human hybrid pipelines: e.g., (Shang et al., 9 Feb 2025) leverages mitmproxy for log capture, while (Jang et al., 19 May 2025) utilizes LLM/ML models for action extraction, and (Wu et al., 2024) incorporates user curation and timestamp refinement for precise repurposed clip boundaries. In (Wu et al., 2023), semi-expert, multi-stage emotion labeling and expert re-review are used to moderate subjectivity.

3. Multimodal Feature Extraction and Representation

Rich multimodal feature representations enhance benchmark utility and model expressivity:

• Visual Features: Most datasets extract frame-level CNN (ResNet, ViT, Swin-Transformer, CLIP) embeddings, with temporal aggregation (8 segments per video in (Shang et al., 9 Feb 2025), AvgPool over ViT in (Xue et al., 31 Mar 2025), CLIP ViT-B/32 in (Wu et al., 2024)). Fine-grained features (bounding boxes, UI overlays) are provided in GUI-focused datasets like MONDAY (Jang et al., 19 May 2025).
• Audio/ASR: Automated speech recognition (ASR) is widely used; (Shang et al., 9 Feb 2025) employs SenseVoice-Small and LLaMA3-8B for bilingual output. (Wu et al., 2024) applies WhisperX for time-aligned captioning.
• Structured Metadata: Datasets generally capture categorical and continuous metadata (user demographics, city tier, device specifics in (Shang et al., 9 Feb 2025); topic, platform, author_id, comments in (Xue et al., 31 Mar 2025); task name, action index in (Jang et al., 19 May 2025)).
• Graph-Based Representations: XS-Video (Xue et al., 31 Mar 2025) constructs a heterogeneous graph $G = (V,E,Y)$ with typed nodes for videos, topics, attributes, and interactions (≈5.5M nodes, 1.67B edges). Kuaipedia (Pan et al., 2022) builds hybrid item–aspect–video trees.

4. Technical Validation and Benchmarking Workflows

Datasets are validated by quantitative benchmarks, cluster analysis, coverage analysis, and model-based evaluation:

• Behavioral, Attribute, and Content Validation: (Shang et al., 9 Feb 2025) performs fourfold validation for coverage, content clustering (t-SNE of features by category), recommendation benchmarking (e.g., BM3 achieves Recall@10=0.0238), and filter-bubble analysis via per-user category coverage measures.
• Propagation and Influence Scoring: XS-Video (Xue et al., 31 Mar 2025) assigns videos influence levels ($y\in\{0,1,\dots,9\}$) based on cross-platform indicator alignment using MSPE-minimization; node-specific features are aggregated via RGCN for propagation prediction, yielding long-tailed influence distributions and enabling evaluation of LLM-graph hybrids.
• Scene/UI/Action Extraction Reliability: MONDAY (Jang et al., 19 May 2025) reports OCR-based scene transition (F1=95.04%), UI element detection (99.87% hit ratio), and step-wise action identification accuracy; agent models pretrained on MONDAY achieve +18.11pp average generalization lift on unseen mobile OS.
• Emotion Analysis Pipeline: eMotions (Wu et al., 2023) establishes multi-stage annotator agreement (Fleiss’ κ>0.45), reports per-task results (Audio-Visual classification: Acc=67.08%, WA-F1=66.45%).
• Clip Repurposing Model Benchmarks: Repurpose-10K (Wu et al., 2024) defines a joint classification/regression task and evaluates models by mAP at tIoU (0.5–0.9). The multimodal baseline outperforms video-only and audio-video baselines (mAP=11.57).

5. Use Cases and Research Applications

These datasets advance multiple domains:

• User Modeling: Fine-grained behavioral traces (Shang et al., 9 Feb 2025, Gupta et al., 2022) support investigation of engagement, addiction, and filter-bubble dynamics.
• Recommendation and Retrieval: Explicit and implicit feedback matrices, multimodal metadata, and propagation features enable robust training and evaluation of recommender models (user–item, content-based, graph-based).
• Propagation and Influence Analysis: XS-Video (Xue et al., 31 Mar 2025) and related graph benchmarks facilitate viral trend detection, influence scoring, and cross-platform comparison.
• Emotion and Semantics: eMotions (Wu et al., 2023), 3MASSIV (Gupta et al., 2022), and MV-58k (Nguyen et al., 2016) support emotion recognition, semantic concept classification, and open-world modeling in time-evolving contexts.
• Multimodal Knowledge Modeling: Kuaipedia (Pan et al., 2022) enables entity typing/linking, multi-modal VQA, and serves as retrieval-augmentation resource for LLMs.
• GUI Agent Training: MONDAY (Jang et al., 19 May 2025) uniquely supports training/evaluation of visual GUI agents, measuring cross-OS generalization.
• UGC Repurposing: Repurpose-10K (Wu et al., 2024) provides a benchmark for transforming long-form UGC into short clips, a critical but underexplored problem.

6. Challenges, Limitations, and Future Directions

Several challenges persist:

• Annotation Subjectivity: Emotion and semantic class labeling (Wu et al., 2023) require careful multi-rater curation and subjectivity mitigation; moderate agreement may constrain downstream modeling depth.
• Long-Tail/Skew Bias: Most datasets display heavy-tailed distributions in both video popularity (user–video interaction sparsity in (Shang et al., 9 Feb 2025), influence level distribution in (Xue et al., 31 Mar 2025)) and label space (concepts, emotions, topics).
• Temporal and Contextual Drift: Both open-world micro-video (MV-58k (Nguyen et al., 2016)) and cross-lingual corpora (Gupta et al., 2022) exhibit fast-changing topic/appearance drift, necessitating adaptive or causal modeling for robust generalization.
• Cross-domain Limits: Cross-platform generalization remains nontrivial; agent pretraining on mono-domain benchmarks delivers subpar performance on novel OS/UI distributions, motivating datasets like MONDAY (Jang et al., 19 May 2025).
• Automated vs Human Annotation: While pipeline automation scales annotation (MONDAY; Repurpose-10K), select tasks require nuanced human curation (emotion annotation, aspect mining, timestamp refinement).
• Access and License Constraints: Not all large datasets provide raw video or full annotation due to copyright or platform policy (e.g., MV-58k access is API-based only (Nguyen et al., 2016)).

Ongoing efforts include integration of foundation models for richer feature representations, weakly- or semi-supervised annotation to scale globally, and toolchains for generative repurposing and multi-language coverage.

7. Representative Dataset Summaries

Dataset	Key Features	Open Access
(Shang et al., 9 Feb 2025)	Full user–video logs, content features	Public/CC
(Xue et al., 31 Mar 2025)	Cross-platform, graph, propagation scoring	Planned/CC
(Jang et al., 19 May 2025)	GUI action annotation, fully automated	Planned
(Wu et al., 2023)	Expert emotion, multi-stage quality control	Planned
(Pan et al., 2022)	200M knowledge videos, multi-modal linking	Planned
(Wu et al., 2024)	Repurposing edits, UGC with dense labeling	Public/CC BY-NC
(Gupta et al., 2022)	11 languages, 6+ annotation axes	Not specified
(Nguyen et al., 2016)	260K Vine videos, temporal open world	API-based

These resources constitute the modern landscape for empirical research in large-scale mobile short-video analysis, supporting advances in user modeling, video understanding, multiscale propagation, knowledge retrieval, and AI agent training (Shang et al., 9 Feb 2025, Xue et al., 31 Mar 2025, Jang et al., 19 May 2025, Wu et al., 2023, Pan et al., 2022, Gupta et al., 2022, Wu et al., 2024, Nguyen et al., 2016).