--- title: Large-Scale Mobile Short-Video Datasets url: https://www.emergentmind.com/topics/large-scale-mobile-short-video-datasets type: topic --- # Large-Scale Mobile Short-Video Datasets Large-scale mobile short-video datasets are structured corpora capturing diverse, multimodal, and temporally indexed short-form video content, interaction traces, device/user attributes, and often richly curated metadata. These resources underpin empirical research across computational social science, multimedia learning, user behavior modeling, and mobile AI agent development. The section below synthesizes key datasets, methodologies, and research implications from recently published datasets and benchmarks, with an emphasis on those providing open, scalable, and heterogeneous mobile video corpora. ## 1. Dataset Landscape: Scope, Scale, and Modalities Large-scale mobile short-video datasets vary in scope: from interaction-centric logs to audio-visual content, cross-platform propagation, semantic annotation, and knowledge graph structures. - **Interaction-Centric Datasets:** The dataset of [2502.05922] documents 10,000 voluntary users (age ≥20), capturing every interaction with 153,561 videos (first-week logs: 1,019,568 records) from a real mobile short-video platform. Modality coverage includes interactions (implicit—view, explicit—like, comment, follow, forward, collect, hate), user attributes (demographics, geography, device model/price), and comprehensive video content (ResNet+ViT 8×256D embeddings per video, bilingual ASR transcripts via SenseVoice-Small and LLaMA3-8B translation). - **Propagation and Influence Graphs:** XS-Video [2503.23746] targets cross-platform propagation with 117,720 unique videos and 381,926 temporal samples across five leading Chinese platforms. Content is annotated with interaction metrics (views, likes, shares, collects, comments, fans), platform/topic/author metadata, and propagation influence ratings ($y\in\{0\dots9\}$) derived from aligned, cumulative metrics. - **GUI/Agent Task Datasets:** MONDAY [2505.12632] introduces 313,000 annotated frames from 20,320 YouTube instructional mobile navigation videos (iOS/Android), annotated for scene, UI layout, and multi-step action sequences, with automated pipeline support. - **Semantic and Affect-Driven Datasets:** eMotions [2311.17335] provides 27,996 "hot" short videos from Douyin, Kuaishou, and TikTok (2019–2023), categorically labeled for emotion (six-way Plutchik taxonomy). - **Multilingual/Multimodal Collections:** 3MASSIV [2203.14456] provides 50K annotated, 100K unlabelled short videos (mean 20s, 11 languages), labeled for concept themes (34), affective states (11), audio/video type, and language. - **Knowledge-Structured Repositories:** Kuaipedia [2211.00732] systematizes over 200M short videos as a massive multi-modal knowledge graph linking 26M items, 2.5M aspects, and millions of item–aspect–video triplets mined from Kuaishou uploads. - **Clip Repurposing Benchmarks:** Repurpose-10K [2412.08879] encompasses 11,210 long-form YouTube videos and 120,925 repurposed short clips, targeting the long-to-short user-edited transformation benchmark. Notably, most datasets include split-by-user or chronological stratification for robust train/val/test benchmarking, and several provide open licenses with public download (e.g., [2502.05922], [2412.08879]). ## 2. Schema Design and Annotation Protocols Data schemas are multilevel, typically organized around the following core files and modalities: | Dataset | Users | Videos | Interactions / Tasks | Content Modalities | Annotation Protocol | |-----------------|----------------|-------------|------------------------------|---------------------------------------|--------------------------------------------| | [2502.05922] | 10,000 | 153,561 | {view, like, comment...} | Visual frames, ASR (Chinese+English) | Volunteer logs + proxy, k-means + manual | | [2503.23746] | N/A | 117,720 | views, likes, shares, etc. | Video, title, description, comments | Crawling, cross-platform temporal sampling | | [2505.12632] | N/A | 20,320 vid | navigation, UI actions | RGB screens, narration transcripts | OCR/ML-based pipeline, LLM-guided actions | | [2211.00732] | N/A | >200M | item-aspect-video triplets | Caption, OCR/ASR, cover image | BERT+ResNet multi-modal classification | | [2412.08879] | N/A | 11,210 | user repurposed clip points | Video, audio, caption | SaaS UGC annotation, LLM segmenting | Annotation protocols employ a mix of machine-human hybrid pipelines: e.g., [2502.05922] leverages mitmproxy for log capture, while [2505.12632] utilizes LLM/ML models for action extraction, and [2412.08879] incorporates user curation and timestamp refinement for precise repurposed clip boundaries. In [2311.17335], semi-expert, multi-stage emotion labeling and expert re-review are used to moderate subjectivity. ## 3. Multimodal Feature Extraction and Representation Rich multimodal feature representations enhance benchmark utility and model expressivity: - **Visual Features:** Most datasets extract frame-level CNN (ResNet, ViT, Swin-Transformer, CLIP) embeddings, with temporal aggregation (8 segments per video in [2502.05922], AvgPool over ViT in [2503.23746], CLIP ViT-B/32 in [2412.08879]). Fine-grained features (bounding boxes, UI overlays) are provided in GUI-focused datasets like MONDAY [2505.12632]. - **Audio/ASR:** Automated speech recognition (ASR) is widely used; [2502.05922] employs SenseVoice-Small and LLaMA3-8B for bilingual output. [2412.08879] applies WhisperX for time-aligned captioning. - **Structured Metadata:** Datasets generally capture categorical and continuous metadata (user demographics, city tier, device specifics in [2502.05922]; topic, platform, author_id, comments in [2503.23746]; task name, action index in [2505.12632]). - **Graph-Based Representations:** XS-Video [2503.23746] constructs a heterogeneous graph $G = (V,E,Y)$ with typed nodes for videos, topics, attributes, and interactions (≈5.5M nodes, 1.67B edges). Kuaipedia [2211.00732] builds hybrid item–aspect–video trees. ## 4. Technical Validation and Benchmarking Workflows Datasets are validated by quantitative benchmarks, cluster analysis, coverage analysis, and model-based evaluation: - **Behavioral, Attribute, and Content Validation:** [2502.05922] performs fourfold validation for coverage, content clustering (t-SNE of features by category), recommendation benchmarking (e.g., BM3 achieves Recall@10=0.0238), and filter-bubble analysis via per-user category coverage measures. - **Propagation and Influence Scoring:** XS-Video [2503.23746] assigns videos influence levels ($y\in\{0,1,\dots,9\}$) based on cross-platform indicator alignment using MSPE-minimization; node-specific features are aggregated via RGCN for propagation prediction, yielding long-tailed influence distributions and enabling evaluation of LLM-graph hybrids. - **Scene/UI/Action Extraction Reliability:** MONDAY [2505.12632] reports OCR-based scene transition (F1=95.04%), UI element detection (99.87% hit ratio), and step-wise action identification accuracy; agent models pretrained on MONDAY achieve +18.11pp average generalization lift on unseen mobile OS. - **Emotion Analysis Pipeline:** eMotions [2311.17335] establishes multi-stage annotator agreement (Fleiss’ κ>0.45), reports per-task results (Audio-Visual classification: Acc=67.08%, WA-F1=66.45%). - **Clip Repurposing Model Benchmarks:** Repurpose-10K [2412.08879] defines a joint classification/regression task and evaluates models by mAP at tIoU (0.5–0.9). The multimodal baseline outperforms video-only and audio-video baselines (mAP=11.57). ## 5. Use Cases and Research Applications These datasets advance multiple domains: - **User Modeling:** Fine-grained behavioral traces ([2502.05922], [2203.14456]) support investigation of engagement, addiction, and filter-bubble dynamics. - **Recommendation and Retrieval:** Explicit and implicit feedback matrices, multimodal metadata, and propagation features enable robust training and evaluation of recommender models (user–item, content-based, graph-based). - **Propagation and Influence Analysis:** XS-Video [2503.23746] and related graph benchmarks facilitate viral trend detection, influence scoring, and cross-platform comparison. - **Emotion and Semantics:** eMotions [2311.17335], 3MASSIV [2203.14456], and MV-58k [1603.09439] support emotion recognition, semantic concept classification, and open-world modeling in time-evolving contexts. - **Multimodal Knowledge Modeling:** Kuaipedia [2211.00732] enables entity typing/linking, multi-modal VQA, and serves as retrieval-augmentation resource for LLMs. - **GUI Agent Training:** MONDAY [2505.12632] uniquely supports training/evaluation of visual GUI agents, measuring cross-OS generalization. - **UGC Repurposing:** Repurpose-10K [2412.08879] provides a benchmark for transforming long-form UGC into short clips, a critical but underexplored problem. ## 6. Challenges, Limitations, and Future Directions Several challenges persist: - **Annotation Subjectivity:** Emotion and semantic class labeling ([2311.17335]) require careful multi-rater curation and subjectivity mitigation; moderate agreement may constrain downstream modeling depth. - **Long-Tail/Skew Bias:** Most datasets display heavy-tailed distributions in both video popularity (user–video interaction sparsity in [2502.05922], influence level distribution in [2503.23746]) and label space (concepts, emotions, topics). - **Temporal and Contextual Drift:** Both open-world micro-video (MV-58k [1603.09439]) and cross-lingual corpora ([2203.14456]) exhibit fast-changing topic/appearance drift, necessitating adaptive or causal modeling for robust generalization. - **Cross-domain Limits:** Cross-platform generalization remains nontrivial; agent pretraining on mono-domain benchmarks delivers subpar performance on novel OS/UI distributions, motivating datasets like MONDAY [2505.12632]. - **Automated vs Human Annotation:** While pipeline automation scales annotation (MONDAY; Repurpose-10K), select tasks require nuanced human curation (emotion annotation, aspect mining, timestamp refinement). - **Access and License Constraints:** Not all large datasets provide raw video or full annotation due to copyright or platform policy (e.g., MV-58k access is API-based only [1603.09439]). Ongoing efforts include integration of foundation models for richer feature representations, weakly- or semi-supervised annotation to scale globally, and toolchains for generative repurposing and multi-language coverage. ## 7. Representative Dataset Summaries | Dataset | Key Features | Open Access | |-----------------|--------------------------------------------|------------------| | [2502.05922] | Full user–video logs, content features | Public/CC | | [2503.23746] | Cross-platform, graph, propagation scoring | Planned/CC | | [2505.12632] | GUI action annotation, fully automated | Planned | | [2311.17335] | Expert emotion, multi-stage quality control | Planned | | [2211.00732] | 200M knowledge videos, multi-modal linking | Planned | | [2412.08879] | Repurposing edits, UGC with dense labeling | Public/CC BY-NC | | [2203.14456] | 11 languages, 6+ annotation axes | Not specified | | [1603.09439] | 260K Vine videos, temporal open world | API-based | These resources constitute the modern landscape for empirical research in large-scale mobile short-video analysis, supporting advances in user modeling, video understanding, multiscale propagation, knowledge retrieval, and AI agent training [2502.05922][2503.23746][2505.12632][2311.17335][2211.00732][2203.14456][2412.08879][1603.09439]. Source: https://www.emergentmind.com/topics/large-scale-mobile-short-video-datasets