---
title: 'CIPE-Dance: Large-Scale Dance Video Dataset'
url: https://www.emergentmind.com/topics/cipe-dance
type: topic
---

# CIPE-Dance: Large-Scale Dance Video Dataset

Searching arXiv for the cited works to ground the article in current papers.
Search query: "OmniDance Multimodal Driven Dance Video Generation with Large-scale Internet Data arXiv"
CIPE-Dance is a large-scale Internet-sourced dance video dataset introduced together with the OmniDance framework for multimodal dance video generation. In that setting, the term denotes both a concrete data resource and a data-construction paradigm: a corpus of approximately 300k high-quality dance clips totaling more than 400 hours, paired with synchronized music and choreography-informed text annotations, and assembled through a progressive expert pipeline designed to filter for quality, single-dancer clarity, and choreography relevance [2606.30019]. CIPE-Dance is explicitly curated for three conditional generation regimes—text–image-to-video (TI2V), music–image-to-video (MI2V), and music–text–image-to-video (MTI2V)—and serves as the principal training and evaluation substrate for OmniDance [2606.30019].

## 1. Definition, scope, and nomenclature

CIPE-Dance is defined as a **large-scale Internet-sourced dance video dataset** with **Choreography-Informed text annotations**, constructed via a **Progressive Expert-based pipeline**. In the OmniDance formulation, the acronym CIPE expands to **Choreography-Informed, Progressive Experts** rather than to a motion encoder or a graph positional encoding scheme [2606.30019].

Its stated purpose is to address three deficits in prior dance-generation resources: lack of scale, lack of in-the-wild diversity, and lack of multimodal, choreography-aware annotation. The dataset is therefore organized around clips that contain a clear single dancer, stable camera conditions, a usable reference frame for identity control, and synchronized audio suitable for music-conditioned generation [2606.30019].

A common source of ambiguity is the acronym itself. In a separate graph-Transformer literature, CIPE refers to **Communicability-Inspired Positional Encoding**, a graph positional encoding derived from communicability and heat-kernel geometry [2606.25293]. That work is unrelated in domain and objective to CIPE-Dance. In the dance-video context, CIPE-Dance names a dataset and collection pipeline rather than a graph positional encoding method.

## 2. Progressive expert pipeline and data curation

CIPE-Dance is built through a six-stage **Progressive Expert-Based Data Collection Pipeline** applied to web-scraped short-form video from **Douyin and TikTok**. The guiding principle is that simpler checks are assigned to lightweight experts and more difficult checks to heavier experts. Source videos are taken from about **500 dance content creators** with **>50K followers**; only videos **after 2018** are retained; and videos are segmented into **5-second clips at 16 FPS**, yielding an initial pool of **630k clips** [2606.30019].

The filtering stages are summarized below.

| Stage | Tool | Retention |
|---|---|---:|
| Visual Quality Verification | VBench | 96.76% |
| Reference Clarity Verification | Qwen3-VL-2B | 81.45% |
| Dance Video Verification | Qwen3-VL-2B | 89.24% |
| Single-Dancer Filtering | Qwen3-VL-2B | 74.45% |
| Scene Stability Filtering | two Qwen3-VL-8B models | 91.65% |

The **Visual Quality Verification** stage removes low-quality, artifact-heavy, or visually cluttered clips using **VBench** thresholds of **Image quality (IQ) > 60.00** and **Aesthetic quality (AQ) > 50.00**. The **Reference Clarity Verification** stage uses **Qwen3-VL-2B** to reject clips whose first frame has low visibility, heavy occlusions, back-facing poses, or missing clear human facial cues. The **Dance Video Verification** stage again uses **Qwen3-VL-2B** to exclude talking, posing, and static-motion clips [2606.30019].

The **Single-Dancer Filtering** stage is notable because it inverts the detection problem: instead of directly proving singularity, it removes three negative cases—**group dance**, **follow-along imitation** videos with a small overlaid reference, and **mirror-reflection artifacts**. These three filters retain **80.35%**, **97.02%**, and **95.57%** of inputs respectively, giving **74.45%** overall retention for the stage. The final **Scene Stability Filtering** stage uses **two Qwen3-VL-8B models** to remove abrupt scene or subject transitions and aggressive camera motion, with **93.95%** and **97.55%** retention for the two subfilters and **91.65%** overall retention [2606.30019].

After all stages, the pipeline yields **approximately 300k high-quality 5s clips**. This suggests that CIPE-Dance is designed not merely as a passive scraped corpus, but as a benchmark-quality dataset in which identity stability, scene continuity, and dance salience are treated as first-class curation constraints.

## 3. Dataset statistics, modalities, and choreography-informed annotations

The resulting dataset comprises approximately **300,000 clips**, **>400 hours** of video, and clips of **5 seconds** at **16 FPS**, giving **77 frames per clip**. It spans **over 30 dance genres** and exhibits a **long-tailed** genre distribution. The dataset also includes substantial diversity in environment, performer appearance, motion complexity, and camera setup, while retaining scene stability through filtering [2606.30019].

CIPE-Dance is multimodal. Each item contains a full-body dance video, the original audio track, and **five separate textual descriptions** generated by **Qwen3-VL-8B**, each aligned to a distinct choreographic aspect [2606.30019]. These five aspects are:

1. **Body Dynamics**  
2. **Choreographic Content**  
3. **Expressiveness**  
4. **Camera Presentation**  
5. **Overall Look**

The annotation scheme is explicitly choreography-aware. **Body Dynamics** describes moving body parts and how they move; **Choreographic Content** captures genre, style, and named movement techniques; **Expressiveness** characterizes emotion and performance intensity; **Camera Presentation** records framing, angle, and camera movement; and **Overall Look** summarizes clothing, styling, and environment [2606.30019]. Because the annotations are clip-level and multimodal, they support text-only generation, music-only generation, and combined text-plus-music control.

For evaluation, the authors **randomly sample 100 clips** as a **held-out test split**, excluding them from training. No detailed separate validation split is specified in the provided description [2606.30019].

## 4. Role in OmniDance and multimodal training

CIPE-Dance functions as the primary training and evaluation dataset for **OmniDance**, a framework for integrating music into a text-image-to-video foundation model while preserving controllability and visual fidelity [2606.30019]. OmniDance treats text as carrying low-frequency semantics and music as carrying high-frequency temporal dynamics, and this division of labor is directly motivated by the annotation and synchronization structure of CIPE-Dance.

The training objective is a **Flow Matching** loss over latent video trajectories. For a CIPE-Dance clip encoded into latent $\mathbf{x}_0$, Gaussian noise $\epsilon \sim \mathcal{N}(0,I)$, and timestep $t \sim \mathcal{U}(0,1)$, the model uses
$$
\mathbf{x}_t = (1 - t)\cdot \mathbf{x}_0 + t\cdot \epsilon,
\qquad
\mathbf{v}_t = \epsilon - \mathbf{x}_0,
$$
with objective
$$
\mathcal{L}_{\text{fm}}
=
\mathbb{E}_{\mathbf{x}_0,\epsilon,t}
\left[
\left\|
\mathbf{v}_\theta - \mathbf{v}_t
\right\|_2^2
\right].
$$
This loss is applied under different conditioning configurations corresponding to TI2V, MI2V, and MTI2V [2606.30019].

OmniDance uses a three-stage curriculum built entirely on CIPE-Dance. **Stage I** is a **Text-Adaptation Warm-Up** using TI2V only, with the music branch frozen. **Stage II** is **Anchored Music Integration**, training on both TI2V and MTI2V while enabling the music branch with zero-initialized residual projection and a warmed-up learning rate. **Stage III** is **Modality Decoupling and Specialization**, training on TI2V, MI2V, and MTI2V jointly [2606.30019]. A plausible implication is that the dataset’s paired text–music–video structure is not ancillary metadata but the enabling condition for this staged multimodal optimization.

Two architectural mechanisms are explicitly tied to the CIPE-Dance design. First, **Music–Text Progressive Specialization (MTPS)** scales the audio residual depth-wise:
$$
\mathbf{x}_\ell \leftarrow \mathbf{x}_\ell
+ \left( \frac{\ell}{L} \right)^{\gamma} \cdot \mathbf{r}_{\text{audio}}^\ell,
$$
with $\gamma=1.5$ and $L=30$. Second, OmniDance uses a **modality-specialized time-dependent CFG strategy**, with text guidance decaying from **5→1** over denoising time and music guidance decaying more slowly from **4→2**, increasing the relative influence of music in later stages of generation [2606.30019].

## 5. Benchmarking and evaluation on CIPE-Dance

CIPE-Dance is also used as a benchmark for dance video generation. All reported metrics in OmniDance’s main comparison table are computed on the **100-clip CIPE-Dance test set** [2606.30019]. The evaluation spans video quality, motion quality, and alignment.

For **video quality**, the benchmark uses VBench-style metrics: **IQ (Image Quality)**, **AQ (Aesthetic Quality)**, **SC (Subject Consistency)**, **BC (Background Consistency)**, **MS (Motion Smoothness)**, and **TF (Temporal Flickering)** [2606.30019]. For **motion quality**, the pipeline extracts **2D keypoints** using **ViTPose**, computes **kinetic** and **geometric** features following AI Choreographer-style descriptors, and reports **FID\_k**, **FID\_g**, **DIV\_k**, and **DIV\_g**. The Fréchet distance is given by
$$
\text{FID}(P_r, P_g) = \|\mu_r - \mu_g\|_2^2 + \mathrm{Tr}\left(\Sigma_r + \Sigma_g - 2(\Sigma_r \Sigma_g)^{1/2}\right),
$$
where $(\mu_r,\Sigma_r)$ and $(\mu_g,\Sigma_g)$ are statistics of real and generated feature distributions [2606.30019].

For **alignment**, the benchmark uses **BAS (Beat Alignment Score)** for music–motion synchronization and **OC (Overall Consistency)** as a text–video consistency score. In MI2V, a generic prompt—“a person is dancing”—is used for OC computation [2606.30019]. The qualitative analysis attributes OmniDance’s gains on this benchmark to more diverse hand articulations, natural smiles and facial expressions, stable identity, and rhythm-aware choreography. The paper also reports that MTI2V outputs achieve **BAS ≈ 0.287** compared with **0.288** for real CIPE-Dance videos, indicating near-ground-truth beat alignment on that metric [2606.30019].

This benchmark design differs from earlier dance-generation evaluations centered on pose reconstruction, beat hit rate, or style-embedding FID. For example, prior 2D music-to-dance work emphasized **style embeddings**, **beat hit rate**, and FID in style space [2104.14802], while later 3D dual-learning work used **Fréchet Dist**, **Diversity**, and **Beats Align** in SMPL motion space [2201.11999]. CIPE-Dance extends evaluation into full video generation, where motion fidelity must coexist with appearance consistency, background stability, and multimodal controllability.

## 6. Research context, misconceptions, and limitations

CIPE-Dance emerges in a lineage of dance-generation research that previously relied on substantially smaller or more specialized resources. Earlier work on style-aware music-to-dance synthesis built a dataset of **anime**, **popping**, and **locking** videos with approximately **1.48M frames** after filtering and used 2D OpenPose keypoints with learned style embeddings [2104.14802]. Dual music–dance modeling later advanced to 3D SMPL sequences using Tang et al. and AIST++-derived data, but training still occurred on short fixed windows and without Internet-scale choreography-aware video captions [2201.11999]. CIPE-Dance differs by targeting full-body dance video generation in the wild, with synchronized music and rich clip-level text annotations [2606.30019].

One misconception is to treat CIPE-Dance as a pose dataset analogous to AIST++ or FineDance. The provided description does not present stored 2D or 3D pose annotations as part of the dataset itself; instead, **ViTPose** keypoints are extracted on the fly for evaluation [2606.30019]. Another misconception is to interpret the dataset name as implying a direct use of the graph CIPE positional encoding framework. The available description does not state such a connection; the shared acronym reflects different expansions and different research problems [2606.25293].

The reported limitations are largely those of the source domain and collection strategy. Because CIPE-Dance is assembled from **Douyin/TikTok dance creators with >50K followers**, it is likely biased toward East Asian and global pop dance trends and toward styles common on short-video platforms. Recording conditions are also skewed toward vertical, phone-camera, influencer-style production. The dataset is clip-oriented at **5 seconds**, so longer choreographic structure is not explicitly annotated. The paper also does not provide a detailed legal or ethical discussion beyond noting that the source videos are **publicly available** [2606.30019].

These constraints do not diminish the dataset’s importance; rather, they define its research position. CIPE-Dance supplies a large-scale, choreography-aware, multimodal benchmark for TI2V, MI2V, and MTI2V dance video generation, while leaving open problems in cultural coverage, longer-horizon choreography, richer structured labels, and explicit bias analysis [2606.30019].

Source: https://www.emergentmind.com/topics/cipe-dance