---
title: MMLottie-2M Dataset for Multi-modal Animation
url: https://www.emergentmind.com/topics/mmlottie-2m-dataset
type: topic
---

# MMLottie-2M Dataset for Multi-modal Animation

MMLottie-2M is a large-scale, unified dataset of vector animations, created to advance research in multi-modal generative modeling for Lottie animations. Comprising two million samples paired with rich textual and visual annotations, MMLottie-2M emphasizes both professional design diversity and procedural motion coverage, supporting modeling tasks conditioned on text, image, and video modalities. Its curation incorporates a principled tokenizer and normalization pipeline to transform raw Lottie JSON files into structured, model-friendly representations, thereby facilitating the training and evaluation of next-generation vector animation models [2603.02138].

## 1. Dataset Composition and Source Breakdown

The MMLottie-2M dataset, denoted $\mathcal{D}$, totals $|\mathcal{D}| = 2\,000\,000$ vector animation samples. Source distribution comprises two primary classes:
- **Web-crawled Lottie files**: $1.2$ million instances sampled from five distinct platforms: LottieFiles (42.3%), IconScout (23.7%), Flaticon (18.1%), Iconfont (9.8%), Icons8 (6.1%).
- **Synthetic SVG-derived Lottie files**: $0.8$ million instances produced by animating static SVGs (from the OmniSVG collection) using $1$–$3$ randomly assigned procedural motion templates.

Data splits follow a standard protocol: 96% train ($|\mathcal{D}_{\text{train}}| = 1\,920\,000$), 2% validation ($|\mathcal{D}_{\text{val}}| = 40\,000$), 2% test ($|\mathcal{D}_{\text{test}}| = 40\,000$). For benchmarking, the dataset defines MMLottie-Bench, a hold-out evaluation subset comprising $900$ cases (split equally by real/synthetic content and by modeling task), all disjoint from the training set.

## 2. Modalities, Annotation Schema, and Data Pairing

Each animation is annotated with multiple modalities:
- **Textual descriptions**
  - *Coarse caption*: A global summary (mean $86 \pm 21$ words) describing high-level identity, colors, and visual style.
  - *Fine-grained temporal caption*: Sequential details (mean $114 \pm 25$ words) structured temporally, emphasizing object identities, spatial layout, and explicit motion verbs (e.g., "fading in," "rotating clockwise").
- **Visual frames**
  - *Rendered video*: $512\times512$ px resolution, $30$ fps, capturing the full animation sequence.
  - *Keyframe image*: Single frame from $t \in [0.2, 0.8] \cdot T$ to support the Text-Image-to-Lottie (TI₂Lottie) modality.
- **Multi-modal labels**
  - (image, text) pairs for TI₂Lottie.
  - Raw video instructions for Video-to-Lottie tasks.
- **Additional metadata**
  - *Semantic tags*: Fifteen high-level categories (e.g., “UI elements”: 50%, “abstract patterns”: 20%, others).
  - *Color vocabulary*: Dominant color tokens (e.g., "blue," "red," "white" among top-5).
  - *Motion types*: Distribution primarily favoring translation, then rotation, scaling, opacity changes, and path morphing.

The annotation pipeline leverages Qwen2.5-VL for dual-stage captioning, sequentially producing a global overview and temporally organized per-frame details with explicit use of geometry and motion keywords.

## 3. Data Pipeline: Collection, Curation, and Normalization

The dataset construction follows a structured pipeline:
- **A. Source aggregation**: Web crawling yields $\sim1.2$ million raw Lottie JSON files. Additionally, $\sim2$ million static SVGs (OmniSVG) are converted to Lottie format, with procedural animation yielding $0.8$ million synthetic clips.
- **B. Cleaning and filtering**: Lottie files containing base64 image layers, audio/camera, After Effects expressions, or non-parameterizable (e.g., 3D/data) layers are discarded.
- **C. Spatio-temporal normalization**:
  - *Spatial*: All animations are centered and scaled to a $512 \times 512$ canvas using scaling factor $r = \min(512/w_{\text{orig}}, 512/h_{\text{orig}})$.
  - *Temporal*: All animation key times, including $ip$, $op$, and intermediate keyframes, are normalized to $[0, 16]$ (approx. $0$–$60$ frames at 30 fps), so $t_{\text{norm}} = 16\cdot (t_{\text{orig}} - ip)/(op - ip)$.
- **D. Rendering**: Each Lottie is rendered as MP4 (512 × 512 px, 30 fps), overlayed on a random pastel background (20-color palette), with a keyframe extracted for image annotation.
- **E. Multi-modal annotation**: Automated, two-stage captioning augmented with geometry/motion terms for alignment with downstream generative tasks.

## 4. Statistical and Structural Properties

The MMLottie-2M dataset displays extensive variation in both temporal and structural dimensions:

- **Temporal characteristics**
  - *Web-crawled*: Mean duration $3.2$ s ($\sigma=2.1$ s); 67% in $[1,5]$ s, 21% in $[5,8]$ s, 12% exceeding $8$ s.
  - *Synthetic*: Mean duration $2.8$ s ($\sigma=0.9$ s).
  - After normalization, all animations occupy a uniform 16-unit time window (approx. $90$ frames at $30$ fps).

- **Structural features**
  - *Layer count*: Web-crawled mean $8.6$ (max $324$), synthetic mean $3.2$ (max $45$).
  - *Layer-type distribution*: Shape (86.8%), Precomp (8.2%), Null (2.9%), Solid (1.5%), Text (0.6%).
  - *Nesting depth*: 78.3% of samples have depth $=1$, 16.7% depth $=2$, 5.0% depth $\geq3$.

- **Control parameter statistics**
  - *Position*: $(x, y)$ clustered in $[-256, +256]$.
  - *Scale*: Usually $[0.5, 2.0]$, with outliers up to $4\times$.
  - *Rotation*: Range $[0^\circ, 360^\circ]$ (looped) or oscillatory $\pm180^\circ$.
  - *Opacity*: $[0, 100]$%.
  - Keyframe times: $[0, 16]$; color channels: integer $[0, 255]$.

## 5. File Format Structure and Tokenization

To facilitate learning and ensure lossless representation, MMLottie-2M employs a structured Lottie tokenizer:

- **JSON schema**
  - Core fields: $v$, $fr$, $ip$, $op$, $w$, $h$, $nm$, $ddd$, layers;
  - Conditional: $assets$, $markers$, $fonts$, $chars$ (present only as used).
- **Tokenizer mapping**
  - Each Lottie file is decomposed to metadata $\mathcal{M}$ and a set of layers $\{\mathcal{L}_i\}$, where each layer $\mathcal{L}_i = (\tau_i, \mathcal{A}_{\tau_i}, \mathcal{T}_i, \mathcal{E}_i)$ for type $\tau_i \in \{0,1,3,4,5\}$.
  - Continuous parameters quantized: $\mathrm{token}(x, t) = \lfloor x \cdot s_t \rfloor + o_t$, where $s_t$ is parameter-type-specific scale and $o_t$ is the vocabulary offset.
  - The token sequence: $\mathcal{T} = [\text{CMD}_1, p_{1,1}, \ldots, p_{1,k_1}, \ldots, \text{CMD}_M, p_{M,k_M}]$.
  - Text fields (e.g., font names, captions) are subword-tokenized using a pretrained VLM tokenizer, prefixed with a length token.
  - Special marker and <END> tokens ensure full, structure-preserving encoding.
  - Reconstruction is lossless via inversion: $p = (\mathrm{token} - o_t)/s_t$ and $text = V_{\text{text}}^{-1}(\ldots)$.

## 6. Comparison with Prior Datasets and Benchmarks

MMLottie-2M represents a substantial advance relative to previous publicly available Lottie datasets:
- **Scale**: $2\,000\,000$ samples, substantially larger than prior real-Lottie corpora (typically $<100\,000$) and uniquely offering comprehensive multi-modal annotation.
- **Modal coverage**: First dataset to provide unified text, image, and video conditioning for vector animation tasks.
- **Annotation depth**: Incorporates dual-stage, temporally fine-grained captions. Prior datasets offer only single-sentence or coarse-grained descriptions.
- **Content diversity**: Integrates professionally designed assets from five distinct sources and procedurally generated SVG-based animations, spanning broad stylistic and motion categories.
- **Benchmarking**: Establishes MMLottie-Bench, a standardized evaluation suite comprising both real and synthetic subsets, with $900$ holdout samples and LLM-judged alignment metrics (e.g., object/motion alignment).

A plausible implication is that the size, diversity, and annotation richness of MMLottie-2M enable robust model training and facilitate rigorous benchmark comparisons for generative systems targeting vector animation, such as OmniLottie [2603.02138].

---

**Table: Source Composition of MMLottie-2M**

| Source Type    | Fraction (%) | Number of Files    |
|----------------|-------------|--------------------|
| LottieFiles    | 42.3        | $\approx$507,600   |
| IconScout      | 23.7        | $\approx$284,400   |
| Flaticon       | 18.1        | $\approx$217,200   |
| Iconfont       | 9.8         | $\approx$117,600   |
| Icons8         | 6.1         | $\approx$73,200    |
| Synthetic SVGs | —           | 800,000            |

---

MMLottie-2M establishes a new standard for vector animation datasets, with design decisions tailored for supporting multi-modal generative modeling, evaluation, and analysis within the domain of Lottie JSON-encoded content [2603.02138].

Source: https://www.emergentmind.com/topics/mmlottie-2m-dataset