---
title: 'Sakuga-42M: Cartoon Animation Dataset'
url: https://www.emergentmind.com/topics/sakuga-42m
type: topic
---

# Sakuga-42M: Cartoon Animation Dataset

Sakuga-42M is a large-scale, multimodal dataset of authentic, hand-drawn 2D cartoon animation, constructed to address the domain bias that separates cartoons from natural video in contemporary video foundation models. Comprising 42 million non-repetitive keyframes, the dataset is sourced from more than 150,000 publicly available cartoon videos across diverse regions, years, and artistic styles. Sakuga-42M provides comprehensive semantic annotations, including temporally coherent captions, fine-grained anime tags, rich content taxonomies, and auxiliary metadata. Empirical results demonstrate that contemporary models such as Video CLIP, Video Mamba, and SVD, when fine-tuned with Sakuga-42M, achieve significant gains in cartoon comprehension and generation tasks, evidencing a robust scaling law without saturation.

## 1. Dataset Construction and Scope

The Sakuga-42M dataset was built via a multi-step data-collection and preprocessing pipeline:

- **Raw Video Crawling**: Custom crawlers aggregated approximately 150,000 videos from YouTube, Twitter, and animation communities, strictly adhering to privacy policies. Source formats included mkv, mp4, and webm.
- **Shot/Clip Splitting**: PySceneDetect segmented videos into ∼1.4 million clips, with each clip having a minimum duration of 18 frames and extending up to 200+ frames.
- **Keyframe Detection**: Leveraging the tendency of cartoons to "animate on twos or threes," SSIM was computed between adjacent frames, discarding duplicates and yielding exactly 42,000,000 keyframes by removing 45% of redundant frames.
- **Semantic Captioning**: wd14-swin-v2 generated anime tags every 12 frames, BLIP-v2 provided initial captions, and ChatGPT-175B integrated tags and BLIP outputs to formulate coherent, temporally aware descriptions. Raw BLIP-v2 captions were preserved for future comparison.

Quantitative summary:

| Attribute             | Statistic/Proportion                                                                         |
|-----------------------|----------------------------------------------------------------------------------------------|
| Total keyframes       | $N_{\text{total}} = 42\,\mathrm{M}$                                                          |
| Mean clip length      | 35 frames                                                                                     |
| Clip duration buckets | $<$12 (23.02%), 12–24 (27.51%), 24–48 (27.00%), 48–96 (15.99%), $>$96 (6.48%)                |
| Regional distribution | JP: 45%, US: 20%, CN: 20%, EU: 15%                                                           |
| Artistic style        | Raster-based Asian: 70%, Vector-based Western: 30%                                           |
| Temporal coverage     | Uniform across 1950s–2020s, each decade $\sim$12–18%, $N_{k\text{-}decade}\approx\frac{42\,\mathrm{M}}{8}$ |

This structure ensures broad diversity in region, time, and style, spanning raster-based Asian animation and vector-based Western styles, and provides coverage from the 1950s through the 2020s [2405.07425].

## 2. Semantic Annotations and Taxonomy

Sakuga-42M is annotated across multiple modalities and hierarchical taxonomies:

- **Video–Text Description Pairs**: Generated via BLIP-v2 and ChatGPT-175B, with an average length of 40 words; over 85% are longer than 20 words. Descriptions are temporally coherent, leveraging anime-tag priors.
- **Fine-grained Anime Tags**: Over 600 tags from wd14-swin-v2, capturing character traits, objects, and scene semantics.
- **Content Taxonomies**: Six orthogonal dimensions:
  - Time (per decade)
  - Venue (region: JP, US, CN, EU)
  - Media (Cel-animation, Raster, Vector)
  - Objective (Action, Comedy, Educational, Documentary, etc.)
  - Composition (Static, Dynamic, Pan, Zoom, Camera Move)
  - Character Type (Human, Animal, Creature, Object)
- **Auxiliary Metadata**: CAFe-aesthetic score, dynamicity ratio (keyframes/frames), safety ratings (general, sensitive, questionable, explicit), text-presence probability, resolution, aspect ratio, split boundaries, URLs, and hash IDs.

Taxonomy is provided in a nested structure, facilitating multi-attribute queries and aggregated analysis.

## 3. Foundation Model Adaptation and Fine-Tuning

Sakuga-42M was used to fine-tune multiple contemporary architectures for enhanced cartoon comprehension and generation:

- **Video–Language Understanding**: ViCLIP-200M and VideoMamba-25M (dual encoder: vision/text) trained with a contrastive loss:
  $$
  \mathcal{L}_{\mathrm{contrast}} = -\sum_{i=1}^N \Bigl[\,       \log\frac{\exp\bigl(\mathrm{sim}(f_i^V,f_i^T)/\tau\bigr)}                    {\sum_j\exp\bigl(\mathrm{sim}(f_i^V,f_j^T)/\tau\bigr)}     + \log\frac{\exp\bigl(\mathrm{sim}(f_i^T,f_i^V)/\tau\bigr)}                    {\sum_j\exp\bigl(\mathrm{sim}(f_i^T,f_j^V)/\tau\bigr)}     \Bigr].
  $$
  Timesheet predictor objective:
  $$
  \mathcal{L}_{\mathrm{timesheet}} = -\sum_{c=1}^C t_c\,\log\bigl(\mathrm{softmax}(y)_c\bigr).
  $$
  The training regimen entailed 1 epoch on 90% train split, batch size 256, masked learning probability 0.9, executed on 2×A6000 GPUs with DeepSpeed and Accelerate.

- **Video Generation (SVD-based)**: SVD-MV Stage-III with latent diffusion objective:
  $$
  \mathcal{L}_{\mathrm{SVD}} = \mathbb{E}_{z_t,\epsilon}\bigl\|\,\epsilon - \epsilon_\theta(z_t,t,c)\bigr\|^2,\quad c=\{\text{conditioning from keyframes}\}.
  $$
  Conditioning was performed with timesheet class predictions from ViCLIP/VideoMamba plus predictor. Fine-tuning involved 5,000 iterations, learning rate $5\times 10^{-6}$, frame resolution $8@576\times320$, on 2×A6000 GPUs, otherwise default SVD settings.

These adaptation protocols facilitated the measurement of cartoon‐specific comprehension and generation performance across scaling regimes.

## 4. Experimental Evaluation and Scaling Effects

Extensive empirical evaluation demonstrated substantial performance increases on cartoon-specific tasks:

### Zero-Shot Retrieval

Models fine-tuned with Sakuga-42M achieved significant increases in Recall@k on text-to-video (T2V) and video-to-text (V2T) retrieval tasks, tested on a 5% split (44,000 clips, 2.01M keyframes).

| Model + Domain         | R@1 T2V | R@5 T2V | R@10 T2V | R@1 V2T | R@5 V2T | R@10 V2T |
|-----------------------|---------|---------|----------|---------|---------|----------|
| ViCLIP@InternVid-200M | 22.4    | 38.2    | 45.1     | 16.8    | 30.8    | 37.7     |
| + Sakuga-42M          | 35.5    | 54.2    | 61.6     | 29.7    | 48.6    | 56.5     |
| VideoMamba@NatVid     | 18.3    | 31.2    | 38.2     | 11.7    | 23.7    | 30.0     |
| + Sakuga-42M          | 57.1    | 75.3    | 81.2     | 56.4    | 74.4    | 80.5     |

Scaling law analysis shows monotonic improvements with increased data scale (Sakuga-Small $\rightarrow$ Sakuga-Aesthetic $\rightarrow$ full 42M), with no observed performance plateau at 42M keyframes.

### Cartoon Generation

Comparative generation metrics: IS, FID, FVD, and CLIPSIM. Fine-tuning SVD on Sakuga-42M demonstrated notable gains.

| Model                  | IS    | FID   | FVD    | CLIPSIM |
|------------------------|-------|-------|--------|---------|
| SVD@LVD (natural)      | 1.09  | 23.10 | 307.6  | 0.267   |
| SVD + Sakuga-42M       | 1.12  | 14.67 | 92.3   | 0.279   |

Ablations confirm that both comprehension and generation tasks benefit from fine-tuning on larger corpus subsets.

### Domain Gap Visualization

FID/FVD analyses indicate pronounced distributional divergence between natural and cartoon video domains: FID $\sim$80–81, FVD $\sim$475–498. t-SNE projections reveal isolated Sakuga-42M clusters, substantiating the domain gap hypothesis.

## 5. Benefits and Applications

Sakuga-42M supports a broad spectrum of cartoon-specific research and practical applications:

- **Domain Bridging**: Data scale dramatically narrows the domain gap for models trained on natural footage, enabling robust adaptation to hand-drawn animation.
- **Generalization**: Fine-tuned models generalize well across diverse styles (cel, limited, vector) and decades.
- **Data-Efficiency**: Large volumes of high-quality, paired captions eliminate dependence on costly human annotation.

Principal applications include:

1. Text-to-cartoon and cartoon-to-text retrieval
2. Automated inbetweening (latent diffusion, frame interpolation)
3. Line-art colorization (sketch-to-color referencing)
4. Avatar and style transfer (via Stable Diffusion, AnimateDiff, LoRA/T2I-Adapter)
5. Cartoon generation at scale (open-source Pika-style models)
6. Video editing and sprite decomposition (layered editing, toon tracking)

## 6. Limitations and Future Directions

Several constraints and areas for improvement are identified:

- **Caption Coherence**: Some LLM-generated descriptions lack precise spatial relationship modeling. Proposed future work includes reinforcement learning from human feedback.
- **Resolution**: 80% of clips are 480p, limiting achievable fidelity; higher-resolution data would improve generation quality.
- **Taxonomy/Tag Bias**: Over-representation of Asian cel-animation and gender/human-centric tags. Recommendations include more balanced tagging strategies and increased coverage of underrepresented styles.
- **Safety**: While >99.5% of content is general, $\sim$0.4% is questionable and $\sim$0.07% is explicit. Users are encouraged to apply filtering according to application requirements.

Sakuga-42M establishes a new standard for cartoon research, enabling scalable methods for robust comprehension and high-fidelity generation, and bridging the gap to enable style-aware animation tools for academic and production settings [2405.07425].

Source: https://www.emergentmind.com/topics/sakuga-42m