LAION-400M: Large-Scale Vision–Language Dataset

Updated 19 November 2025

LAION-400M is an open dataset of 400 million image–alt-text pairs curated from web data, supporting contrastive vision–language pretraining and multimodal research.
It employs a multi-stage filtering pipeline using CLIP-based semantic similarity, deduplication, and NSFW classification to ensure data quality and safety.
The dataset provides CLIP embeddings and FAISS indices for scalable multimodal retrieval, pretraining, and bias analysis in downstream models.

LAION-400M is a large-scale, open dataset consisting of approximately 400 million image–alt-text pairs scraped from the Common Crawl web archive and filtered using CLIP-based semantic similarity. It is designed to support research on contrastive vision–language pretraining, multimodal retrieval, and large-scale model benchmarking, providing both the metadata and the CLIP embeddings required for efficient experimentation at web scale (Schuhmann et al., 2021).

1. Data Collection, Filtering, and Structure

The LAION-400M dataset was built via a multi-stage web-scale pipeline:

Crawling and Extraction: The Common Crawl archives (HTML content spanning snapshots from 2014 through 2021) were parsed for <img> HTML tags containing an alt attribute. Each candidate yields a tuple of the form (URL, alt-text), with basic metadata such as width, height, and file size (Schuhmann et al., 2021, Birhane et al., 2023).
Initial Filters: Early filters drop any pair where the alt-text is shorter than 5 characters or the image size is less than 5 KB. A Bloom filter is used to deduplicate based on (URL, alt-text) pairs.
CLIP Similarity Filtering: Both image and text are embedded by an off-the-shelf CLIP model. The cosine similarity between the image and text embeddings is computed as

$s(x, y) = \frac{⟨f_{\mathrm{image}}(x),\,f_{\mathrm{text}}(y)⟩} {‖f_{\mathrm{image}}(x)‖\;‖f_{\mathrm{text}}(y)‖}$

Only pairs with $s(x, y) \ge 0.3$ are retained (Schuhmann et al., 2021, Birhane et al., 2023, Birhane et al., 2021).

NSFW Tagging: Each image is passed through a CLIP-based NSFW classifier, assigning one of {“UNLIKELY”, “UNSURE”, “NSFW”}, as well as a continuous score in 0,1.
Sharding: The resulting dataset (~413.9 million pairs) is uniformly distributed into 32 Parquet-format shards for scalable downstream access (Birhane et al., 2023, Birhane et al., 2023).

Each record in LAION-400M contains the following fields:

Field	Description
sample_id	Unique integer index
url	Image URL
alt_text	Scraped alt-text (caption)
width/height	Image dimensions
file_size	Image file size in bytes
license	License information if declared
clip_score	CLIP cosine similarity
nsfw	Discrete flag: “UNLIKELY”, “UNSURE”, or “NSFW”

Accompanying the metadata are the precomputed CLIP embeddings (NumPy array, $d=512$ or $d=768$ ), and FAISS-based kNN indices (Schuhmann et al., 2021).

2. Content Statistics and Composition

Total pairs: $N = 413,871,335$ after deduplication.
Image resolution: Approximately 211M images have both dimensions ≥256px, 67M ≥512px, and 9.6M ≥1024px (Schuhmann et al., 2021).
Language composition: Captions are scraped from a web-scale multilingual corpus, with English dominant and “long tail” coverage of many other languages (Spanish, Hindi, etc.), but no explicit per-language breakdown is provided (Schuhmann et al., 2021, Birhane et al., 2023, Birhane et al., 2021).
NSFW prevalence: <1% of images are flagged as NSFW by the CLIP-based image classifier, though full prevalence tables are not reported (Schuhmann et al., 2021, Birhane et al., 2021).

Alt-texts are often brief or generic, and their topical and linguistic quality is highly variable. The dataset includes substantial social, cultural, and geographic biases reflective of its open web source (Schuhmann et al., 2021, Birhane et al., 2021).

3. Audit of Harmful and Problematic Content

Multiple independent audits have systematically documented non-trivial levels of harmful content in LAION-400M alt-texts and images.

3.1. Quantitative Toxicity Analysis

The “Hate Content Rate” (HCR) metric measures the proportion of alt-texts predicted as hateful, targeted, or aggressive by an open-source “pysentimiento” NLP model. For a threshold $P_{\mathrm{th}}=0.5$ , the “any-of-the-three” HCR on a uniform random 3.2M-sample subset is

$\bar\psi_{400M}(0.5) = 0.298\% \;\;\; \text{(95\% CI: [0.292\%,\,0.304\%])}$

Category-wise breakdown (at $P_{\mathrm{th}}=0.5$ ):

Category	HCR ( $\%$ )
hateful	0.285
targeted	0.12
aggressive	0.01

3.2. Qualitative and Manual Audits

Automated string-matching, keyword audits, and qualitative spot checks uncover prevalent malignant stereotypes and explicit content:

Search for “Desi”: 34,516 matches; ~34.1% match an explicit NSFW pattern.
Search for “Latina”: 37,769 matches; 28.2% NSFW.
Multiple queries (“Maa”, “Nun”, “Black woman”, etc.) yield significant rates of pornographic or stereotyping co-occurrence (Birhane et al., 2021).

Audit spot checks reveal that filtering procedures fail to cull misogyny, sexual violence, and explicit racist/ethnic slurs (Birhane et al., 2021, Birhane et al., 2023).

3.3. NSFW Filtering Limitations

Filtering by NSFW (image-based) scores fails to fully remove toxic or hateful text content. Even after conservative NSFW score thresholding, ≈0.24% of captions remain hateful and ≈0.03% targeted (per pysentimiento model) (Birhane et al., 2023, Birhane et al., 2023). Image “safety” and caption toxicity are only weakly correlated (Pearson $r\approx 0.22$ ) (Birhane et al., 2023).

4. Demographic Annotation and Bias Transfer

Person-centric annotations for LAION-400M combine YOLOv11-l object detection (COCO, confidence ≥0.25) and CLIP-based (ViT-B-16 SigLIP) gender and race/ethnicity classifiers, yielding 199.9M high-quality bounding boxes with perceived attributes and captions (Girrbach et al., 4 Oct 2025).

Demographic statistics (bounding boxes):
- Gender: 42% male, 35% female, 13% mixed, 10% unclear.
- Race/ethnicity: 28% White, 7% Black, 6% East Asian, 4% South Asian, 3% Latino, 3% Southeast Asian, 2% Middle Eastern, 50% unclear.

Bias Measurement: Co-occurrence of demographic labels with 63 crime-related keywords shows relative overrepresentation for males (+57%), Black individuals (+51%), and Middle Eastern (+206%); White and East Asian are underrepresented (–22%) (Girrbach et al., 4 Oct 2025).

Transfer to Model Bias: For CLIP and Stable Diffusion, 60–70% of measured downstream gender bias is linearly explained by first-order demographic co-occurrences in LAION-400M; similar patterns hold for crime association (Girrbach et al., 4 Oct 2025).

Bias Metric Notation (model–data correlation):

$d_{\mathrm{model}}(c) \approx \alpha \cdot b_{\mathrm{data}}(c) + \beta$

where $b_{\mathrm{data}}(c)$ is the fraction of category $c$ associated with a given group; $d_{\mathrm{model}}(c)$ is model-assigned group score.

5. Practical Use, Retrieval, and Downstream Impact

5.1. Access and Retrieval

LAION-400M is distributed as:

Component	Format / Details
Metadata	32 Apache Parquet parts, fields as above
Embeddings	NumPy arrays, shape $(N, d)$ , $d=512$ (ViT-B), $d=768$ (ViT-L)
Indices	FAISS IndexIVFPQ, one per embedding shard

Standard text-to-image and image-to-text retrieval is supported via CLIP embedding and FAISS kNN search (Schuhmann et al., 2021).

5.2. Pretraining and Representation Learning

The MLCD framework clusters LAION-400M into $k=1,000,000$ centroids using k-means on CLIP embeddings, then assigns each image $l=8$ nearest pseudo-labels to capture multi-object semantics. The resulting representations yield state-of-the-art transfer (e.g., $+1.1\%$ average linear-probe over CLIP on 26 datasets, $+3.9\%$ zero-shot over OpenCLIP) (An et al., 24 Jul 2024).

Clustering Objective:

$\min_{W \in \mathbb{R}^{d \times k}} \; \frac{1}{n}\sum_{i=1}^n \min_{y_i \in \{0,1\}^k,\, y_i^\top 1_k = 1} \|e_i - Wy_i\|_2^2$

Model and data scale synergistically raise representation quality; the diversity and size of LAION-400M are critical (An et al., 24 Jul 2024).

5.3. Downstream Bias

Zero-shot classification audits of CLIP models pretrained on LAION-400M using the Chicago Face Dataset demonstrate that only 18.6% of human-face images are recognized as “human being,” and the top-1 “criminal” rate for Black female faces is 21.2%, 14% for Black male faces. A plausible implication is that model bias is strongly inherited from dataset composition (Birhane et al., 2023).

6. Documented Limitations and Recommendations

Documented limitations:

Alt-text is frequently noisy, brief, or generic.
Web-scale sampling disproportionately embeds specific cultural, political, and demographic biases.
Copyright and licensing are variable, and images are distributed only as URLs and metadata; users must address licensing compliance (Schuhmann et al., 2021).
Simple NSFW filtering is insufficient to ensure safety; high rates of hateful/targeted speech persist (Birhane et al., 2023, Birhane et al., 2023, Birhane et al., 2021).
Scaling the dataset does not reduce, and often increases, the prevalence of harmful content (12% relative HCR increase from LAION-400M to LAION-2B-en) (Birhane et al., 2023, Birhane et al., 2023).

Recommendations for dataset curation:

Report raw and thresholded scores for NSFW, hate, targeted, and aggressive content, enabling tailored downstream filtering.
Combine image-based and text-based filters to mitigate cross-modal harms.
Transparently document provenance, language distribution, and bias metrics.
Release both datasets and compute-efficient auditing tools to enable independent review.
Actively rebalance or curate using detailed demographic annotations to mitigate bias transfer (Girrbach et al., 4 Oct 2025, Birhane et al., 2023).

7. Significance and Community Impact

LAION-400M filled the critical gap for open web-scale datasets suitable for vision–language pretraining, enabling direct reproducibility and innovation in contrastive learning, generative modeling, and large-scale multimodal retrieval (Schuhmann et al., 2021). However, its construction from minimally filtered web-crawled data entrenched societal, demographic, and representational biases; these are empirically shown to propagate robustly to downstream models (Girrbach et al., 4 Oct 2025, Birhane et al., 2023, Birhane et al., 2021). Systematic auditing, annotation, and community vigilance are essential for safer, fairer, and more accountable large-scale multimodal data curation.