ReasonLite-42M: Visual Reasoning Dataset

• ReasonLite-42M is a large-scale dataset with 42M image–caption pairs that integrate visually grounded, commonsense inferences beyond simple descriptive alignment.
• The dataset augments each image from CC12M with three descriptive captions and three open-form reasoning captions per evidence cluster, operationalizing diversity in supervision.
• Empirical evaluations show significant improvements in retrieval and reasoning benchmarks, demonstrating the benefits of reasoning-aware pretraining for CLIP-style models.

Searching arXiv for the specified paper to ground the article in the cited source. ReasonLite-42M is a 42.0 M image–caption dataset of short, open-form, visually verifiable commonsense inferences built on top of CC12M for continual pretraining of CLIP-style encoders. It was introduced as one of the core resources in the ReasonCLIP framework, whose objective is to integrate large-scale reasoning supervision into CLIP without architectural changes. In that framework, ReasonLite-42M supplies Stage 1 supervision for “Reasoning-Aware Alignment,” while the later Stage 2 introduces the more structured ReasonPro-16M dataset for category-level reasoning (Zhang et al., 25 Jun 2026).

1. Motivation and conceptual role

Standard CLIP-style pretraining is described as focusing almost entirely on descriptive alignment, exemplified by captions such as “this is a plate of eggs,” whereas many downstream tasks require visually grounded commonsense inferences, such as “those eggs are neatly aligned so you can carry and eat them more easily.” ReasonLite-42M was created to inject “reasoning” into CLIP’s representation space by augmenting each image in a large descriptive corpus with one or more short, verifiable commonsense inferences (Zhang et al., 25 Jun 2026).

The dataset is therefore not a generic caption resource. Its defining property is that each reasoning caption is grounded in observable visual evidence while expressing an inferential relation that goes beyond literal scene description. In the paper’s formulation, this allows a pretrained CLIP encoder to be nudged toward reasoning while still preserving its original alignment.

Two representative examples clarify the intended supervision format. For an image of “a plate of evenly spaced eggs,” the descriptive caption is “A white plate holds six eggs arranged in two neat rows,” and the associated reasoning caption is “Because they’re neatly aligned, you can pick up the whole plate and carry it easily without eggs rolling off.” For an image of “a person slicing a lemon over a salad,” the descriptive caption is “A hand holds a yellow lemon, slicing it above a bowl of green salad,” and the reasoning caption is “By slicing it right above the salad, the juice drips directly onto the greens for immediate flavor.” These examples illustrate that the dataset targets single-step, visually grounded commonsense rather than unrestricted chain-of-thought generation.

2. Corpus composition and scale

ReasonLite-42M is derived from 4,668,515 unique images from CC12M after initial filtering. For each image, three high-quality descriptive captions $T_{(b)}$ are regenerated, and for each such caption three visually grounded reasoning captions $T_{(rl)}$ are produced, yielding nine reasoning instances per image (Zhang et al., 25 Jun 2026).

Component	Value	Brief description
Unique images	4,668,515	From CC12M after initial filtering
Descriptive captions per image	3	High-quality captions $T_{(b)}$
Evidence clusters per $T_{(b)}$	3	$E_1, E_2, E_3$
Reasoning captions per image	9	One $T_{(rl)}$ per evidence cluster for each $T_{(b)}$
Total image–caption pairs	42.0 M	$4.668\text{ M} \times 3 \times 3$
Average $T_{(rl)}$ length	25.0 words	$\sigma = 8.1$

The construction is explicitly tied to a descriptive precursor corpus. The pipeline starts from CC12M’s 10.4 M valid images, from which Qwen2.5-VL-72B generates three concise, factual descriptive captions per image, producing CC12M-Refined with 31.2 M pairs. A subset of 4.7 M images is then reserved for ReasonLite. For each image and each descriptive caption, the model extracts key visual evidence $T_{(rl)}$0 and clusters it into three groups $T_{(rl)}$1; one open-form, commonsense-style reasoning caption is then generated per cluster.

A plausible implication is that the dataset’s scale is not simply a by-product of caption multiplication. The three-caption, three-cluster design operationalizes diversity at two levels: alternative descriptive framings of the same image and multiple inferential decompositions grounded in distinct evidence clusters.

3. Annotation pipeline and quality control

The annotation pipeline is model-assisted and multi-stage. Qwen2.5-VL-72B first produces the descriptive captions, then receives as input the image plus one $T_{(rl)}$2, and is prompted to extract key visual evidence and cluster it into three groups. For each cluster $T_{(rl)}$3, it generates one open-form reasoning caption $T_{(rl)}$4 that is strictly grounded in $T_{(rl)}$5 and consistent with $T_{(rl)}$6 (Zhang et al., 25 Jun 2026).

Quality control combines manual review and automatic filtering. In a manual pilot, 500 samples from ReasonLite were independently reviewed by 5 graduate students, and prompts were iterated until pass rate exceeded 99.5%. In a post-hoc review, another random 500 samples achieved more than 99.0% pass. Automatic filtering removed any non-English outputs, excessively short captions with fewer than 10 tokens, excessively long captions with more than 60 tokens, and obvious repetition or degeneration; the removal rate was less than 0.01‰.

These procedures matter because the dataset does not rely on rigid categorical annotation. Its reliability instead depends on prompt design, consistency constraints, and downstream filtering. The paper does not provide a closed-form metric for diversity or grounding; instead, caption-length distributions and the low automated removal rate are used as proxy measures of coverage and quality. This suggests that ReasonLite-42M should be understood as a carefully constrained synthetic supervision layer rather than a manually exhaustively labeled reasoning corpus.

4. Data schema and supervision semantics

Each ReasonLite-42M entry is stored as a JSON object with an image pointer, three refined descriptive captions, and a nested list of reasoning captions whose ordering corresponds to the clustered evidence:

$T_{(b)}$7

The semantic constraints of the captions are central to the dataset’s identity. There is no rigid category label in ReasonLite, unlike ReasonPro. Instead, each $T_{(rl)}$7 is open-form but constrained by the prompt to remain single-step, visually verifiable, and factually consistent (Zhang et al., 25 Jun 2026).

A common misconception is that ReasonLite-42M is already a category-structured reasoning dataset. It is not. Its supervision is intentionally open-form, and the category-level structure enters only in Stage 2 through ReasonPro-16M. Another possible misconception is that the dataset contains unrestricted free-form rationales; the paper instead emphasizes short, verifiable, and evidence-grounded captions.

5. Function within the two-stage continual-pretraining framework

ReasonLite-42M is the core resource for Stage 1, termed “Reasoning-Aware Alignment.” In this stage, the model is trained for one epoch on the 42 M ReasonLite pairs mixed with the same 31.2 M refined $T_{(rl)}$8 pairs under a dual-supervision loss that combines descriptive and reasoning signals while regularizing toward the original parameters (Zhang et al., 25 Jun 2026).

The Stage 1 objective is

$T_{(rl)}$9

where $T_{(b)}$0 is the backbone-specific contrastive loss, which is InfoNCE for CLIP; $T_{(b)}$1 is a piecewise-linear schedule; and $T_{(b)}$2 is the weight on $T_{(b)}$3 regularization toward the original parameters $T_{(b)}$4. The schedule is described with the example of starting at 0.6 on $T_{(b)}$5, linearly decaying to 0.3, and then settling at 0.5.

The structure of this objective is significant. It does not replace descriptive supervision with reasoning supervision; it interpolates between them. This suggests that ReasonLite-42M is intended to reshape representational geometry without discarding the descriptive alignment that made CLIP useful for retrieval and transfer in the first place.

In Stage 2, the framework discards $T_{(b)}$6 and trains on 16 M ReasonPro pairs for explicit category discrimination. Even so, the paper characterizes ReasonLite as the bedrock of the model’s initial reasoning awareness.

6. Empirical effects and interpretive significance

The paper reports that ablating Stage 1 alone yields large gains on both retrieval and reasoning benchmarks. For CLIP-ViT-B/32 on COCO-5K image→text R@1, performance increases from 50.0 to 56.2 after Stage 1, corresponding to a gain of +6.2. On RCLIP-Bench at the V3 hardest tier, the reported changes for CLIP-B/32 are 22.3% to 23.6% on Baseline Visual Grounding (V1), 16.2% to 21.3% on Evidence Awareness (V2), and 22.6% to 23.8% on Structured Reasoning (V3) (Zhang et al., 25 Jun 2026).

These results are important for how ReasonLite-42M is interpreted within multimodal representation learning. The reported improvements occur before the application of ReasonPro-16M, which the paper uses for more explicit category-structured reasoning supervision. The authors therefore present ReasonLite-42M as evidence that merely infusing open-form visual reasoning at scale can substantially extend CLIP’s capabilities while also enhancing zero-shot retrieval performance.

The broader ReasonCLIP framework is also described as functioning as a drop-in visual encoder for multimodal LLMs such as LLaVA-NeXT, delivering consistent gains without additional inference cost. Within that broader claim, ReasonLite-42M occupies the role of the initial large-scale reasoning supervision source. A plausible implication is that its main value lies not in producing discrete reasoning labels, but in shifting the encoder toward features that better preserve visually grounded causal, functional, and compositional relations.

Taken together, ReasonLite-42M defines a specific intermediate supervision regime between descriptive pretraining and explicit reasoning classification: large-scale, open-form, single-step commonsense captions anchored to clustered visual evidence. In the ReasonCLIP formulation, that regime is sufficient to improve both representational alignment and reasoning-sensitive evaluation without modifying CLIP’s original architecture.