---
title: 'CoCo-10K: Dual-Domain Dataset Overview'
url: https://www.emergentmind.com/topics/coco-10k
type: topic
---

# CoCo-10K: Dual-Domain Dataset Overview

In the literature supplied here, **CoCo-10K** has two explicit dataset-level meanings. In speech-and-vision research, it denotes the **10-hour / 10,000-spoken-caption portion** of **Speech-COCO** used for a preliminary study on **Unsupervised Term Discovery (UTD)**. In text-to-image generation, it denotes a **curated dataset of over 10K structured supervision instances** used to train **CoCo (Code-as-CoT)**, a framework in which a model generates **executable code**, renders a deterministic **draft image**, and then performs **draft-guided refinement** to obtain the final image [1707.08435][2603.08652].

## 1. Disambiguation and naming

The supplied literature suggests that the string **“CoCo-10K”** is polysemous rather than canonical. Its two exact uses are dataset-specific and arise in unrelated technical domains.

| Context | Meaning | Supervision form |
|---|---|---|
| Speech-COCO | 10-hour / 10,000-spoken-caption subset | speech aligned to image and caption text |
| CoCo (Code-as-CoT) | curated dataset of over 10K structured supervision instances | Text–Code pairs and Text–Draft Image–Final Image triplets |

The first usage is tied to the augmentation of **MSCOCO** with synthetic spoken captions and is explicitly a **small, early-use subset** of a much larger corpus. The second usage is tied to a **two-stage T2I pipeline** and is explicitly not a generic image-caption dataset; its supervision targets executable structure and draft-preserving visual refinement. A central source of confusion is that both usages inherit the surface form “CoCo-10K” while differing in modality, annotation granularity, and downstream task family.

## 2. CoCo-10K as a subset of Speech-COCO

Within **Speech-COCO**, CoCo-10K is the **subset of Speech-COCO used in the preliminary study**. The paper describes it as **about 10 hours of speech** and **about 10,000 spoken captions**, and emphasizes that it is **not a separate dataset** but a **slice of Speech-COCO** chosen for initial experimentation on **unsupervised term discovery**. Its role is to provide a **tractable scale** for early multimodal and unsupervised speech-learning experiments before scaling to the full corpus [1707.08435].

Speech-COCO itself is an augmentation of **MSCOCO** in which speech is added to image and text. The construction starts from the **MSCOCO training and validation sets**; each image has **at least five human-written captions**, and those captions are synthesized into spoken utterances using **Voxygen** TTS. The resulting corpus contains **123,287 images**, **616,767 captions**, and **more than 600 hours** of speech overall. The paper reports **8 voices**—**4 UK voices** (*Paul, Elizabeth, Judith, Bronwen*) and **4 US voices** (*Phil, Bruce, Amanda, Jenny*)—and notes that the synthesis is **corpus-based concatenative synthesis**, not parametric synthesis. For the full corpus, the reported average caption length is **10.79 tokens** and the average WAV duration is **3.52 seconds**.

In this setting, CoCo-10K functions as a **proof-of-concept subset**. The paper’s stated rationale is not merely computational convenience. The subset is intended to test whether the corpus is already **hard enough for discovery**, whether it can serve as a first benchmark for **multimodal learning**, and whether visually grounded synthetic speech can support **speech unit discovery**.

## 3. Annotation regime, perturbation, and UTD evaluation in the Speech-COCO setting

A defining property of Speech-COCO, inherited by CoCo-10K, is that alignment is **not produced by forced alignment after the fact**. Instead, the annotations are created **during TTS synthesis**. Each spoken-caption WAV file is paired with a **JSON** file containing metadata and timing information with the structure `duration`, `speaker`, `synthesisedCaption`, `timecode`, `speed`, `wavFilename`, `captionID`, `imgID`, and `disfluency`. The `timecode` annotations are available at **word level**, **syllable level**, and **phoneme level**, and the accompanying Python tooling can parse these annotations and convert them to **Praat TextGrid** format [1707.08435].

The corpus introduces two explicit forms of variability. First, **speed perturbation** is applied using **Sox tempo**, with values **1.1**, **0.9**, and **1.0**, each chosen with equal probability; this is intended to introduce **intra-speaker variability**. Second, **disfluencies** are inserted to make the synthesized captions sound more natural. The fillers listed are **“um”**, **“uh”**, **“er”**, **“huh”**, **“oh”**, and **“ah”**. Each caption has a **0.3 probability** of receiving a filler, and the insertion position is **beginning**, **middle**, or **end** with equal probability.

The paper’s preliminary experiment uses CoCo-10K for **UTD** with **ZRTools**, which applies **segmental dynamic time warping (SDTW)** to find acoustically similar segments and then uses graph clustering to form discovered unit clusters. Evaluation uses the **Term Discovery Evaluation (TDE)** toolkit, which reports **matching**, **clustering**, and **parsing**, together with **precision \(P\)**, **recall \(R\)**, **F-score \(F\)**, **NED**, and **coverage**. A generic F-score is given as
$$
F = \frac{2PR}{P+R}.
$$
The authors conclude that CoCo-10K is challenging for UTD because **precision is low**, **recall is even lower**, and clusters often reflect **same-speaker similarity** rather than true lexical identity. At the same time, the system does recover useful recurrent patterns such as **“a man riding”**, **“fire hydrant”**, **“skiing down”**, and **“a bunch of bananas.”**

## 4. CoCo-10K as a structured supervision corpus for Code-as-CoT

In the **CoCo (Code-as-CoT)** framework, CoCo-10K is a **curated dataset of over 10K structured supervision instances**. It is explicitly **not a generic image-caption dataset**. Instead, it is designed to supervise two distinct capabilities: **structured draft construction** via code generation and **corrective visual refinement** from draft to final image. Its two reported supervision formats are **Text–Code pairs** and **Text–Draft Image–Final Image triplets** [2603.08652].

The dataset was created in response to two concrete failure modes observed in a zero-shot pilot study on **Bagel**. First, **code generation was unreliable** because the model often produced **non-executable** code, which prevented draft rendering. Second, **drafts were ignored during refinement** because the model often generated a new image instead of editing the draft. CoCo-10K therefore provides explicit training signal for producing **valid, executable code** and for learning to **preserve and refine** a rendered draft rather than overwrite it.

The resulting generation pipeline is summarized as
$$
p \rightarrow c \rightarrow I_d \rightarrow I_f,
$$
where \(p\) is the text prompt, \(c\) is executable code, \(I_d\) is the draft image produced by sandbox execution, and \(I_f\) is the final image after refinement. The paper treats the sandbox as important because it makes the reasoning process **safe**, **deterministic**, and **verifiable**.

## 5. Construction pipeline, training role, and empirical effects in the Code-as-CoT setting

CoCo-10K is built through two pipelines: an **Editing Dataset** and a **Synthesis Dataset**. The Editing Dataset is derived from **StructVisuals** and contains structured chart images with an **A-Image** and a **B-Image**. The corrections involve changing **data values**, fixing **labels**, or adjusting **formatting**, while keeping the overall chart structure intact. The Synthesis Dataset is aimed at **structured visual generation**. Its automated process synthesizes prompts for **scientific concepts**, **charts**, **posters**, **infographics**, **annotated diagrams**, and **other text-intensive visual formats**; uses **Gemini-3-Pro** to generate code specifying **visual layout**, **object positions**, **textual placement**, and **structural organization**; executes that code in a **sandboxed environment** to produce the draft **A-Image**; and then uses **Nano Banana** to refine the draft into the final **B-Image** [2603.08652].

The paper states that supervised fine-tuning begins from **Bagel** and uses **token-level cross-entropy loss** on code and **MSE loss** on the **VAE tokens** for the final image. Training proceeds in two phases: **text-to-code fine-tuning** first, followed by **full-parameter fine-tuning for 16K steps**. The authors also report that the collected data are organized exactly around the pipeline’s intermediate representations, which is the key difference from ordinary T2I datasets that pair only text with a target image.

The empirical effect of this dataset is reported indirectly through the performance of the full CoCo system. On **StructT2IBench**, CoCo achieves **73.52% overall accuracy**, compared to **49.58%** for **GPT-Image**, and the abstract reports improvements of **+68.83%** over direct generation. The abstract also reports improvements of **+54.8%** on **OneIG-Bench** and **+41.23%** on **LongText-Bench**. On text-heavy benchmarks, the paper reports **0.853** overall on **OneIG-Bench** and **0.754** overall on **LongText-Bench**. The ablations further attribute a central role to the dataset design: without CoCo-style training, **Bagel** achieves only a **9.06% compilation success rate** on **LongText-Bench**, whereas CoCo fine-tuning reaches **100% compilation success**. The mixture-ratio study reports that a small amount of Text–Code supervision is sufficient to induce structured reasoning, while the bulk of training is more effective when allocated to draft-to-final refinement.

## 6. Non-equivalent usages in adjacent literatures

Several nearby papers in the supplied literature are relevant precisely because they **do not** define a dataset literally named **CoCo-10K**. The foundational **MS COCO** paper defines **Microsoft Common Objects in COntext** and its official train/validation/test releases, but it does **not** mention a subset explicitly named **“CoCo-10K”**; it also reports that the dataset includes **five written captions per image**, which later enabled Speech-COCO-style augmentation [1405.0312].

Likewise, **COCONut** modernizes COCO segmentation and introduces a dataset with **383K images** and **more than 5.18M panoptic masks**, but the paper states that it does **not** define a dataset called **“CoCo-10K”** exactly. Its closest named smaller-scale subsets are **relabeled COCO-val** with **5K images, 67K masks** and **COCONut-val** with **25K images, 437K masks** [2404.08639]. **COCO-ReM** is similarly COCO-derived and manually verified, but the paper states that it does **not** define anything named **“CoCo-10K”**; its validation split contains **40,689 instances**, not a 10K-branded subset [2403.18819].

Outside COCO-style vision benchmarks, the label can drift further. In long-sequence recommendation, the Douyin paper states that **CoCo-10K is not a benchmark name, dataset, or separate model/system**; rather, the paper concerns a **10k-sequence production setting** [2511.06077]. In KID readout electronics, the exact term is **not named explicitly**, but the paper is described as addressing the same **10 mW/pixel** milestone that “CoCo-10K” appears to denote in that community. This suggests an informal detector-readout usage centered on a **10,000-pixel-class kinetic inductance detector readout target** rather than a COCO-derived dataset [2305.00928].

The practical implication is that **CoCo-10K** should not be read as a stable universal identifier. In the supplied literature, it denotes either a **pilot-scale visually grounded speech subset** or a **structured supervision corpus for code-driven T2I**, while several adjacent papers use related COCO naming, 10K-sized regimes, or 10K-scale milestones without introducing an exact dataset of that name.

Source: https://www.emergentmind.com/topics/coco-10k