---
title: 'Churro: Historical Text Recognition Model'
url: https://www.emergentmind.com/topics/churro
type: topic
---

# Churro: Historical Text Recognition Model

Searching arXiv for the specified paper to ground the article.
arxiv_search query: 2509.19768
CHURRO is both a model and a broader project centered on historical text recognition. In the paper, the model is presented as **Churro**, a **3B-parameter open-weight vision-language model (VLM)** specialized for reading historical documents, and it is trained on **Churro-DS**, a large unified dataset of historical page images paired with full-page transcriptions. Its stated goal is to make difficult historical materials—printed and handwritten, multilingual, multi-script, degraded, and often irregular in layout—more readable through page-level image-to-text transcription. The central claim is that a relatively compact open model, when fine-tuned on a sufficiently broad and carefully curated historical dataset, can outperform both open and commercial alternatives on this task while remaining far cheaper to run [2509.19768].

## 1. Historical text recognition as a distinct problem domain

The work positions historical text recognition as materially different from modern OCR. Existing vision-language models are described as being designed for modern, standardized texts and therefore as not equipped to read the diverse languages and scripts, irregular layouts, and frequent degradation found in historical materials. The paper identifies several sources of difficulty: **visual degradation** such as environmental wear, page creases, faded ink, bleed-through, scanning artifacts, extra margins, page curvature, and slight misalignment; **layout complexity** such as multiple columns, marginal notes, mixed orientations, tables, headers, captions, and nonstandard reading orders; **linguistic and orthographic variation** across historical forms; **script and font diversity** including Fraktur and transitional scripts; **abbreviation** in premodern manuscripts; **multilinguality and dead languages**; and **non-LTR reading conventions**, including right-to-left scripts and East Asian vertical layouts [2509.19768].

The task formulation is intentionally simplified and made uniform: **page-level image-to-text transcription**. Each input is a page image, and the output is a **single text string representing the page content in correct reading order**. This includes handwritten or printed text, tables, captions, headers, main text, and other visible text, while skipping non-text elements. The intended transcription style is **diplomatic** rather than normalized: the model should reproduce what is on the page, not modernize spelling or expand abbreviations.

This formulation is important because many historical OCR and HTR workflows rely on collection-specific supervised training with fine-grained line- or paragraph-level annotations. The paper presents page-level transcription as a way to use many historical corpora that have page-level or reconstructable transcriptions but lack layout annotations suitable for pipeline OCR. A plausible implication is that the work reframes historical OCR less as detection-plus-recognition and more as end-to-end sequence generation over complex pages.

## 2. Model definition and technical specification

Churro is not a new architecture from scratch; it is a **fine-tuned specialization of Qwen 2.5 VL (3B)**. The base model is explicitly given as **Qwen 2.5 VL 3B Instruct**, with model ID `Qwen/Qwen2.5-VL-3B-Instruct`. The final model is produced by supervised fine-tuning on Churro-DS, and the paper emphasizes that the principal adaptation is **domain-specific supervised fine-tuning**, not a custom decoding algorithm or OCR-specific module [2509.19768].

The paper states that the model was chosen because it is compact, already strong at zero-shot OCR relative to similarly sized models, and can handle **high-resolution, variable-sized images**, which is important for dense and irregular historical pages. It also states several implementation-level constraints. For fine-tuning, input images are resized to a maximum of **5,120 image patches**, each of size **\(28 \times 28\)** pixels. This was chosen so that the total number of image and text tokens per example stays **under 25,000**. For dataset preprocessing in general, large images were resized to fit within a **2500 × 2500 pixel** box while preserving aspect ratio. Decoding uses **temperature 0**, i.e. **greedy decoding**, with a maximum generated length of **20,000** tokens for non-reasoning models and **40,000** for reasoning models.

The paper is explicit about several absences. It does **not** provide a layer-by-layer architectural schematic beyond identifying the base model family. Exact internal vision encoder design, projector type, tokenization internals, and explicit architectural modifications to the cross-modal stack are **not specified in the paper**. It also does **not** provide an explicit training loss formula. The text states only that the model was fine-tuned in a supervised fashion. Likewise, the paper does not specify exact prompt formatting during fine-tuning or whether metadata such as language or script labels were injected into the training prompt.

A recurring misconception in multimodal OCR is that performance is primarily a function of parameter count. The paper argues against that view indirectly: Churro’s gains are attributed to specialized training on a broad historical corpus rather than to increased scale or architectural novelty.

## 3. Churro-DS: corpus design, curation, and representation

Churro-DS is described as the largest and most diverse historical text recognition dataset assembled to date. It unifies **155 historical corpora** comprising **99,491 pages**, spanning **22 centuries** of textual heritage across **46 language clusters**, including historical variants and dead languages. Its temporal range is given as the **3rd century BC to the 20th century**. The dataset covers **three writing directions** and **14 scripts** from **5 script families**, and it is divided into **printed** and **handwritten** subsets [2509.19768].

Each example contains three elements: a page image, page-level metadata indicating language and script, and a single page-level text string in reading order. Documents include **newspapers, books, handwritten diaries, government records**, and other genres. The dataset’s stated significance lies in scale, diversity, and task formulation: instead of line-level OCR on a narrow collection, it provides **page-level full-text transcriptions** across a wide historical and linguistic range.

The inclusion criteria are narrow and explicit. Corpora were included only if the documents originate prior to the mid-20th century, the license permits research use, and the dataset includes human-annotated gold text covering entire pages. Distantly supervised datasets such as VieBookRead were excluded. Character-, word-, and line-level datasets were included only when annotations could be combined into full page text.

The curation pipeline is organized around three goals for each page transcription: correct **reading order**, high **accuracy**, and high **faithfulness** to the image. The authors standardized formats such as **ALTO XML, PAGE XML, JSON, and plain text** into a single string per page. When explicit reading order was absent, they used dataset-specific heuristics for consistent layouts, a VLM to infer reading order for complex layouts, and manual spot-checking. They report that manual checks indicate **more than 98% of reading orders are accurate**. For metadata enrichment, the authors used **ISO 639-3** for languages and historical variants, grouped them into **language clusters**, and used **ISO 15924** and **Glottolog** for scripts and script families.

The paper also reports annotation corrections in **sixteen datasets**. To repair omissions and transcription errors, the authors used **Gemini 2.5 Pro** to suggest corrections based on image bounding boxes, then reviewed and validated them. For faithfulness, Churro-DS standardizes on **diplomatic transcription**. Original forms were retained where datasets provided both original and expanded forms, and heavily modernized datasets were excluded where faithful transcription could not be recovered. A VLM was also used, guided by bounding boxes and transcribed text, to restore spacing, line breaks, and diacritics through **minimal edits**; character-level edit distances between original and edited transcriptions reportedly remained **below 5%**.

Pages with fewer than **30 tokens**, as tokenized by Qwen 2.5 VL, were removed. Near-duplicates were removed using **MinHash**, eliminating approximately **15,000 near-duplicates**. Final dataset sizes are **97,151 training samples**, **1,170 validation samples**, and **1,170 test samples**. Evaluation splits are balanced by language cluster and document type, with **60 pages per language cluster per document type** sampled when possible and split evenly into validation and test. Because of data scarcity, only **29 of the 46 language clusters** appear in validation and test, while the remaining **17** appear only in training.

The paper stresses severe imbalance. Examples include printed Chinese with only **6 training samples** and handwritten Portuguese with only **12 training samples**. This imbalance is analytically important because later results tie poor performance on some subsets directly to low training-data availability.

## 4. Training methodology and benchmark protocol

The training procedure is described as straightforward supervised fine-tuning rather than a multi-stage curriculum or instruction-tuning pipeline. The model was trained for **5 epochs** on the Churro-DS training split using **32 NVIDIA H100 GPUs**, an effective batch size of **128** with gradient accumulation, a learning rate of **\(5 \times 10^{-5}\)**, and a **cosine** schedule. The reported wall-clock training time is approximately **25 hours**, and the total compute for the paper, including fine-tuning and inference for all open-weight models, is about **6,000 H100 GPU-hours** [2509.19768].

The supervision format is the page image paired with the page-level diplomatic transcription. The paper does not describe data augmentation, curriculum learning, instruction tuning, reinforcement learning, or any intermediate pretraining stage beyond supervised fine-tuning. In the limitations, it explicitly notes that the authors **only experimented with standard supervised fine-tuning**, leaving more advanced methods to future work.

The evaluation protocol is unusually broad. The benchmark includes closed VLMs, open-weight VLMs, OCR systems, a hybrid system, and an oracle upper bound. Closed VLMs include GPT-5, GPT-5 Mini, GPT-5 Nano, GPT-4.1, GPT-4.1 Mini, GPT-4.1 Nano, GPT-4o, GPT-4o Mini, O1, O3, O4 Mini, Claude Sonnet 3.7, Claude Sonnet 4, Claude Opus 4.1, Gemini 2.5 Flash, and Gemini 2.5 Pro. Open-weight VLMs include Qwen 2.5 VL 3B and 72B, Gemma 3 27B, MiMo VL, Nemotron Nano VL, InternVL 3.5, R, Phi 4 Multimodal, and page-level OCR-tuned VLMs such as NuMarkdown, olmOCR, RolmOCR, and Nanonets OCR. OCR systems include **Azure OCR** and **Mistral OCR**. The hybrid system is **Azure OCR + Gemini 2.5 Pro**, where Azure first detects boxes and reading order, then Gemini transcribes box crops individually. DeepSeek-VL2 was excluded because its supported context length was insufficient for the long outputs in Churro-DS.

All VLMs use the same zero-shot prompt and **temperature 0**, and results are from a **single run**. The metric is **normalized Levenshtein similarity**, defined by character-level Levenshtein distance normalized by the length of the longer string and converted to similarity by subtracting from 1. The paper reports separate averages for printed and handwritten documents. Model predictions are normalized using the same text normalization applied in dataset curation, and Arabic-script texts additionally normalize diacritics and hamza using **PyArabic**.

## 5. Performance, comparative results, and efficiency

The headline empirical result is that Churro is best overall on the Churro-DS test set. On **printed** documents, Churro achieves **82.3%** normalized Levenshtein similarity, compared with **80.9%** for **Gemini 2.5 Pro**, a margin of **+1.4**. On **handwritten** documents, Churro achieves **70.1%**, compared with **63.6%** for Gemini 2.5 Pro, a margin of **+6.5** [2509.19768].

Among open-weight zero-shot VLMs, the best printed result is **NuMarkdown (8B)** at **72.7%**, while the best handwritten result is **Qwen 2.5 VL 72B** at **54.5%**. Among OCR systems, **Azure OCR** scores **71.9%** on printed and **47.7%** on handwritten text. The base model, **Qwen 2.5 VL 3B**, scores **67.8%** on printed and **42.9%** on handwritten pages. The paper therefore attributes a gain of **14.5 points on average** on printed pages and **27.2 points** on handwritten pages to fine-tuning from the same 3B base model. It also states that Churro outperforms the 3B Qwen 2.5 VL by **21.3%** on the full test set in the performance-cost figure and by **4.1%** relative to the second-best model on the full test set.

The language-level breakdown reinforces the role of domain adaptation. On printed text, Churro beats Gemini 2.5 Pro on **12 of 18 languages**, matches it on Czech, and gains especially on Romanian, Japanese, and Finnish. Reported printed scores include **Bulgarian: 96.1**, **Czech: 95.6**, **Dutch: 95.7**, **Hindi: 94.6**, **Slovenian: 97.6**, **English: 91.0**, **German: 82.3**, **Japanese: 74.1**, and **Chinese: 6.2**. The poor Chinese printed score is explicitly linked to the fact that printed Chinese had only **6 training examples**. On handwriting, Churro does better than Gemini 2.5 Pro on **18 of 21 languages**, trailing only on Arabic, Persian, and Norwegian. Large fine-tuning gains are reported for **Greek: +62.6**, **Japanese: +54.7**, **Turkish: +42.3**, **Hebrew: +42.1**, and **Persian: +36.8**. Reported handwritten scores include **Catalan: 90.2**, **Italian: 88.4**, **Swedish: 85.4**, **German: 83.1**, **English: 84.0**, **Persian: 78.0**, **Chinese: 78.2**, **Hebrew: 42.3**, **Khmer: 25.7**, and **Sanskrit: 21.5**.

The paper’s efficiency claim is also central. Churro is reported to be **15.5× more cost-effective** than Gemini 2.5 Pro. Costs for closed models are measured using the **50% batching discount** offered by providers, and costs for open-weight models are estimated following the methodology of **olmOCR**. The paper does not fully enumerate the underlying dollar-per-page formula in the provided text, but the practical implication is clear: a small open model can be run locally rather than through a paid API. This is particularly salient for archives, libraries, and research groups processing large document collections or operating under privacy constraints.

A frequent assumption in OCR practice is that a hybrid pipeline combining box detection with crop-wise transcription should dominate end-to-end methods. The reported results do not support that assumption here. The **Azure OCR + Gemini 2.5 Pro** hybrid does **not** outperform its individual components overall; it trails at least one of its components in every language on printed data and similarly fails to win on handwritten data. The paper interprets this as evidence that decomposition into box-level crops may introduce segmentation and ordering errors rather than solving them.

## 6. Error profile, limitations, and scholarly significance

The paper includes an error analysis rather than classical architecture or data-scaling ablations. In a sample of 50 predictions, zero-shot **Qwen 2.5 VL 3B** exhibited major reading-order errors in **42%** of cases, whereas **Churro** exhibited such errors in **16%**. A common failure is incorrect column ordering, and East Asian vertical layouts are identified as especially difficult for zero-shot models, which often assume modern left-to-right, top-to-bottom ordering [2509.19768].

Hallucination is treated as another major failure mode. In the same sample, **36%** of zero-shot Qwen predictions had **major hallucinations**, while **none** of Churro’s predictions did. The appendix example of an 18th-century Dutch letter, where a zero-shot model generated plausible but fabricated content such as “Dit is een brief” instead of the actual transcription, is presented as a historically important failure mode: when recognition becomes uncertain, generic VLMs may switch to language modeling and invent content. Minor recurring errors include confusion of visually similar characters, errors in proper names, mistakes caused by page curvature near bindings, and normalization tendencies that produce modernized characters instead of diplomatic forms. Even after curation, the authors found **2 of 50** sampled examples with lingering gold-text omissions inherited from the source data.

The paper also analyzes difficult subsets. Performance generally drops as page text length increases. Historical Chinese is singled out as the most difficult printed subset; even the oracle reaches only **14.4%** there. The stated causes are many two-page newspaper images, vertical text, non-rectangular article boundaries, and interspersed Latin characters. For handwriting, the hardest languages for all systems are **Sanskrit**, **Khmer**, and **Hebrew**. Even the oracle ensemble reaches only **26.1** for Sanskrit, **32.9** for Khmer, and **59.1** for Hebrew. The oracle averages, **88.9** for printed and **76.3** for handwritten, indicate substantial residual headroom even if one could select the best system on each page.

The limitations are explicit. Churro-DS is still **imbalanced** and underrepresents many languages; the authors note that it includes **no languages native to the African continent**. The work explores only **standard supervised fine-tuning**. Persistent difficult cases include severely challenging scripts or low-resource languages, very long pages, unusual layouts such as postcards or vertical multicolumn newspapers, heavily degraded scans, documents with small densely packed characters, mixed or transitional scripts, and bound pages causing curvature near the spine. Appendix examples also show failures on postcard reading order, difficult Latin manuscripts with repetitive degeneration, and complex glossed or tabular manuscript layouts.

The project’s broader significance lies in openness and standardization. The paper states that **model, dataset, and code are released**, that Churro is **open-weight**, and that Churro-DS is released under **CC BY-SA 4.0**. It also states that this release includes the **first publicly available historical OCR datasets for Persian and Turkish**, and specifically mentions an **Ottoman Turkish HTR dataset transcribed by the authors**. For cultural-heritage institutions and digital-humanities research, the release provides both a deployable system for **page-level first-draft transcriptions** and a common benchmark spanning many languages, scripts, and centuries. This suggests that historical OCR should be treated as a distinct, under-served problem in multimodal document understanding, and that open, domain-specific VLMs can advance it without relying on maximal scale alone.

Source: https://www.emergentmind.com/topics/churro