Churro: Historical Text Recognition Model
- CHURRO is a 3B-parameter open-weight vision-language model engineered for end-to-end page-level transcription of complex historical documents.
- It is fine-tuned on Churro-DS—a unified dataset of over 99,000 pages from 155 corpora spanning 22 centuries, multiple languages, and diverse scripts.
- The model achieves superior accuracy on both printed and handwritten texts while outperforming commercial systems and remaining cost-effective.
Searching arXiv for the specified paper to ground the article. arxiv_search query: (Semnani et al., 24 Sep 2025) CHURRO is both a model and a broader project centered on historical text recognition. In the paper, the model is presented as Churro, a 3B-parameter open-weight vision-LLM (VLM) specialized for reading historical documents, and it is trained on Churro-DS, a large unified dataset of historical page images paired with full-page transcriptions. Its stated goal is to make difficult historical materials—printed and handwritten, multilingual, multi-script, degraded, and often irregular in layout—more readable through page-level image-to-text transcription. The central claim is that a relatively compact open model, when fine-tuned on a sufficiently broad and carefully curated historical dataset, can outperform both open and commercial alternatives on this task while remaining far cheaper to run (Semnani et al., 24 Sep 2025).
1. Historical text recognition as a distinct problem domain
The work positions historical text recognition as materially different from modern OCR. Existing vision-LLMs are described as being designed for modern, standardized texts and therefore as not equipped to read the diverse languages and scripts, irregular layouts, and frequent degradation found in historical materials. The paper identifies several sources of difficulty: visual degradation such as environmental wear, page creases, faded ink, bleed-through, scanning artifacts, extra margins, page curvature, and slight misalignment; layout complexity such as multiple columns, marginal notes, mixed orientations, tables, headers, captions, and nonstandard reading orders; linguistic and orthographic variation across historical forms; script and font diversity including Fraktur and transitional scripts; abbreviation in premodern manuscripts; multilinguality and dead languages; and non-LTR reading conventions, including right-to-left scripts and East Asian vertical layouts (Semnani et al., 24 Sep 2025).
The task formulation is intentionally simplified and made uniform: page-level image-to-text transcription. Each input is a page image, and the output is a single text string representing the page content in correct reading order. This includes handwritten or printed text, tables, captions, headers, main text, and other visible text, while skipping non-text elements. The intended transcription style is diplomatic rather than normalized: the model should reproduce what is on the page, not modernize spelling or expand abbreviations.
This formulation is important because many historical OCR and HTR workflows rely on collection-specific supervised training with fine-grained line- or paragraph-level annotations. The paper presents page-level transcription as a way to use many historical corpora that have page-level or reconstructable transcriptions but lack layout annotations suitable for pipeline OCR. A plausible implication is that the work reframes historical OCR less as detection-plus-recognition and more as end-to-end sequence generation over complex pages.
2. Model definition and technical specification
Churro is not a new architecture from scratch; it is a fine-tuned specialization of Qwen 2.5 VL (3B). The base model is explicitly given as Qwen 2.5 VL 3B Instruct, with model ID Qwen/Qwen2.5-VL-3B-Instruct. The final model is produced by supervised fine-tuning on Churro-DS, and the paper emphasizes that the principal adaptation is domain-specific supervised fine-tuning, not a custom decoding algorithm or OCR-specific module (Semnani et al., 24 Sep 2025).
The paper states that the model was chosen because it is compact, already strong at zero-shot OCR relative to similarly sized models, and can handle high-resolution, variable-sized images, which is important for dense and irregular historical pages. It also states several implementation-level constraints. For fine-tuning, input images are resized to a maximum of 5,120 image patches, each of size pixels. This was chosen so that the total number of image and text tokens per example stays under 25,000. For dataset preprocessing in general, large images were resized to fit within a 2500 × 2500 pixel box while preserving aspect ratio. Decoding uses temperature 0, i.e. greedy decoding, with a maximum generated length of 20,000 tokens for non-reasoning models and 40,000 for reasoning models.
The paper is explicit about several absences. It does not provide a layer-by-layer architectural schematic beyond identifying the base model family. Exact internal vision encoder design, projector type, tokenization internals, and explicit architectural modifications to the cross-modal stack are not specified in the paper. It also does not provide an explicit training loss formula. The text states only that the model was fine-tuned in a supervised fashion. Likewise, the paper does not specify exact prompt formatting during fine-tuning or whether metadata such as language or script labels were injected into the training prompt.
A recurring misconception in multimodal OCR is that performance is primarily a function of parameter count. The paper argues against that view indirectly: Churro’s gains are attributed to specialized training on a broad historical corpus rather than to increased scale or architectural novelty.
3. Churro-DS: corpus design, curation, and representation
Churro-DS is described as the largest and most diverse historical text recognition dataset assembled to date. It unifies 155 historical corpora comprising 99,491 pages, spanning 22 centuries of textual heritage across 46 language clusters, including historical variants and dead languages. Its temporal range is given as the 3rd century BC to the 20th century. The dataset covers three writing directions and 14 scripts from 5 script families, and it is divided into printed and handwritten subsets (Semnani et al., 24 Sep 2025).
Each example contains three elements: a page image, page-level metadata indicating language and script, and a single page-level text string in reading order. Documents include newspapers, books, handwritten diaries, government records, and other genres. The dataset’s stated significance lies in scale, diversity, and task formulation: instead of line-level OCR on a narrow collection, it provides page-level full-text transcriptions across a wide historical and linguistic range.
The inclusion criteria are narrow and explicit. Corpora were included only if the documents originate prior to the mid-20th century, the license permits research use, and the dataset includes human-annotated gold text covering entire pages. Distantly supervised datasets such as VieBookRead were excluded. Character-, word-, and line-level datasets were included only when annotations could be combined into full page text.
The curation pipeline is organized around three goals for each page transcription: correct reading order, high accuracy, and high faithfulness to the image. The authors standardized formats such as ALTO XML, PAGE XML, JSON, and plain text into a single string per page. When explicit reading order was absent, they used dataset-specific heuristics for consistent layouts, a VLM to infer reading order for complex layouts, and manual spot-checking. They report that manual checks indicate more than 98% of reading orders are accurate. For metadata enrichment, the authors used ISO 639-3 for languages and historical variants, grouped them into language clusters, and used ISO 15924 and Glottolog for scripts and script families.
The paper also reports annotation corrections in sixteen datasets. To repair omissions and transcription errors, the authors used Gemini 2.5 Pro to suggest corrections based on image bounding boxes, then reviewed and validated them. For faithfulness, Churro-DS standardizes on diplomatic transcription. Original forms were retained where datasets provided both original and expanded forms, and heavily modernized datasets were excluded where faithful transcription could not be recovered. A VLM was also used, guided by bounding boxes and transcribed text, to restore spacing, line breaks, and diacritics through minimal edits; character-level edit distances between original and edited transcriptions reportedly remained below 5%.
Pages with fewer than 30 tokens, as tokenized by Qwen 2.5 VL, were removed. Near-duplicates were removed using MinHash, eliminating approximately 15,000 near-duplicates. Final dataset sizes are 97,151 training samples, 1,170 validation samples, and 1,170 test samples. Evaluation splits are balanced by language cluster and document type, with 60 pages per language cluster per document type sampled when possible and split evenly into validation and test. Because of data scarcity, only 29 of the 46 language clusters appear in validation and test, while the remaining 17 appear only in training.
The paper stresses severe imbalance. Examples include printed Chinese with only 6 training samples and handwritten Portuguese with only 12 training samples. This imbalance is analytically important because later results tie poor performance on some subsets directly to low training-data availability.
4. Training methodology and benchmark protocol
The training procedure is described as straightforward supervised fine-tuning rather than a multi-stage curriculum or instruction-tuning pipeline. The model was trained for 5 epochs on the Churro-DS training split using 32 NVIDIA H100 GPUs, an effective batch size of 128 with gradient accumulation, a learning rate of , and a cosine schedule. The reported wall-clock training time is approximately 25 hours, and the total compute for the paper, including fine-tuning and inference for all open-weight models, is about 6,000 H100 GPU-hours (Semnani et al., 24 Sep 2025).
The supervision format is the page image paired with the page-level diplomatic transcription. The paper does not describe data augmentation, curriculum learning, instruction tuning, reinforcement learning, or any intermediate pretraining stage beyond supervised fine-tuning. In the limitations, it explicitly notes that the authors only experimented with standard supervised fine-tuning, leaving more advanced methods to future work.
The evaluation protocol is unusually broad. The benchmark includes closed VLMs, open-weight VLMs, OCR systems, a hybrid system, and an oracle upper bound. Closed VLMs include GPT-5, GPT-5 Mini, GPT-5 Nano, GPT-4.1, GPT-4.1 Mini, GPT-4.1 Nano, GPT-4o, GPT-4o Mini, O1, O3, O4 Mini, Claude Sonnet 3.7, Claude Sonnet 4, Claude Opus 4.1, Gemini 2.5 Flash, and Gemini 2.5 Pro. Open-weight VLMs include Qwen 2.5 VL 3B and 72B, Gemma 3 27B, MiMo VL, Nemotron Nano VL, InternVL 3.5, R, Phi 4 Multimodal, and page-level OCR-tuned VLMs such as NuMarkdown, olmOCR, RolmOCR, and Nanonets OCR. OCR systems include Azure OCR and Mistral OCR. The hybrid system is Azure OCR + Gemini 2.5 Pro, where Azure first detects boxes and reading order, then Gemini transcribes box crops individually. DeepSeek-VL2 was excluded because its supported context length was insufficient for the long outputs in Churro-DS.
All VLMs use the same zero-shot prompt and temperature 0, and results are from a single run. The metric is normalized Levenshtein similarity, defined by character-level Levenshtein distance normalized by the length of the longer string and converted to similarity by subtracting from 1. The paper reports separate averages for printed and handwritten documents. Model predictions are normalized using the same text normalization applied in dataset curation, and Arabic-script texts additionally normalize diacritics and hamza using PyArabic.
5. Performance, comparative results, and efficiency
The headline empirical result is that Churro is best overall on the Churro-DS test set. On printed documents, Churro achieves 82.3% normalized Levenshtein similarity, compared with 80.9% for Gemini 2.5 Pro, a margin of +1.4. On handwritten documents, Churro achieves 70.1%, compared with 63.6% for Gemini 2.5 Pro, a margin of +6.5 (Semnani et al., 24 Sep 2025).
Among open-weight zero-shot VLMs, the best printed result is NuMarkdown (8B) at 72.7%, while the best handwritten result is Qwen 2.5 VL 72B at 54.5%. Among OCR systems, Azure OCR scores 71.9% on printed and 47.7% on handwritten text. The base model, Qwen 2.5 VL 3B, scores 67.8% on printed and 42.9% on handwritten pages. The paper therefore attributes a gain of 14.5 points on average on printed pages and 27.2 points on handwritten pages to fine-tuning from the same 3B base model. It also states that Churro outperforms the 3B Qwen 2.5 VL by 21.3% on the full test set in the performance-cost figure and by 4.1% relative to the second-best model on the full test set.
The language-level breakdown reinforces the role of domain adaptation. On printed text, Churro beats Gemini 2.5 Pro on 12 of 18 languages, matches it on Czech, and gains especially on Romanian, Japanese, and Finnish. Reported printed scores include Bulgarian: 96.1, Czech: 95.6, Dutch: 95.7, Hindi: 94.6, Slovenian: 97.6, English: 91.0, German: 82.3, Japanese: 74.1, and Chinese: 6.2. The poor Chinese printed score is explicitly linked to the fact that printed Chinese had only 6 training examples. On handwriting, Churro does better than Gemini 2.5 Pro on 18 of 21 languages, trailing only on Arabic, Persian, and Norwegian. Large fine-tuning gains are reported for Greek: +62.6, Japanese: +54.7, Turkish: +42.3, Hebrew: +42.1, and Persian: +36.8. Reported handwritten scores include Catalan: 90.2, Italian: 88.4, Swedish: 85.4, German: 83.1, English: 84.0, Persian: 78.0, Chinese: 78.2, Hebrew: 42.3, Khmer: 25.7, and Sanskrit: 21.5.
The paper’s efficiency claim is also central. Churro is reported to be 15.5× more cost-effective than Gemini 2.5 Pro. Costs for closed models are measured using the 50% batching discount offered by providers, and costs for open-weight models are estimated following the methodology of olmOCR. The paper does not fully enumerate the underlying dollar-per-page formula in the provided text, but the practical implication is clear: a small open model can be run locally rather than through a paid API. This is particularly salient for archives, libraries, and research groups processing large document collections or operating under privacy constraints.
A frequent assumption in OCR practice is that a hybrid pipeline combining box detection with crop-wise transcription should dominate end-to-end methods. The reported results do not support that assumption here. The Azure OCR + Gemini 2.5 Pro hybrid does not outperform its individual components overall; it trails at least one of its components in every language on printed data and similarly fails to win on handwritten data. The paper interprets this as evidence that decomposition into box-level crops may introduce segmentation and ordering errors rather than solving them.
6. Error profile, limitations, and scholarly significance
The paper includes an error analysis rather than classical architecture or data-scaling ablations. In a sample of 50 predictions, zero-shot Qwen 2.5 VL 3B exhibited major reading-order errors in 42% of cases, whereas Churro exhibited such errors in 16%. A common failure is incorrect column ordering, and East Asian vertical layouts are identified as especially difficult for zero-shot models, which often assume modern left-to-right, top-to-bottom ordering (Semnani et al., 24 Sep 2025).
Hallucination is treated as another major failure mode. In the same sample, 36% of zero-shot Qwen predictions had major hallucinations, while none of Churro’s predictions did. The appendix example of an 18th-century Dutch letter, where a zero-shot model generated plausible but fabricated content such as “Dit is een brief” instead of the actual transcription, is presented as a historically important failure mode: when recognition becomes uncertain, generic VLMs may switch to language modeling and invent content. Minor recurring errors include confusion of visually similar characters, errors in proper names, mistakes caused by page curvature near bindings, and normalization tendencies that produce modernized characters instead of diplomatic forms. Even after curation, the authors found 2 of 50 sampled examples with lingering gold-text omissions inherited from the source data.
The paper also analyzes difficult subsets. Performance generally drops as page text length increases. Historical Chinese is singled out as the most difficult printed subset; even the oracle reaches only 14.4% there. The stated causes are many two-page newspaper images, vertical text, non-rectangular article boundaries, and interspersed Latin characters. For handwriting, the hardest languages for all systems are Sanskrit, Khmer, and Hebrew. Even the oracle ensemble reaches only 26.1 for Sanskrit, 32.9 for Khmer, and 59.1 for Hebrew. The oracle averages, 88.9 for printed and 76.3 for handwritten, indicate substantial residual headroom even if one could select the best system on each page.
The limitations are explicit. Churro-DS is still imbalanced and underrepresents many languages; the authors note that it includes no languages native to the African continent. The work explores only standard supervised fine-tuning. Persistent difficult cases include severely challenging scripts or low-resource languages, very long pages, unusual layouts such as postcards or vertical multicolumn newspapers, heavily degraded scans, documents with small densely packed characters, mixed or transitional scripts, and bound pages causing curvature near the spine. Appendix examples also show failures on postcard reading order, difficult Latin manuscripts with repetitive degeneration, and complex glossed or tabular manuscript layouts.
The project’s broader significance lies in openness and standardization. The paper states that model, dataset, and code are released, that Churro is open-weight, and that Churro-DS is released under CC BY-SA 4.0. It also states that this release includes the first publicly available historical OCR datasets for Persian and Turkish, and specifically mentions an Ottoman Turkish HTR dataset transcribed by the authors. For cultural-heritage institutions and digital-humanities research, the release provides both a deployable system for page-level first-draft transcriptions and a common benchmark spanning many languages, scripts, and centuries. This suggests that historical OCR should be treated as a distinct, under-served problem in multimodal document understanding, and that open, domain-specific VLMs can advance it without relying on maximal scale alone.