PastReader: Historical Spanish OCR Task

• PastReader is a shared task focused on transcribing degraded historical Spanish texts using OCR techniques on challenging archival materials.
• It comprises two subtasks: cleaning up noisy OCR outputs and performing end-to-end transcription directly from PDF page images.
• The evaluation compares distinct OCR paradigms using metrics like Levenshtein, WER, and BLEU to assess literal accuracy and semantic preservation.

Searching arXiv for papers directly about “PastReader” to ground the article in the relevant literature. PastReader is an IberLEF 2025 shared task on transcribing texts from the past, centered on OCR for historical Spanish newspaper and periodical pages whose visual and documentary properties make them difficult to process automatically. In the reported participation study by the GRESEL team, the task is framed as a realistic historical-document transcription problem involving degraded scans, mixed layouts, illustrations, stains, low contrast, and decorative typography, and is approached through a comparative evaluation of three distinct paradigms: a historical OCR platform, a conventional OCR engine, and a compact multimodal model (Torterolo-Orta et al., 7 Jul 2025).

1. Task definition and operational scope

PastReader is organized around two subtasks. Task 1 is clean OCR post-correction, in which participants are given intentionally noisy OCR outputs in plain text and are asked to clean them so that they better match the ground truth. Task 2 is end-to-end OCR from PDF files, in which participants ignore the provided OCR text and instead produce transcription directly from the PDF page images. The GRESEL study participated only in Task 2, primarily because of time constraints and because the comparison of complete OCR pipelines was a central objective (Torterolo-Orta et al., 7 Jul 2025).

The task is defined by the difficulty of historical Spanish print culture rather than by a single document genre or a narrowly controlled image condition. The source material comprises historical Spanish newspaper and periodical pages whose visual state and editorial structure introduce OCR failure modes that are not incidental but constitutive of the benchmark. The paper emphasizes degraded scans, mixed layouts, illustrations, stains, low contrast, decorative typography, handwritten marginalia, stamps, and non-linear reading order. A plausible implication is that PastReader functions not merely as an OCR benchmark, but as a stress test for the interaction among document layout analysis, transcription fidelity, and historically variable orthography.

A recurrent misconception would be to treat PastReader as a standard OCR task on noisy but otherwise regular pages. The reported evidence argues against that simplification: pages may contain little or no text, images interrupting text flow, decorative elements, or complex layout structures that disrupt conventional reading order. This makes the task materially different from OCR on modern, clean text.

2. Documentary basis and corpus heterogeneity

The corpus used in the GRESEL report consists of isolated pages from 8 periodical publications from the 19th and 20th centuries. The publications span a wide range of genres, including general news and cultural magazines, satirical/humorous magazines, spiritualist chronicles, serial fiction and poetry, fashion/feminine customs, and scientific/medical publications. The resulting variation is multidimensional: the pages differ in font styles, layout complexity, paper color/background tone, print quality, stains and degradation, handwritten marginalia, stamps and illustrations, and decorative initials and small caps (Torterolo-Orta et al., 7 Jul 2025).

This heterogeneity is methodologically important because it constrains what can be inferred from aggregate OCR scores. A system may perform adequately on ordinary running text while failing on ornate pages, columnar layouts, or documents with sparse textual content. The paper explicitly argues that robust performance would likely require a carefully curated historical Spanish training set with philological and paleographic transcription guidelines, which had not yet been created for the project. This suggests that PastReader is as much a corpus-design and editorial-normalization problem as it is a model-selection problem.

The shared task therefore occupies a space between document analysis and historical text scholarship. Its practical stakes include not only recognition accuracy, but also preservation of original lineation, orthographic form, and layout-sensitive reading order.

3. Evaluation protocol and metric structure

The shared-task evaluation used eight metrics, although the organizers had initially planned six. The final metric set comprised Levenshtein distance, WER, NED, BLEU, ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-LSum (Torterolo-Orta et al., 7 Jul 2025).

Metric family	Metrics	Interpretation in the report
Literal similarity	Levenshtein distance, WER, NED	Lower is better; rewards exact character/word reproduction
Overlap / semantic preservation	BLEU, ROUGE-1, ROUGE-2, ROUGE-L, ROUGE-LSum	Higher is better; measures broader textual or semantic overlap

The paper characterizes Levenshtein distance, WER, and NED as measures of literal similarity, and BLEU and ROUGE as measures of overlap and broader textual or semantic preservation. No explicit formulas are printed in the report, but the interpretations are standard: WER reflects substitution, insertion, and deletion at the word level; Levenshtein counts edit operations; NED normalizes edit distance by length; and BLEU/ROUGE assess n-gram overlap and recall-oriented similarity.

One of the clearest methodological findings is that evaluation is highly sensitive to formatting decisions. In the Transkribus runs, preserving line breaks improved literal/precision-oriented metrics, whereas joining lines improved semantic metrics. This demonstrates that OCR quality in PastReader cannot be interpreted independently of output normalization and segmentation policy.

4. Compared system paradigms

The GRESEL report evaluates three OCR paradigms under consumer-grade hardware constraints: Transkribus, Tesseract, and Granite3.2-vision:2b. The study is explicitly presented as a comparative exploration rather than as the introduction of a single new system (Torterolo-Orta et al., 7 Jul 2025).

Submitted run	Method	Configuration
GRESEL1_run1 / GRESEL2_run2	Transkribus	Historical Spanish model Coloso Español; line breaks preserved vs. joined
GRESEL1_run2 / GRESEL1_run3	Tesseract 5.5.0	Baseline vs. fine-tuned on task data
GRESEL2_run1	Granite3.2-vision:2b	Fine-tuned with QLoRA

Transkribus

The Transkribus experiment used the historical Spanish model Coloso Español in an inference-only setting. The workflow was to submit PDF pages, obtain OCR output, export two variants—one preserving line breaks and one joining lines—and convert them to the required TXT format. The paper notes that Transkribus is attractive for historical documents because it supports layout annotation, collaborative transcription, and model training.

Tesseract

The Tesseract experiment used Tesseract 5.5.0 on Windows, with one off-the-shelf baseline and one fine-tuned variant. The training workflow included renaming transcripts to the .gt.txt convention, converting PDFs to TIFF at roughly 300 DPI, generating .box files, creating .lstmf files with --psm 6 and --oem 1, building list.txt, training from spa.traineddata, and packaging with combine_tessdata. Development experiments used subsets of 100, 1,000, 2,000, and the full dataset. Despite this, the non-fine-tuned baseline consistently performed best during development.

Granite3.2-vision:2b

The multimodal system used Granite3.2-vision:2b, fine-tuned with QLoRA, 4-bit NF4 quantization, double quantization, bfloat16 computation, LoRA adapters on the projection layers, gradient checkpointing, batch size 1, and gradient accumulation 8. The reported hardware was an Nvidia RTX 5080, 16 GB VRAM, and 32 GB RAM. Because full-resolution images exhausted memory, pages were resized to approximately 414 × 585 pixels, preserving aspect ratio and padding to a fixed size. The model was prompted in a chat-style format with a system prompt instructing exact extraction, no correction or interpretation, no invented content, and only raw text output.

5. Reported results and observed failure modes

The Transkribus runs provide the sharpest evidence for metric-format interaction. GRESEL1_run1, which preserved line breaks, performed better on Levenshtein: 105.1823, WER: 0.2933, and NED: 0.0356. GRESEL2_run2, which joined lines, performed better on BLEU: 0.6949, ROUGE-1: 0.8686, ROUGE-2: 0.7914, ROUGE-L: 0.8626, and ROUGE-LSum: 0.8643 (Torterolo-Orta et al., 7 Jul 2025). The immediate significance is that output formatting is not a peripheral implementation detail; it changes the apparent ranking of systems depending on the metric family.

The Tesseract results show that fine-tuning did not uniformly improve performance. On the official test set, the baseline was better in 6 of 8 metrics. The fine-tuned version improved only the character-level measures, reaching Levenshtein: 89.1427 and NED: 0.0302, while the baseline had better WER: 0.3650 vs. 0.3846, BLEU: 0.6229 vs. 0.6220, ROUGE-1: 0.8306 vs. 0.8232, ROUGE-2: 0.7114 vs. 0.6907, ROUGE-L: 0.8256 vs. 0.8180, and ROUGE-LSum: 0.8299 vs. 0.8226. The paper interprets this as a likely indication of overfitting or of limited usefulness of the fine-tuning setup on a highly heterogeneous historical corpus.

The Granite system was the strongest overall submission from the study in many respects. GRESEL2_run1 achieved Levenshtein: 97.2399, WER: 0.2643, NED: 0.0330, BLEU: 0.6890, ROUGE-1: 0.8841, ROUGE-2: 0.8049, ROUGE-L: 0.8806, and ROUGE-LSum: 0.8837. According to the report, this corresponded to 2nd place in WER and all ROUGE metrics, and 3rd place in BLEU, while remaining weaker on character-level accuracy than the best Tesseract or Transkribus scores.

The qualitative error analysis is equally important. Both Tesseract variants struggled especially with pages that contained almost no text, had low contrast, had diagram-like or unusual layouts, or included decorative elements. Granite showed a different profile: it tended to join hyphenated line breaks, sometimes “corrected” historical spellings into modern Spanish forms, and could hallucinate when page regions were unclear. It handled blank pages safely by not generating text, ignored a stamp reading “BIBLIOTECA NACIONAL” in one example while correctly recognizing a drop cap, and failed badly on a two-column page, where it ignored one column and hallucinated menu items and portion sizes. These observations indicate that multimodal OCR and classical OCR fail for different reasons, and that PastReader exposes both literal-recognition errors and higher-level document-understanding errors.

6. Practical implications, computational considerations, and future work

PastReader, as instantiated in the GRESEL report, demonstrates three broader points. First, the benchmark is a realistic comparison space across OCR paradigms: a specialized archival OCR platform, a standard OCR engine, and a compact vision-LLM can all be evaluated under the same historical-document constraints. Second, line-break handling has direct consequences for evaluation, making formatting strategy part of system design rather than post-processing. Third, a small multimodal model can be competitive with classical OCR tools on historical Spanish material even under severe hardware constraints (Torterolo-Orta et al., 7 Jul 2025).

The computational profile reinforces this distinction. Tesseract is reported as much more efficient than the fine-tuned multimodal approach. Granite fine-tuning and inference were much more costly computationally. Transkribus emissions could not be precisely quantified because the platform is cloud-based, though the platform estimated around 3 hours of processing time. This suggests that PastReader is not only a question of recognition quality, but also of deployment regime, energy cost, and access to hardware.

The future-work agenda in the paper is correspondingly hybrid. The authors propose creating a dataset variant with transcription guidelines grounded in philology and paleography, testing other compact multimodal models, trying larger models, and fine-tuning with better hardware/cloud resources to avoid aggressive image downsampling. In collaboration with Biblioteca Nacional de España (BNE), they also plan to continue exploring techniques using the Spanish-language dataset provided by the shared task. Within that trajectory, PastReader emerges as a benchmark for historical Spanish OCR that binds together document-image degradation, editorial fidelity, model behavior under layout complexity, and the methodological problem of how transcription quality should be measured.