---
title: 'Baseer: Arabic Document-to-Markdown OCR'
url: https://www.emergentmind.com/topics/baseer
type: topic
---

# Baseer: Arabic Document-to-Markdown OCR

to=arxiv_search.search 娱乐彩票注册  天天中彩票微信 平台直属 code:
{"query":"arXiv:2509.18174 OR Title: Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR","max_results":5,"sort_by":"relevance"}【อ่านข้อความเต็มanalysis to=all code not needed?
to=arxiv_search.search  微信天天彩票 code:
{"query":"\"Baseer: A Vision-Language Model for Arabic Document-to-Markdown OCR\"","max_results":10,"sort_by":"relevance"}
to=arxiv_search.search code:
{"query":"2509.18174","max_results":10,"sort_by":"relevance"}
Baseer is a vision-language model for Arabic document-to-Markdown OCR, introduced to address the persistent difficulty of extracting accurate text and structure from Arabic documents with cursive script, diverse fonts, optional but semantically important diacritics, right-to-left orientation, and complex page layouts. It is built by fine-tuning Qwen2.5-VL-3B-Instruct with a decoder-only strategy on a hybrid corpus of 500,000 image–text pairs, while keeping the vision encoder frozen. The model emits Markdown with HTML tables and specialized layout tags, and is evaluated together with Misraj-DocOCR, an expert-verified benchmark for Arabic OCR and document-to-Markdown conversion. On that benchmark, Baseer reports a Word Error Rate of 0.25 and the best structural metrics among the compared systems, establishing a new state-of-the-art in Arabic document OCR [2509.18174].

## 1. Problem setting and scope

Baseer is situated in the specific problem setting of Arabic document OCR rather than generic scene text recognition or unconstrained multimodal generation. The paper characterizes Arabic OCR as uniquely difficult because the script is cursive, ligatures are extensive, character shapes are context-sensitive, fonts and styles vary across publishing domains, diacritics are optional yet semantically important, and document content is organized in a right-to-left format that may coexist with complex visual structure such as tables, footnotes, and multi-column layouts [2509.18174].

Within this setting, the target output is not plain text. Baseer is designed for document-to-Markdown OCR, meaning that the model is expected to recover both textual content and document structure. In the reported formulation, headings, lists, and emphasis are preserved in Markdown, while tables are represented in HTML to capture diverse structures and complex layouts. Specialized tags are used to mark watermarks, page numbers, and embedded images. This output design makes the task closer to layout-aware transcription than to line-level OCR.

The paper also identifies recurring failure modes of general-purpose multimodal large language models on Arabic documents. These include occasional left-to-right generation and suboptimal handling of diacritized text. The significance of Baseer lies in treating these errors as a domain-adaptation problem: rather than redesigning the model backbone, the work specializes a pre-trained multimodal model for Arabic document parsing and generation.

## 2. Model design and adaptation strategy

The base architecture of Baseer is Qwen2.5-VL-3B-Instruct. The reported adaptation strategy is decoder-only fine-tuning: only the language decoder is updated, while the vision encoder remains frozen [2509.18174]. The paper states that this preserves general visual features learned during pretraining while specializing language reasoning and generation for Arabic document OCR.

No structural changes to the backbone are reported. The model uses the base model’s tokenizer, and no tokenizer modification is described. The specialization is achieved through data, prompt controls, and post-processing for structure normalization and right-to-left content fidelity rather than through architectural intervention. This design choice is central to the paper’s framing: Baseer is presented as a domain-adapted vision-language model rather than as a newly engineered OCR architecture.

The training objective is standard next-token prediction in a causal language-model setup. During loss computation, the system prompt and image-embedding tokens are masked. The paper describes this as part of the training procedure and stabilization strategy. Training is conducted for 3 epochs with AdamW, a learning rate of \(1 \times 10^{-4}\) with cosine decay, weight decay 0.01, 100 warm-up steps, batch size 640, and maximum sequence length 4096, using \(8 \times\) NVIDIA H100 GPUs [2509.18174].

Ablation results reported in the paper support the decoder-only choice. On a 50k-sample, 2-epoch comparison using ChrF, Baseer-Decoder achieves 89.79, Baseer-LoRA 85.52, and Baseer-Full 84.79. The paper’s conclusion is that preserving the pre-trained vision encoder while specializing the language decoder yields the highest character-level fidelity for Arabic OCR. This suggests that, in this task formulation, language-side adaptation is more beneficial than jointly altering the visual representation.

## 3. Data pipeline and document representation

The training corpus contains 500,000 image–text pairs and is explicitly hybrid. Of these, 300,000 pairs are synthetic and 200,000 are real-world [2509.18174]. The synthetic portion is derived from markdown-formatted documents filtered from Common Crawl using methods described as analogous to an earlier Misraj dataset. The filtering pipeline includes perplexity filtering with KenLM to retain coherent Arabic text and a table sparsity filter that discards documents with more than 25% empty cells in markdown tables.

The synthetic rendering pipeline proceeds from Markdown to HTML to Word to PDF and finally to page-level images. The paper emphasizes visual diversity in this process. Synthetic pages use 39 Arabic fonts and page sizes A4, A5, Letter, Legal, Tabloid, and A3, including landscape variants. Background and text colors are drawn from curated distributions, with 8 light shades and 5 dark shades for backgrounds, and 9 light and 16 dark text colors. Alignment is distributed as Right 65%, Left 5%, and Center 30%; column counts are 1 column in 75% of pages, 2 columns in 20%, and 3 columns in 5%. Font sizes are even values from 8 to 22 pt, with margins from 1.0 to 2.5 cm, line height from 1.0 to 1.6, and column spacing from 0.5 to 1.2 cm. Special formatting includes random highlights and colored paragraphs, and right-to-left mode is enabled in 95% of synthetic pages.

Augmentation is extensive. The paper reports 29 transformations spanning eight categories: pre-print adjustments, printing defects, human marks, paper aging, digital noise, geometric distortions, lighting, and blur. A total of 150,000 images undergo 1 to 3 random transformations, and the originals of augmented samples are discarded to prevent redundancy. This augmentation scheme is presented as a source of robustness to noise and distortions.

The real-world subset consists of 200,000 pairs drawn from books, magazines, educational documents, and academic papers. Selection is guided by vision-based layout analysis using paragraph-level bounding boxes and alignment or overlap patterns to target challenging structures such as tables, figures, indexes, skewed layouts, embedded images, and colorful backgrounds. Transcriptions are produced by a state-of-the-art VLM, and a representative subset is manually verified by experts for textual accuracy and structural fidelity.

Across both synthetic and real-world data, the ground-truth representation is Markdown with HTML tables and special tags for layout elements. The paper treats this explicit structure encoding as part of the training signal rather than as a downstream post hoc conversion. A plausible implication is that structural supervision is one reason Baseer performs particularly strongly on layout-aware metrics.

## 4. Input–output behavior and evaluation protocol

Baseer takes document page images as input. Synthetic pages are rendered at high resolution from PDF pages, while additional examples are collected from real-world books and magazines. The output is Markdown, with HTML used for tables, intended to preserve headings, lists, emphasis, and other structural elements. The appendix is said to include qualitative examples showing fidelity on complex layouts, footnotes, and multi-column pages [2509.18174].

Right-to-left handling is reinforced in two ways reported in the paper: the predominance of right-to-left synthetic data and the fine-tuning process itself. The authors state that fine-tuning mitigates the base model’s occasional left-to-right reversion and diacritics artifacts. No tokenizer change is introduced for this purpose; improvement is attributed to decoder-only fine-tuning and right-to-left-rich data.

Evaluation is conducted using Misraj-DocOCR, a benchmark introduced in the same work. It contains 400 high-quality images covering diverse document types, layouts, and fonts, including synthetic and real-world pages. Every image’s transcription and structure are reviewed by human experts. The paper also releases a reviewed and corrected version of KITAB-bench pdf-to-markdown, of which 30 samples are used in the reported evaluation, specifically to address deficiencies such as hallucinations, missing page numbers, and small-font omissions in prior data.

The evaluation protocol standardizes system outputs before scoring. The reported post-processing steps are: removing HTML tags outside tables, converting Markdown tables to HTML, normalizing horizontal lines such as `---`, standardizing header formatting, unifying formatting tags in HTML tables such that `<strong>` and `<b>` become `<b>`, and removing model-specific tags such as `<page_number>` and `<watermark>` used by Baseer and Nanonets.

Text fidelity is measured with Word Error Rate, Character Error Rate, BLEU, and ChrF. Structure fidelity is measured with Tree Edit Distance Similarity and MARS. The paper explicitly states the standard OCR error formulas:
$$
\mathrm{WER} = \frac{S + D + I}{N}
$$
$$
\mathrm{CER} = \frac{S + D + I}{N}
$$
where \(S\), \(D\), and \(I\) denote substitutions, deletions, and insertions, and \(N\) is the total number of words for WER or characters for CER [2509.18174].

## 5. Empirical performance

On Misraj-DocOCR, which consists of 400 samples, Baseer reports WER 0.25, CER 0.53, BLEU 76.18, ChrF 87.77, TEDS 66, and MARS 76.885 [2509.18174]. According to the reported comparison, Gemini-2.5-pro attains WER 0.37, CER 0.31, BLEU 77.92, ChrF 89.55, TEDS 52, and MARS 70.775, while Azure AI Document Intelligence attains CER 0.27, WER 0.44, TEDS 42, and MARS 62.245. Selected additional baselines include Dots.ocr with WER 0.50, Nanonets with WER 0.71, Qari with WER 0.76, Qwen2.5-VL-32B with WER 0.76, GPT-4o-mini with WER 1.36, and Aya-vision with WER 1.41.

The comparative pattern is specific. Baseer is best on WER, TEDS, and MARS; Gemini-2.5-pro is best on BLEU and ChrF; Azure AI Document Intelligence is best on CER. The paper therefore characterizes Baseer as the strongest overall system on this benchmark, with a particular advantage in structural fidelity. This distinction matters because the task is document-to-Markdown OCR rather than isolated text extraction.

On the corrected KITAB-bench pdf-to-markdown subset of 30 samples, the comparison is narrower and restricted to open-source systems. Dots.ocr achieves WER 0.39, CER 0.28, BLEU 59.28, ChrF 83.16, TEDS 43, and MARS 63.08. Baseer achieves WER 0.61, CER 0.40, BLEU 55.78, ChrF 80.26, TEDS 56, and MARS 68.13 [2509.18174]. Here, Baseer leads on TEDS and MARS, while Dots.ocr is stronger on text-centric metrics. The paper explicitly notes that KITAB-bench is small and that, on the larger Misraj-DocOCR benchmark, Baseer’s advantage widens.

Taken together, the reported results support a division between text-level and structure-level performance. Baseer is not uniformly best on every metric, but it is reported as state-of-the-art in WER and as best-in-class on the principal structural metrics in the benchmark most strongly emphasized by the paper.

## 6. Analyses, limitations, and practical implications

The paper includes several analyses beyond headline metrics. Context-length ablation shows ChrF 82.69 at sequence length 2048, 89.79 at 4096, and 87.52 at 8192 [2509.18174]. The reported interpretation is that 4096 balances capacity and relevance, whereas 8192 introduces excessive padding for typical page content and dilutes salient information. The paper also states that qualitative evaluation across multiple open-source vision-language models favored Qwen2.5-VL-3B-Instruct for Arabic tasks because of better right-to-left preservation and coherence than alternatives.

Error analysis is described at a high level. Before fine-tuning, the model exhibits occasional left-to-right generation and weaker diacritics handling. After fine-tuning, right-to-left consistency and structure fidelity improve. The paper does not report a dedicated diacritics-specific ablation, and it notes that CER is not the best on Misraj-DocOCR because Azure achieves the lowest CER. This identifies a concrete limitation: Baseer’s strongest gains are not equivalent to dominance on every character-level criterion.

Several additional limitations are stated directly. The work focuses on decoder-only fine-tuning without architectural changes; vision-encoder adaptation and specialized Arabic tokenization are not explored. KITAB-bench evaluation is limited to a corrected subset of 30 samples, which makes scores sensitive to individual errors. Broader public benchmarks are said to remain needed.

Practical considerations are also delimited. Inference speed and memory footprint are not reported. Deployment details are not described. Model and code release status is unspecified, although the benchmark datasets are openly available on Hugging Face and qualitative outputs are included in the appendix. The output format—standardized Markdown with HTML tables and special tags—makes the system well suited for downstream parsing and rendering pipelines, but this is an implication of the representation design rather than a deployment study.

The paper’s central technical takeaway is that decoder-only fine-tuning of a strong pre-trained vision-language model, combined with a large and diverse Arabic document dataset and explicit structural supervision, yields substantial gains in Arabic document-to-Markdown OCR. More specifically, the results suggest that preserving general visual features while specializing the language decoder is an effective strategy for morphologically rich, right-to-left document domains [2509.18174].

Source: https://www.emergentmind.com/topics/baseer