Baseer: Arabic Document-to-Markdown OCR
- Baseer is a vision-language model specialized for converting Arabic documents into Markdown while preserving both textual content and layout structure.
- It employs decoder-only fine-tuning on a pre-trained model to effectively handle right-to-left text, cursive scripts, and semantically important diacritics.
- Empirical evaluations on benchmarks like Misraj-DocOCR demonstrate state-of-the-art performance, especially in structural fidelity and low word error rates.
to=arxiv_search.search 娱乐彩票注册 天天中彩票微信 平台直属 code: {"7query7 OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7"," to=all code not needed? to=arxiv_search.search 微信天天彩票 code: {"7query7 A Vision-LLM for Arabic Document-to-Markdown OCR7\7 OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7query7,"7sort_by7 to=arxiv_search.search code: {"7query7 OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7query7,"7sort_by7 Baseer is a vision-LLM for Arabic document-to-Markdown OCR, introduced to address the persistent difficulty of extracting accurate text and structure from Arabic documents with cursive script, diverse fonts, optional but semantically important diacritics, right-to-left orientation, and complex page layouts. It is built by fine-tuning Qwen7max_results7.7query7 with a decoder-only strategy on a hybrid corpus of 7query7query7query7,7query7query7query7^ image–text pairs, while keeping the vision encoder frozen. The model emits Markdown with HTML tables and specialized layout tags, and is evaluated together with Misraj-DocOCR, an expert-verified benchmark for Arabic OCR and document-to-Markdown conversion. On that benchmark, Baseer reports a Word Error Rate of 7query7.7max_results7query7^ and the best structural metrics among the compared systems, establishing a new state-of-the-art in Arabic document OCR (&&&7query7&&&).
7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7. Problem setting and scope
Baseer is situated in the specific problem setting of Arabic document OCR rather than generic scene text recognition or unconstrained multimodal generation. The paper characterizes Arabic OCR as uniquely difficult because the script is cursive, ligatures are extensive, character shapes are context-sensitive, fonts and styles vary across publishing domains, diacritics are optional yet semantically important, and document content is organized in a right-to-left format that may coexist with complex visual structure such as tables, footnotes, and multi-column layouts (&&&7query7&&&).
Within this setting, the target output is not plain text. Baseer is designed for document-to-Markdown OCR, meaning that the model is expected to recover both textual content and document structure. In the reported formulation, headings, lists, and emphasis are preserved in Markdown, while tables are represented in HTML to capture diverse structures and complex layouts. Specialized tags are used to mark watermarks, page numbers, and embedded images. This output design makes the task closer to layout-aware transcription than to line-level OCR.
The paper also identifies recurring failure modes of general-purpose multimodal LLMs on Arabic documents. These include occasional left-to-right generation and suboptimal handling of diacritized text. The significance of Baseer lies in treating these errors as a domain-adaptation problem: rather than redesigning the model backbone, the work specializes a pre-trained multimodal model for Arabic document parsing and generation.
7max_results7. Model design and adaptation strategy
The base architecture of Baseer is Qwen7max_results7.7query7 The reported adaptation strategy is decoder-only fine-tuning: only the language decoder is updated, while the vision encoder remains frozen (&&&7query7&&&). The paper states that this preserves general visual features learned during pretraining while specializing language reasoning and generation for Arabic document OCR.
No structural changes to the backbone are reported. The model uses the base model’s tokenizer, and no tokenizer modification is described. The specialization is achieved through data, prompt controls, and post-processing for structure normalization and right-to-left content fidelity rather than through architectural intervention. This design choice is central to the paper’s framing: Baseer is presented as a domain-adapted vision-LLM rather than as a newly engineered OCR architecture.
The training objective is standard next-token prediction in a causal language-model setup. During loss computation, the system prompt and image-embedding tokens are masked. The paper describes this as part of the training procedure and stabilization strategy. Training is conducted for 7sort_by7^ epochs with AdamW, a learning rate of PRESERVED_PLACEHOLDER_7query7^ with cosine decay, weight decay 7query7.7query7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7, 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7query7query7^ warm-up steps, batch size 7\7relevance7query7, and maximum sequence length 7relevance7query7sort_by7\7, using PRESERVED_PLACEHOLDER_7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7^ NVIDIA H7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7query7query7^ GPUs (&&&7query7&&&).
Ablation results reported in the paper support the decoder-only choice. On a 7query7query7k-sample, 7max_results7-epoch comparison using ChrF, Baseer-Decoder achieves 89.79, Baseer-LoRA 87query7.7query7max_results7 and Baseer-Full 87relevance7.79. The paper’s conclusion is that preserving the pre-trained vision encoder while specializing the language decoder yields the highest character-level fidelity for Arabic OCR. This suggests that, in this task formulation, language-side adaptation is more beneficial than jointly altering the visual representation.
7sort_by7. Data pipeline and document representation
The training corpus contains 7query7query7query7,7query7query7query7^ image–text pairs and is explicitly hybrid. Of these, 7sort_by7query7query7,7query7query7query7^ pairs are synthetic and 7max_results7query7query7,7query7query7query7^ are real-world (&&&7query7&&&). The synthetic portion is derived from markdown-formatted documents filtered from Common Crawl using methods described as analogous to an earlier Misraj dataset. The filtering pipeline includes perplexity filtering with KenLM to retain coherent Arabic text and a table sparsity filter that discards documents with more than 7max_results7query7% empty cells in markdown tables.
The synthetic rendering pipeline proceeds from Markdown to HTML to Word to PDF and finally to page-level images. The paper emphasizes visual diversity in this process. Synthetic pages use 7sort_by79 Arabic fonts and page sizes A7relevance7, A7query7, Letter, Legal, Tabloid, and A7sort_by7, including landscape variants. Background and text colors are drawn from curated distributions, with 8 light shades and 7query7^ dark shades for backgrounds, and 9 light and 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7\7^ dark text colors. Alignment is distributed as Right 7\7query7%, Left 7query7%, and Center 7sort_by7query7%; column counts are 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7^ column in 77query7% of pages, 7max_results7^ columns in 7max_results7query7%, and 7sort_by7^ columns in 7query7%. Font sizes are even values from 8 to 7max_results7max_results7^ pt, with margins from 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7.7query7^ to 7max_results7.7query7^ cm, line height from 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7.7query7^ to 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7.7\7, and column spacing from 7query7.7query7^ to 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7.7max_results7^ cm. Special formatting includes random highlights and colored paragraphs, and right-to-left mode is enabled in 97query7% of synthetic pages.
Augmentation is extensive. The paper reports 7max_results79 transformations spanning eight categories: pre-print adjustments, printing defects, human marks, paper aging, digital noise, geometric distortions, lighting, and blur. A total of 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7query7query7,7query7query7query7^ images undergo 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7^ to 7sort_by7^ random transformations, and the originals of augmented samples are discarded to prevent redundancy. This augmentation scheme is presented as a source of robustness to noise and distortions.
The real-world subset consists of 7max_results7query7query7,7query7query7query7^ pairs drawn from books, magazines, educational documents, and academic papers. Selection is guided by vision-based layout analysis using paragraph-level bounding boxes and alignment or overlap patterns to target challenging structures such as tables, figures, indexes, skewed layouts, embedded images, and colorful backgrounds. Transcriptions are produced by a state-of-the-art VLM, and a representative subset is manually verified by experts for textual accuracy and structural fidelity.
Across both synthetic and real-world data, the ground-truth representation is Markdown with HTML tables and special tags for layout elements. The paper treats this explicit structure encoding as part of the training signal rather than as a downstream post hoc conversion. A plausible implication is that structural supervision is one reason Baseer performs particularly strongly on layout-aware metrics.
7relevance7. Input–output behavior and evaluation protocol
Baseer takes document page images as input. Synthetic pages are rendered at high resolution from PDF pages, while additional examples are collected from real-world books and magazines. The output is Markdown, with HTML used for tables, intended to preserve headings, lists, emphasis, and other structural elements. The appendix is said to include qualitative examples showing fidelity on complex layouts, footnotes, and multi-column pages (&&&7query7&&&).
Right-to-left handling is reinforced in two ways reported in the paper: the predominance of right-to-left synthetic data and the fine-tuning process itself. The authors state that fine-tuning mitigates the base model’s occasional left-to-right reversion and diacritics artifacts. No tokenizer change is introduced for this purpose; improvement is attributed to decoder-only fine-tuning and right-to-left-rich data.
Evaluation is conducted using Misraj-DocOCR, a benchmark introduced in the same work. It contains 7relevance7query7query7^ high-quality images covering diverse document types, layouts, and fonts, including synthetic and real-world pages. Every image’s transcription and structure are reviewed by human experts. The paper also releases a reviewed and corrected version of KITAB-bench pdf-to-markdown, of which 7sort_by7query7^ samples are used in the reported evaluation, specifically to address deficiencies such as hallucinations, missing page numbers, and small-font omissions in prior data.
The evaluation protocol standardizes system outputs before scoring. The reported post-processing steps are: removing HTML tags outside tables, converting Markdown tables to HTML, normalizing horizontal lines such as ---, standardizing header formatting, unifying formatting tags in HTML tables such that <strong> and <b> become <b>, and removing model-specific tags such as <page_number> and <watermark> used by Baseer and Nanonets.
Text fidelity is measured with Word Error Rate, Character Error Rate, BLEU, and ChrF. Structure fidelity is measured with Tree Edit Distance Similarity and MARS. The paper explicitly states the standard OCR error formulas:
PRESERVED_PLACEHOLDER_7max_results7^
PRESERVED_PLACEHOLDER_7sort_by7^
where PRESERVED_PLACEHOLDER_7relevance7, PRESERVED_PLACEHOLDER_7query7, and PRESERVED_PLACEHOLDER_7\7^ denote substitutions, deletions, and insertions, and is the total number of words for WER or characters for CER (&&&7query7&&&).
7query7. Empirical performance
On Misraj-DocOCR, which consists of 7relevance7query7query7^ samples, Baseer reports WER 7query7.7max_results7query7 CER 7query7.7query7sort_by7 BLEU 77\7.7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR78, ChrF 87.77, TEDS 7\7\7, and MARS 77\7.887query7^ (&&&7query7&&&). According to the reported comparison, Gemini-7max_results7.7query7 attains WER 7query7.7sort_by77 CER 7query7.7sort_by7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7, BLEU 77.97max_results7, ChrF 89.7query7query7, TEDS 7query7max_results7, and MARS 77query7.777query7 while Azure AI Document Intelligence attains CER 7query7.7max_results77 WER 7query7.7relevance7relevance7 TEDS 7relevance7max_results7, and MARS 7\7max_results7.7max_results7relevance7query7 Selected additional baselines include Dots.ocr with WER 7query7.7query7query7 Nanonets with WER 7query7.77(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7, Qari with WER 7query7.77\7 Qwen7max_results7.7query7 with WER 7query7.77\7 GPT-7relevance7o-mini with WER 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7.7sort_by7\7, and Aya-vision with WER 7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7.7relevance7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7.
The comparative pattern is specific. Baseer is best on WER, TEDS, and MARS; Gemini-7max_results7.7query7 is best on BLEU and ChrF; Azure AI Document Intelligence is best on CER. The paper therefore characterizes Baseer as the strongest overall system on this benchmark, with a particular advantage in structural fidelity. This distinction matters because the task is document-to-Markdown OCR rather than isolated text extraction.
On the corrected KITAB-bench pdf-to-markdown subset of 7sort_by7query7^ samples, the comparison is narrower and restricted to open-source systems. Dots.ocr achieves WER 7query7.7sort_by7 CER 7query7.7max_results7 BLEU 7query79.7max_results7 ChrF 87sort_by7.7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7\7, TEDS 7relevance7sort_by7, and MARS 7\7sort_by7.7query7 Baseer achieves WER 7query7.7\7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7, CER 7query7.7relevance7query7 BLEU 7query7query7.78, ChrF 87query7.7max_results7\7 TEDS 7query7\7, and MARS 7\78.7(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7sort_by7^ (&&&7query7&&&). Here, Baseer leads on TEDS and MARS, while Dots.ocr is stronger on text-centric metrics. The paper explicitly notes that KITAB-bench is small and that, on the larger Misraj-DocOCR benchmark, Baseer’s advantage widens.
Taken together, the reported results support a division between text-level and structure-level performance. Baseer is not uniformly best on every metric, but it is reported as state-of-the-art in WER and as best-in-class on the principal structural metrics in the benchmark most strongly emphasized by the paper.
7\7. Analyses, limitations, and practical implications
The paper includes several analyses beyond headline metrics. Context-length ablation shows ChrF 87max_results7.7\7 at sequence length 7max_results7query7relevance78, 89.79 at 7relevance7query7sort_by7\7, and 87.7query7max_results7^ at 87(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7sort_by7max_results7^ (&&&7query7&&&). The reported interpretation is that 7relevance7query7sort_by7\7^ balances capacity and 7relevance7 whereas 87(Hennara et al., 17 Sep 2025) OR Title: Baseer: A Vision-LLM for Arabic Document-to-Markdown OCR7sort_by7max_results7^ introduces excessive padding for typical page content and dilutes salient information. The paper also states that qualitative evaluation across multiple open-source vision-LLMs favored Qwen7max_results7.7query7 for Arabic tasks because of better right-to-left preservation and coherence than alternatives.
Error analysis is described at a high level. Before fine-tuning, the model exhibits occasional left-to-right generation and weaker diacritics handling. After fine-tuning, right-to-left consistency and structure fidelity improve. The paper does not report a dedicated diacritics-specific ablation, and it notes that CER is not the best on Misraj-DocOCR because Azure achieves the lowest CER. This identifies a concrete limitation: Baseer’s strongest gains are not equivalent to dominance on every character-level criterion.
Several additional limitations are stated directly. The work focuses on decoder-only fine-tuning without architectural changes; vision-encoder adaptation and specialized Arabic tokenization are not explored. KITAB-bench evaluation is limited to a corrected subset of 7sort_by7query7^ samples, which makes scores sensitive to individual errors. Broader public benchmarks are said to remain needed.
Practical considerations are also delimited. Inference speed and memory footprint are not reported. Deployment details are not described. Model and code release status is unspecified, although the benchmark datasets are openly available on Hugging Face and qualitative outputs are included in the appendix. The output format—standardized Markdown with HTML tables and special tags—makes the system well suited for downstream parsing and rendering pipelines, but this is an implication of the representation design rather than a deployment study.
The paper’s central technical takeaway is that decoder-only fine-tuning of a strong pre-trained vision-LLM, combined with a large and diverse Arabic document dataset and explicit structural supervision, yields substantial gains in Arabic document-to-Markdown OCR. More specifically, the results suggest that preserving general visual features while specializing the language decoder is an effective strategy for morphologically rich, right-to-left document domains (&&&7query7&&&).