CSFormula: Multilevel Formula Recognition Dataset

• CSFormula is a comprehensive dataset designed for formula recognition, featuring over 1 million training samples across line, paragraph, and page levels.
• It integrates detailed annotations with precise LaTeX markup and bounding box localizations, covering diverse domains including mathematics, physics, and chemistry.
• The dataset addresses challenges in visual variability and structural complexity, advancing research in end-to-end document parsing and OCR accuracy.

CSFormula is a large-scale, multidisciplinary dataset for mathematical and scientific formula recognition, introduced in the context of automated understanding of complex documents. Created for benchmarking and training vision-to-LaTeX systems, it comprehensively spans line-level, paragraph-level, and full-page formula recognition challenges while reflecting the broad diversity of real-world STEM document formats and symbol vocabularies. The collection, first presented alongside the DocTron-Formula framework, offers a unified benchmark intended to advance the state of the art in formula OCR and scientific content extraction (Zhong et al., 1 Aug 2025).

1. Dataset Composition and Multilevel Structure

CSFormula is composed of formula samples sourced from STEM-focused StackExchange forums, targeting authenticity in both symbol coverage and formula layout. Before deduplication, the dataset contains 1,884,532 formulas; after deduplication, the official statistics are 1,008,467 training samples and 3,000 test samples, spanning three levels of structural complexity:

• Line-level: Isolated printed formulas, single or multi-line, typically without surrounding text (Train: 741,016; Test: 1,000).
• Paragraph-level: Formulas embedded within running text fragments, preserving natural reading order and contextual entanglement (Train: 135,575; Test: 1,000).
• Page-level: Full-page scientific layouts, entangling multiple formulas, text, tables, and varying document artifacts. Each includes granular localization for formula regions (Train: 131,876; Test: 1,000).

Within these structural levels, domain coverage approximates 50% mathematics, 30% physics, 15% chemistry, and 5% other engineering, biology, or computer science content.

2. Annotation Schema and Ground-Truth Representations

Each CSFormula sample consists of an image and corresponding LaTeX markup, targeting precise semantic and structural alignment.

• Images: For line- and paragraph-level samples, the image is a tight crop of the rendered formula; for paragraph-level, it includes the formula within surrounding textual context. Page-level samples present full-page document images.
• LaTeX Markup: All formulas and mathematical environments are supplied in standard LaTeX. For page-level images, axis-aligned bounding boxes (specified via absolute pixel coordinates) are provided for each formula region, with each box linked to an individual LaTeX string.

Example annotations: - Line-level: $E=mc^2$ - Paragraph-level (multi-line):

$$\begin{aligned} a^2 + b^2 &= c^2, \ \int_0^\infty e^{-x^2}\,dx &= \frac{\sqrt{\pi}}{2} \end{aligned}$$

The combination of image cropping, bounding box localization, and LaTeX pairing enables rigorous evaluation at multiple levels of formula recognition and document parsing.

3. Dataset Splits, Evaluation Protocols, and Metrics

CSFormula provides official splits post-deduplication:

Level	Train Samples	Test Samples	Avg. Formula Length (tokens)
Line-level	741,016	1,000	~20
Paragraph-level	135,575	1,000	~65
Page-level	131,876	1,000	~150

No separate validation set is published; users typically withhold training samples for that purpose as required.

Evaluation Metrics:

• Edit Distance (ED): Measures token-level dissimilarity between predicted and ground-truth LaTeX.
• Character Detection Matching (CDM): Renders predictions and references to images, quantifying character-level spatial overlap.
• Optional: Exact-match rates and structural similarity metrics, reflecting strict and weak correctness criteria.

Collectively, these metrics provide nuanced views of both string-level and rendered-image recognition accuracy within varying document contexts.

4. Structural and Domain Complexity

CSFormula targets structural phenomena and document variability underrepresented in previous formula OCR datasets:

• Deep Structure: Dependency on recognition of deeply nested fractions, stacked superscripts/subscripts, multi-row and multi-column matrices, and composite environments (e.g., align, gather, cases).
• Layout Entanglement: Requires segmenting and transcribing formulas that are closely interleaved with text (paragraph-level) or must be localized among diverse block elements at the page level.
• Rendering Variance: Includes high-resolution web math, scanned PDF crops, screenshots, and noisy/distorted images. The symbol vocabulary comprises not only classical mathematics but also discipline-specific representations (chemical notation, physics operators, biological subscripts, engineering units).

This scope demands models that are both robust to visual variability and capable of precise syntactic and semantic parsing.

5. Comparative Analysis with Existing Datasets

CSFormula marks a foundational shift compared to established mathematical OCR corpora:

Dataset	Scale (Train Samples)	Structural Levels	Domains/Symbol Diversity
CROHME	<200K	Handwritten, line	Pure mathematics
Im2LaTeX-160K	~160K	Printed, single-line	Pure mathematics
UniMER	n/a	Printed, single-line	“Real-world” variance, line-level only
CSFormula	>1M	Line/paragraph/page	Mathematics, physics, chemistry, engineering

CSFormula uniquely integrates hierarchical structural levels, full-page layouts, and broad domain coverage in a single benchmark. This facilitates not only model training for complex layouts but also detailed ablation studies on hierarchical transfer (e.g., line-to-paragraph-to-page) and domain robustness (math to chemistry, physics, etc.).

6. Practical Applications and Research Significance

CSFormula enables rigorous training and evaluation of end-to-end vision-to-LaTeX models capable of handling formulas at every scale—from isolated microformulas to dense, multipanel scientific pages. Representative use cases include:

• Benchmarking vision-LLMs: Testing robustness to structural complexity and layout variability in real-world scientific documents.
• Ablation studies: Investigating the impact of hierarchical training regimens, structural generalization, and cross-domain robustness in formula recognition architectures.
• Advancing document understanding paradigms: Supporting automated extraction and semantic parsing of formulas interleaved with text, tables, and figures in authentic scientific content.

CSFormula raises the bar for formula recognition, requiring systems to combine accurate visual parsing, precise syntactic structure extraction, and robust symbol localization across varied scientific disciplines and document archetypes. Its release has established new challenges and opportunities for research in document AI, structured OCR, and scientific literature understanding (Zhong et al., 1 Aug 2025).