---
title: 'IM2LATEX-100K: Math OCR Dataset'
url: https://www.emergentmind.com/topics/im2latex-100k-dataset-ad016d42-2c17-4b9f-a959-bbe2d1d350d9
type: topic
---

# IM2LATEX-100K: Math OCR Dataset

IM2LATEX-100K is a large-scale dataset for mathematical expression recognition, enabling end-to-end machine learning approaches to the challenging task of converting images of rendered LaTeX formulas into their corresponding markup sequences. Sourced from arXiv.org, this dataset serves as a critical benchmark for models seeking to bridge the gap between visual mathematical expressions and structured, machine-readable representations. Its design incorporates paired grayscale image–formula samples, extensive structural diversity, robust preprocessing, and is accompanied by standardized evaluation protocols to facilitate reproducible research in mathematical OCR.

## 1. Dataset Construction and Composition

IM2LATEX-100K comprises 103,556 distinct mathematical expressions, extracted from the LaTeX source files of over 60,000 arXiv papers, predominantly from the High Energy Physics–Theory (hep-th) category [2012.12619][1908.11415]. Each formula is rendered to PNG format via a pipeline of LaTeX → PDF → grayscale PNG, with images exceeding 480 pixels in width discarded to maintain consistency.

The dataset is split as follows:

| Split        | Expressions (ConvMath) | Expressions (MI2LS)      |
|--------------|-----------------------|--------------------------|
| Train        | 65,995                | 83,883                   |
| Validation   | 8,181                 | 9,319                    |
| Test         | 8,301                 | 10,354                   |
| **Total**    | 103,556               | 103,556                  |

While [2012.12619] and [1908.11415] report minor split differences, both document the same overall sample count per the original dataset.

The original formulas feature character lengths ranging from 38 to 997 per expression (mean ≈118, median ≈98). All major LaTeX constructs appear, including fractions, superscripts/subscripts, roots, integrals, summations, and matrix environments.

## 2. Image and Annotation Characteristics

All images are grayscale PNGs with variable dimensions (width ≤480 px; arbitrary height), representing tightly-cropped mathematical formulas [2012.12619][1908.11415]. For computational efficiency and consistent batching:

- Images are grouped into discrete 'buckets' by size—15 in [2012.12619] (e.g., 160×32, 256×64), 20 in [1908.11415]—and padded with a white background to a canonical rectangle size within each group.
- No additional resizing or cropping is performed.
- For multi-formula batch training, formulas are centered within their buckets.

Ground-truth LaTeX is stored in plain UTF-8 text, one file per sample, using the tokenization scheme of Deng et al. (2016): for IM2LATEX-100K, the vocabulary includes 583 unique tokens [2012.12619] or 483 (after minimal token normalization and special <START>/<END> markers) [1908.11415]. Annotation files encode a direct one-to-one mapping between each image and its LaTeX representation. No token-level or region-level spatial alignments are provided; alignment learning is delegated to model attention mechanisms.

## 3. Preprocessing Pipeline and Tokenization

The preprocessing workflow ensures consistency between visual input and textual output:

- **Image preprocessing** performs tight bounding box cropping, optional downsampling for memory efficiency, and bucket-based padding. Original scan resolutions (e.g., 1654×2339 px) are typically halved.
- **LaTeX preprocessing** tokenizes and normalizes expressions to standardize semantically equivalent—but syntactically divergent—formulations. The pipeline uses:
  - Parsing to an abstract syntax tree (AST) using KaTeX [1908.11415].
  - Token emission with reserved minimal units (e.g., '\psi', '\frac', '^', '_', '+', etc.).
  - Special sequence delimiters (<START>, <END>).
- The resulting sequence lengths exhibit a long-tail distribution, with a notable ∼3.4% of instances exceeding 150 tokens—a known challenge for neural sequence models [1908.11415].

## 4. Evaluation Metrics

IM2LATEX-100K is evaluated using multiple complementary metrics to account for the polymorphic nature of LaTeX and the 2D structure of mathematical expressions [2012.12619][1908.11415]:

1. **BLEU Score** (Papineni et al. 2002): Corpus-level, 4-gram cumulative BLEU measures n-gram overlap between predicted and ground-truth LaTeX token sequences.
2. **Column-wise Edit Distance**: Rendered outputs are converted to bitmaps, serialized column-wise, and compared using the Levenshtein distance. The normalized distance (edit operations per column) penalizes insertion, deletion, or substitution errors.
3. **Exact Match Accuracy**: Proportion of test samples for which the model prediction, once rendered, is pixel-identical to the ground-truth image. A variant ignores entirely blank (whitespace) columns.
   
Formally, for $N$ test examples, exact match can be expressed as
$$
\mathrm{Acc} = \frac{1}{N} \sum_{i=1}^N \mathbb{1}(\hat{y}_i = y_i)
$$
where $\hat{y}_i$ represents the prediction and $y_i$ the ground truth.

## 5. Data Challenges and Ambiguities

IM2LATEX-100K’s complexity arises from several domain-specific challenges:

- **LaTeX Polymorphism**: Multiple syntactic LaTeX sequences can represent identical rendered expressions (e.g., alternate ordering of scripts or styling commands). This motivates the use of image-based, in addition to sequence-based, evaluation [1908.11415].
- **Rare Symbols and Long Expressions**: Vocabulary sparsity increases with rarely-used tokens (\alpha, \psi, \oslash) and extended sequences, stressing model generalization. Only about 3.4% of test expressions, but disproportionately influential, exceed the usual length limit of 150 tokens [1908.11415].
- **2D Spatial Structure**: The dataset spans formulas containing higher-order vertical/horizontal relationships, such as matrices or nested scripts. Maintaining spatial locality throughout the encoding and decoding pipeline is critical; techniques like 2D positional encoding are employed for this purpose [1908.11415][2012.12619].

## 6. Successor Dataset: IM2LATEXv2 and Its Improvements

The MathNet project introduced “im2latexv2,” a significant data-centric revision. This variant applies systematic LaTeX normalization (function $f_{\text{norm}}$) to canonicalize markup, removing all extraneous font, spacing, and non-structural commands [2404.13667]:

- **Font Diversity**: Original IM2LATEX-100K used only Computer Modern. IM2LATEXv2 renders each formula in up to 59 fonts (30 for training, 29 held out for validation/test), introducing >2 million images for training alone, thus improving cross-font generalization.
- **Canonical Labels**: $f_{\text{norm}}$ eliminates redundant or optional LaTeX constructs, standardizes bracing and sub/superscript usage, merges synonyms (e.g., '\le' → '\leq'), and purges malformed arrays, shrinking the vocabulary from ~500 to 320 tokens.
- **Data Cleaning**: Blank renders, empty formulas, and entries with fatal normalization errors are removed, yielding a “purer” dataset—final image–formula pairs: 74,245 train, 8,243 validation, 10,118 test [2404.13667].
- **Directory and Metadata Structure**: Each image is linked to rich metadata including font identifiers, DPI (fixed at 600), and image dimensions. Directory layout follows split → images/ (by font) and formulas.lst (formula index, canonical LaTeX).

This revision addresses label noise, model convergence, and robustness to font idiosyncrasies, yielding a benchmark that is both more realistic and more consistent than the original [2404.13667].

## 7. Benchmark Usage and Model Performance

Numerous neural architectures for mathematical expression recognition benchmark their results on IM2LATEX-100K:

- ConvMath employs a fully convolutional encoder-decoder pipeline with multi-layer attention, achieving state-of-the-art accuracy and strong parallel computation efficiency [2012.12619].
- MI2LS integrates 2D positional encoding and sequence-level reinforcement learning, attaining notable improvements in BLEU, edit distance, and exact match (for instance, BLEU: 90.28%, exact match: 82.33% with sequence-level training) [1908.11415].

Comparative performance indicates:

| Model                    | BLEU   | ImageEditDist | Exact Match Accuracy |
|--------------------------|--------|---------------|---------------------|
| WYGIWYS (attn s2s)       | 87.73% | 87.60%        | 77.46%              |
| Double Attention         | 88.42% | 88.57%        | 79.81%              |
| MI2LS (w/ seq-level RL)  | 90.28% | 92.28%        | 82.33%              |

Performance declines modestly but remains robust for longer expressions, and the use of sophisticated attention-based and convolutional sequence models continues to drive improvements.

---

IM2LATEX-100K and its successor, IM2LATEXv2, constitute foundational resources for evaluating mathematical OCR pipelines. Their careful annotation, structural diversity, and rigorous evaluation protocols enable systematic study of crucial challenges in mathematical expression recognition, with ongoing refinements such as LaTeX canonicalization and font augmentation further improving their relevance to real-world use cases [1908.11415][2012.12619][2404.13667].

Source: https://www.emergentmind.com/topics/im2latex-100k-dataset-ad016d42-2c17-4b9f-a959-bbe2d1d350d9