---
title: TableBank-LaTeX Benchmark
url: https://www.emergentmind.com/topics/tablebank-latex
type: topic
---

# TableBank-LaTeX Benchmark

Searching arXiv for papers on TableBank-LaTeX, TabLeX, and related table extraction/detection benchmarks.
TableBank-LaTeX is a label used in the literature for LaTeX-derived table benchmarks built from scientific documents, especially arXiv source files. In the sources considered here, the term covers at least three closely related but non-identical resources: the LaTeX subset of TableBank, which provides weakly supervised page-level table detection and HTML-style structure targets [1903.01949]; TabLeX, described explicitly as “TableBank-LaTeX,” which pairs rendered table images with original LaTeX code for structure and content extraction [2105.06400]; and later arXiv-derived corpora used for active-learning table detection and multimodal table-to-LaTeX generation [2509.20003], [2509.17589]. The unifying idea is supervision from LaTeX source, but the annotation unit varies from page images with bounding boxes to token sequences inside `\begin{tabular}` and `\end{tabular}`.

## 1. Nomenclature and benchmark scope

A recurring source of confusion is that “TableBank-LaTeX” does not designate a single fixed artifact across all papers. In the original TableBank work, the LaTeX subset is a component of a broader benchmark for table detection and recognition [1903.01949]. In the TabLeX paper, the dataset is presented as a large-scale, LaTeX-grounded benchmark and is explicitly summarized as “TableBank-LaTeX” [2105.06400]. Subsequent papers then use “TableBank-LaTeX” for a page-level detection corpus and for a 1.2 M-pair table-image/LaTeX corpus, respectively [2509.20003], [2509.17589].

| Usage in the literature | Supervision target | Representative scale |
|---|---|---|
| TableBank LaTeX subset | Page-level table locations; HTML-style structure sequence | 253,817 training tables for detection; 88,597 training tables for structure |
| TabLeX (“TableBank-LaTeX”) | Rendered table image + corresponding LaTeX source | Over 4.9 million rendered table images |
| Later downstream corpora labeled TableBank-LaTeX | COCO/JSON table boxes or full table-image/LaTeX pairs | 187,199/7,265/5,719 detection split; 1,209,986 generation pairs |

This terminological drift matters because the task definition changes with the corpus. In TableBank, the recognition target is an HTML-style token sequence with `<cell_y>` and `<cell_n>`, whereas in TabLeX the targets are LaTeX token sequences, either “CELL-only” for structure or full tabular content for content extraction [1903.01949], [2105.06400]. A plausible implication is that results reported under the same name may not be directly comparable unless the annotation language, image unit, and evaluation metric are aligned.

## 2. Weakly supervised TableBank construction from LaTeX sources

The original TableBank paper builds the LaTeX subset from the arXiv bulk download covering 2014–2018. The authors retrieved the full `.tex` source together with style files and figures, and each document was compiled with `pdflatex` into a PDF before and after annotation [1903.01949].

For table detection, the weak-supervision pipeline identifies every environment of the form

```latex
\begin{table}[…]
  …
\end{table}
```

and wraps the tabular in an `\fcolorbox` so that recompilation yields a uniquely colored border around the table. The original and annotated PDF pages are then rendered at the same resolution and subtracted pixel-wise; the green frame pixels stand out, enabling extraction of bounding boxes `(x, y, width, height)` for each connected region. These are stored per page in a JSON schema with table identifiers and coordinates in PDF-pixel units [1903.01949].

For table structure recognition, the pipeline uses LaTeXML to convert the `.tex` into XML, followed by a simple transform into an HTML-style token sequence. Cells containing text become `<cell_y>`, and empty cells become `<cell_n>`. The vocabulary has 11 tags, including `<tabular>`, `<thead>`, `<tbody>`, `<tr>`, `<cell_y>`, and `<cell_n>` [1903.01949].

The LaTeX-subset statistics reported for TableBank are specific to this page-level regime. For detection, the LaTeX subset contains 253,817 training tables, 1,000 validation pages with at least one table per page, and 1,000 test pages. For structure recognition, it contains 88,597 training tables, 500 validation tables, and 500 test tables. The layout distribution over structure-token length is also reported: 28.0% in the 0–40 bucket, 30.8% in 40–80, 24.8% in 80–120, 13.6% in 120–200, and 2.8% above 200 tokens [1903.01949].

These design choices made TableBank one of the first large weakly supervised benchmarks for image-based table detection and recognition. At the same time, the structure target is deliberately simplified: complex spanning such as `\multicolumn` and `\multirow` is not explicitly encoded in `<cell_y>` versus `<cell_n>`, and the paper identifies nested or merged cells as a future extension [1903.01949].

## 3. TabLeX as a LaTeX-native structure and content benchmark

TabLeX extends the LaTeX-centric paradigm from page-level detection and simplified structure recognition to direct recovery of tabular structure and content from rendered table images. The dataset comprises over 4.9 million rendered table images in total, each accompanied by its original LaTeX code, and is divided into a Table Structure Dataset (TSD) and a Table Content Dataset (TCD) [2105.06400].

The dataset is built by crawling arXiv preprints from January 2019 to September 2020 across eight subjects, amounting to approximately 347 K papers. Both the `.tex` source and compiled `.pdf` are retained. The construction pipeline extracts every `\begin{tabular}…\end{tabular}` block, strips floating environments such as `figure` and `subfigure`, citations `\cite{}`, references `\ref{}`, labels `\label{}`, graphics `\includegraphics{}`, comments `%`, and the preceding `\~`, and removes nested tabulars entirely to avoid ambiguity [2105.06400].

LaTeX normalization is central to TabLeX. The preprocessing retains only alignment specifiers `l`, `c`, `r` and `|` for vertical lines; row delimiters `\\` and `\tabularnewline`; and key commands such as `\hline`, `\multicolumn`, `\multirow`, `\toprule`, `\midrule`, and `\bottomrule`. Rare environment tokens with frequency below 5,000 are replaced by a single placeholder `\LATEX_TOKEN` to cap vocabulary size [2105.06400].

The two task definitions are sharply separated. In TSD, all cell contents are replaced with the single token `CELL`, so the target is a “CELL-only” version of the tabular environment. In TCD, the target is the full sequence of tokens inside `tabular`, including letters, digits, `&`, `\$`, braces, and math- and text-mode commands [2105.06400]. A preprocessed example is:

```latex
\begin{tabular}{l|cc}
\hline
CELL & CELL & CELL \\
\hline
CELL & CELL & CELL \\
\end{tabular}
```

The released subsets are defined by maximum output length. TSD-250 contains 2,938,392 images with a train/validation/test split of 2,350,713 / 293,839 / 293,840. TSD-500 contains 3,191,891 images with 2,553,512 / 319,189 / 319,190. TCD-250 contains 1,105,636 images with 884,508 / 110,564 / 110,564. TCD-500 contains 1,937,686 images with 1,550,148 / 193,769 / 193,769 [2105.06400].

This shift from HTML-style targets to LaTeX-native targets is methodologically important. Structure extraction and content extraction are no longer conflated, and the output space directly matches the typesetting language used by scientific articles. The paper contrasts this with TableBank, PubTabNet, Table2LaTeX, and SciTSR, emphasizing LaTeX-native ground truth, massive scale, font and aspect-ratio diversity, and a clean split between structure and content tasks [2105.06400].

## 4. Rendering diversity, release format, and annotation regime

TabLeX is designed to expose models to variation that is common in scientific tables but underrepresented in static benchmarks. Each table is rendered in 12 different LaTeX font packages, including `mathptmx`, `bookman`, `charter`, `utopia`, `tgbonum`, `courier`, and `helvet`. For each font variant, two resizing schemes are applied: “Conserved,” in which the largest side is 400 px and the other side is scaled to preserve ratio with `blur=0.8`, and “Fixed,” in which the image is resized to exactly `400 × 400 px` with `blur=0.8` [2105.06400].

The benchmark includes multirow and multicolumn cells, horizontal and vertical lines, mathematical symbols, and varying row and column counts; nested environments are removed. Average row count is approximately 6–8 in TSD, and average column count is approximately 4–5 [2105.06400]. The images are JPEG at 300 dpi with white background and no transparency, while the LaTeX targets are plain-text `.txt` files containing tokenized sequences [2105.06400].

The release structure is task-specific and aspect-ratio-specific, with directories such as `TSD-250/conserved/train/000001.jpg` and `000001.txt`. A `metadata.json` file records per-sample fields including `id`, `split`, `task`, `font`, `aspect_ratio`, `num_rows`, `num_cols`, and `token_count`. The dataset is released under CC-BY-NC-SA 4.0 and is downloadable from `https://www.tinyurl.com/tablatex` [2105.06400].

Later work reuses related LaTeX-derived corpora with different image units and annotation granularity. In the active-learning study, TableBank-LaTeX is described as a table-detection corpus comprising more than 200,000 page images rendered from LaTeX source files of scientific papers, each with COCO/JSON bounding-box annotations around every table and a split of 187,199 training images, 7,265 validation images, and 5,719 test images [2509.20003]. In the Table2LaTeX-RL study, “TableBank–LaTeX” denotes 1,209,986 table-image/LaTeX source pairs collected from open-access arXiv papers from October 2017 to April 2023, where each example consists of a rasterized PNG table image and the ground-truth LaTeX code between `\begin{tabular}` and `\end{tabular}` [2509.17589].

The shared principle is that the LaTeX source acts as supervision, but the representation layer differs substantially: JSON bounding boxes for detection, HTML-like tags for simplified recognition, or raw LaTeX tokens for faithful tabular reconstruction. This suggests that “TableBank-LaTeX” is best understood as a LaTeX-supervised benchmark lineage rather than a single immutable file release.

## 5. Evaluation protocols and empirical baselines

The original TableBank paper reports strong baselines for both detection and recognition. For table detection, the architecture is Faster R-CNN in the Detectron implementation with ResNeXt-101 and ResNeXt-152 backbones, trained on 4 × NVIDIA P100 GPUs with data-parallel SGD and a mini-batch of 16 images; inference uses confidence threshold at least 0.90. On the LaTeX-subset test set, the ResNeXt-152 model trained on LaTeX achieves Precision \(= 0.9867\), Recall \(= 0.9754\), and F1-score \(= 0.9810\). For structure recognition, the baseline is an image-to-markup OpenNMT system with a CNN encoder and LSTM decoder with attention, trained with learning rate 0.1, batch size 24, and vocabulary size 11; the LaTeX-only model achieves BLEU \(= 76.53\) on the LaTeX test set and BLEU \(= 40.48\) on the Word test set [1903.01949].

TabLeX evaluates a different problem: auto-regressive generation of LaTeX tokens from table images. The reported baseline, TIE-ResNet-Transformer (“TRT”), uses a partial ResNet-101 encoder up to the 3rd intermediate block, followed by an FC layer that produces a \(625 \times 1024\) feature map projected to \(625 \times 256\). The decoder is a stacked Transformer with 4 heads in the encoder and 8 heads in the decoder, with learned positional embeddings, and decoding is greedy and auto-regressive. Training uses cross-entropy with label smoothing \(\epsilon = 0.1\), Adam with initial learning rate \(0.1\), 2 K warm-up steps, Noam decay, batch size 32, 10 epochs, dropout \(= 0.1\), and 4 × NVIDIA V100 GPUs [2105.06400].

The evaluation metrics on TabLeX are Exact Match Accuracy (EMA), BLEU-4 using SacreBLEU, and Word Error Rate (WER). The results summary is as follows: TSD-250 reaches EMA up to 74%, BLEU approximately 75, and WER approximately 4%; TSD-500 has EMA approximately 71%, BLEU approximately 83, and WER approximately 2.8%; TCD-250 has EMA approximately 21%, BLEU approximately 96, and WER approximately 14–16%; TCD-500 has EMA approximately 11%, BLEU approximately 94, and WER approximately 11–14% [2105.06400].

The discrepancy between very high BLEU and low exact match in content extraction is explained by specific failure modes. Missing `\$` or `\{ / \}` tokens are especially damaging: removing them from evaluation raises EMA from approximately 21% to approximately 75% on TCD-250. Long token sequences in TCD-500 degrade EMA sharply. Conserved aspect ratios are slightly harder than fixed squares, and complex multirow and multicolumn spans remain challenging [2105.06400]. This suggests that syntactic exactness, rather than broad lexical overlap, is the dominant bottleneck in LaTeX table generation.

A related sequence-modeling study, “Tables to LaTeX,” reports that a Fully-Gated ResNet-Transformer achieves exact match accuracy of 70.35 on table structure extraction and that a Partially-Gated variant reaches 49.69% on content extraction, with especially strong row and column accuracy but persistent errors on delimiters, punctuation, and span arguments [2210.17246]. Although that work uses a distinct Computer-Science-only dataset rather than TableBank-LaTeX proper, its findings align with the failure analysis observed on TabLeX.

## 6. Later reuse, active learning, and reinforcement learning extensions

In subsequent work, TableBank-LaTeX becomes a testbed for annotation-efficient detection and high-fidelity code generation rather than only a static benchmark. The active-learning paper studies page-level table detection on the TableBank-LaTeX corpus with more than 200,000 page images. The iterative active-learning loop begins with a small seed set \(D_0^L\), trains a detector \(f_t\), scores the unlabeled pool with uncertainty \(U(x)\) and diversity \(D(x)\), combines them through
\[
A(x)=\alpha U(x)+\beta D(x), \quad \alpha,\beta>0,\ \alpha+\beta=1,
\]
and repeatedly acquires the top-\(k\) images until the annotation budget \(B\) is exhausted [2509.20003].

Two detector families are evaluated: CascadeTabNet, described as a multi-stage detector built on Cascade-Mask R-CNN with a ResNet-50+FPN backbone, three sequential detection heads, and integrated mask segmentation; and YOLOv9, described as a single-stage, anchor-based network with a CSPDarknet backbone, PANet path aggregation, and decoupled detection heads. Common training settings are SGD with momentum 0.9, weight decay \(10^{-4}\), initial learning rate 0.01 with step-LR decay at epochs 8 and 12, batch size 16, 16 epochs per active-learning round, \(k=2{,}000\) images per round, and total budget \(B=10{,}000\) [2509.20003].

The reported mean Average Precision at IoU at least 0.5 shows that active-learning selection can approach or exceed random sampling under the same budget. Fully supervised models trained on all 187,199 images achieve approximately 96.1% mAP for CascadeTabNet and 95.4% for YOLOv9. With only 2,000 images, YOLOv9 with the Table-Count strategy reaches 89.2% mAP, matching random sampling at 8,000; MaskAmb with CascadeTabNet reaches 92.9% mAP at 10 K, outperforming random by 1.7%; and the paper concludes that a hybrid strategy mixing entropy and diversity with task-specific ambiguity cues yields annotation-cost savings of 20%–75% [2509.20003].

The Table2LaTeX-RL paper uses a different TableBank-LaTeX corpus for multimodal generation. It collects 1,209,986 table-image/LaTeX pairs from open-access arXiv papers, tags each table’s structural complexity by counting `\multirow` or `\multicolumn` commands and cell count, and trains with two stages: supervised fine-tuning on all 1.2 M pairs, followed by reinforced fine-tuning on 5,936 of the hardest “complex” tables using Group Relative Policy Optimization with dual rewards [2509.17589].

The base multimodal large language models are InternVL2-1B and Qwen2.5-VL-3B. Inputs follow the pattern `<image>` plus “Convert this table to LaTeX,” and the decoder generates the full `\begin{tabular}…\end{tabular}` sequence. The reinforcement stage introduces a visual fidelity reward from CW-SSIM and a structure reward from TEDS-Structure; the rewards are binarized as \(r_v=1\) if CW-SSIM is at least 0.6 and \(r_s=1\) if TEDS-Structure is at least 0.9, with clipping \(\varepsilon=0.2\) and KL weight \(\beta=0.02\) [2509.17589].

On the complex subset of the held-out test set, the reported scores are: Mathpix (2025) with CW-SSIM 0.4862, compile-ratio 0.9889, TEDS-Structure 0.8100, and TEDS 0.7176; Qwen2.5-VL-72B (SFT) with 0.5112, 0.9335, 0.8334, and 0.7448; and Qwen2.5-VL-3B-VSGRPO with 0.6145, 0.9917, 0.9218, and 0.8673 [2509.17589]. The paper attributes the largest improvements to structurally complex tables, particularly deep nesting and merged-cell layouts.

The broader significance of these later studies is not that TableBank-LaTeX has become uniform, but that LaTeX-supervised table corpora support multiple methodological regimes: weakly supervised detection, token-level structure recovery, content generation, annotation-efficient acquisition, and reinforcement learning over rendered outputs.

## 7. Limitations, misconceptions, and planned extensions

The main misconception is that all “TableBank-LaTeX” results refer to the same task and same annotation space. In fact, the sources describe page-level detection with bounding boxes, HTML-style structure recognition, CELL-only structural LaTeX generation, full-content LaTeX generation, and multimodal reinforcement learning on generated code [1903.01949], [2105.06400], [2509.17589]. Direct metric comparison is therefore unsafe unless the supervision format and evaluation protocol match.

Each benchmark also has explicit limitations. In TableBank, the HTML-like structure target does not explicitly encode `\multicolumn` or `\multirow`, and the paper reports typical failures such as partial-detection, un-detection, and mis-detection; BLEU drops sharply as structure length increases, with exact-match approximately 57% for fewer than 40 tokens, approximately 40% for 40–80, and below 9% above 200 [1903.01949]. In TabLeX, the authors state that current state-of-the-art table extraction models fail on even simple table images; exact matching deteriorates strongly for content extraction, especially on long sequences and when delimiter tokens are omitted [2105.06400]. In Table2LaTeX-RL, extremely long tables beyond 3,000 characters may truncate or mis-nest, very irregular cell content can still produce minor LaTeX compile errors, and binary reward thresholds can provide coarse feedback [2509.17589].

Planned extensions are likewise explicit. The TabLeX authors plan to continuously scrape new arXiv preprints to add rarer environments, wider table shapes, deeper nesting, side-notes, footnotes, colored rows, and mixed-mode cells, and to provide bounding-box coordinates for hybrid image+PDF extraction research [2105.06400]. The RL paper proposes replacing binary thresholds with continuous differentiable rewards, integrating content-level reward for cell textual accuracy, scaling reinforced fine-tuning to more medium tables and mixed complexities, reducing training overhead through an end-to-end differentiable rendering proxy, and adapting to scanned or photographed tables with noise, distortion, and non-standard fonts [2509.17589].

Taken together, these limitations and extensions indicate that the central challenge in TableBank-LaTeX-style research is not merely locating tables or recovering grid topology. It is achieving exact, compilable, and visually faithful recovery of scientific tables under realistic variation in layout, typography, mathematical notation, and sequence length.

Source: https://www.emergentmind.com/topics/tablebank-latex