Tablext: Image-Based Table Extraction

• Tablext is a general-purpose system for extracting tables from images by converting various document formats into a structured CSV output.
• It employs a staged hybrid pipeline combining custom CNNs, YOLO, and OCR with computer-vision heuristics to localize tables and repair cell boundaries.
• The system achieves competitive precision and recall on diverse table layouts, including borderless and irregular designs, across scanned and locked documents.

Tablext is a general-purpose, image-based table extraction system that recovers both table structure and cell text from documents whose tables may or may not have machine-readable source text. Its defining architectural choice is a staged hybrid pipeline: arbitrary document formats are converted to images; a custom Convolutional Neural Network (CNN) and YOLO localize candidate tables; computer-vision heuristics recover high-level structure through real and inferred lines; a second CNN repairs candidate cell boundaries; and Tesseract OCR transcribes final cell crops into CSV output (Colter et al., 2021).

1. Scope, assumptions, and problem formulation

Tablext addresses table extraction from images rather than native document objects. PDFs can be rasterized with pdf2image, HTML can be rendered to images, and screenshots can be used directly. This makes the system largely independent of the original file format and allows one extraction method to be applied across scanned documents, image-only tables, locked PDFs after rasterization, ordinary text-based PDFs after rasterization, and tables with regular or irregular layouts, including tables with full borders, partial borders, or no explicit cell borders (Colter et al., 2021).

The system is not framed as a monolithic detector. It decomposes the task into table localization, high-level structural analysis, local cell correction, OCR, and structured output assembly. The rationale is explicitly hybrid: lightweight neural networks are used for hard perceptual decisions, conventional computer-vision heuristics are used for explicit geometric regularities such as line finding, and OCR is deferred until cell boundaries have been resolved. This division of labor is intended to let neural networks focus on completing complex tasks while image-processing methods efficiently complete the simpler ones (Colter et al., 2021).

The output target is a structured table representation in CSV. Tablext therefore differs from systems that only detect table regions or only recognize structure without content. It is also distinct from PDF-native parsers that rely on character-position metadata: its central promise is operation on image-rendered documents, including cases where no machine-readable text is available (Colter et al., 2021).

2. Detection and localization pipeline

Table localization combines a custom CNN-based procedure with YOLO. The custom detector does not regress table boxes directly. Instead, it first identifies page rows containing tables and then localizes columns within those row groups. Input pages are converted to grayscale, resized to width $800$ pixels, and sliced into overlapping horizontal strips of height $64$ pixels. Each strip predicts table presence only for its innermost $32$ rows. The vertical-localization CNN has $4$ output nodes, and node $i$, for $0 \le i \le 3$, predicts whether a table exists in row interval $[8i,\; 8i+7]$. Positive rows are grouped into contiguous table bands. For each such band, the corresponding region is resized to $400 \times 400$ and passed to a second CNN for horizontal localization, after which coordinates are mapped back to the original image (Colter et al., 2021).

YOLO is added because the two detectors have complementary error profiles. The custom CNN tends to over-propose regions because line structures strongly influence it, whereas YOLO usually detects the right table areas but often produces bounding boxes that do not fully cover the entire table. When their proposed regions overlap, Tablext takes their union. The stated preference is recall-oriented: in this pipeline, missing a table is worse than over-proposing one (Colter et al., 2021).

Stage	Main mechanism	Result
Vertical localization	Custom CNN on $64$-pixel strips	Table-bearing row bands
Horizontal localization	Second CNN on $400 \times 400$ band crops	Table $64$0-extent
Proposal consolidation	YOLO + union with custom CNN overlaps	Final table regions

For table-identification evaluation, the paper defines precision and recall using predicted table area $64$1 and labeled table area $64$2:

$64$3

On a custom dataset with over $64$4 tables, the custom CNN alone obtained recall $64$5, precision $64$6, and F1 $64$7; YOLO alone obtained recall $64$8, precision $64$9, and F1 $32$0; and the combined system obtained recall $32$1, precision $32$2, and F1 $32$3 (Colter et al., 2021).

3. Structural reconstruction from real and inferred lines

Once a table crop is isolated, Tablext infers structure from two sources: real lines that are visibly present and inferred lines derived from data placement. Real-line identification approximates the second-derivative gradient field of the grayscale table image, using

$32$4

followed by $32$5 max-pooling with stride $32$6. Vertical lines are found where pixels have high $32$7 and similar $32$8 values along the segment; horizontal lines are found analogously. Detected segments are then extended across the full table height or width because the goal is structural division rather than literal line reconstruction (Colter et al., 2021).

Inferred lines are used for partially bordered or borderless tables. The operative assumption is that high-contrast non-line pixels correspond to data; after real lines are removed, a valid inferred separator is a straight line that does not intersect data. Because spanning headers can invalidate a zero-overlap rule, Tablext introduces an adaptive threshold-distance procedure. For vertical inferred lines, threshold distance is defined as the proportion of table height over which a candidate vertical line must avoid touching data; for horizontal inferred lines, the corresponding quantity is defined relative to table width. The algorithm initializes threshold_distance = 0.6, increases it by a small learned constant $32$9 while the number of separable inferred-line groups does not decrease, and returns the best inferred-line set when that condition fails (Colter et al., 2021).

Each inferred line receives a quality score equal to the percentage of the maximum possible line length that the line covers without intersecting data. Nearby inferred lines are grouped. If a real line lies within or very near such a group, only the real line is retained. Otherwise, the best inferred separator is chosen using a Simple Moving Average over quality scores looking forward and backward two pixels, with missing positions assigned score zero. Final structural lines are the union of retained real lines and selected inferred representatives, computed independently for vertical and horizontal directions (Colter et al., 2021).

This stage yields a dominant global grid, but it can over-segment cells that violate the regularized structure, especially spanning headers and other merged regions. That is why Tablext introduces a learned local repair stage rather than relying exclusively on heuristic structure inference (Colter et al., 2021).

4. Cell correction, OCR, and output representation

The second CNN operates on pairs of adjacent candidate cells produced by the structural-line stage. Its role is local repair: it predicts whether the left or top cell contains data, whether the right or bottom cell contains data, and whether the two candidate cells should actually be merged. Each adjacent cell is independently resized to $4$0, the pair is merged into a $4$1 image, and the crop is rotated if necessary so that the potentially improper split always appears as a vertical line in the center (Colter et al., 2021).

The CNN has three branches. The first branch takes the full $4$2 merged image and applies three $4$3 convolution layers with $4$4 filters each, followed by $4$5 max-pooling. The second branch begins with asymmetric average pooling of size $4$6, reducing the image to $4$7, then applies five $4$8-dimensional convolution layers with $4$9 filters of size $i$0. The third branch takes only the centermost $i$1 pixels and applies five $i$2 convolution layers with $i$3 filters each. After dropout with rate $i$4, flattened branch outputs are concatenated, passed through a dense layer of size $i$5, dropout $i$6, a second dense layer of size $i$7, and a final output layer with $i$8 nodes (Colter et al., 2021).

Training data for this network were derived from over $i$9 manually annotated tables from various domains. Tablext first applies its own line-identification algorithm to produce candidate cells, then converts all adjacent cell pairs into supervised examples using XML ground truth to determine emptiness and merge labels (Colter et al., 2021).

After cell repair, OCR is performed per non-empty cell using Tesseract. Structural processing is done on resized images for efficiency, but OCR is run on cell crops taken from the original-resolution table image because recognition on the resized representation is insufficient. The resulting text is placed into a CSV according to the recovered row and column positions. Merged cells are represented by storing content in the original merged cell and placing the keyword EXTEND with a directional arrow in the covered positions, preserving structural information in a CSV-friendly format (Colter et al., 2021).

5. Empirical standing, strengths, and failure modes

On the ICDAR 2013 table dataset, using the same precision/recall methodology as prior work, Tablext reported recall $0 \le i \le 3$0, precision $0 \le i \le 3$1, and F1 $0 \le i \le 3$2. The corresponding F1 values reported for DeepDeSRT and TableNet were $0 \le i \le 3$3 and $0 \le i \le 3$4, respectively. The paper emphasizes that Tablext did not train on ICDAR data, unlike DeepDeSRT and TableNet, which fine-tuned on the remaining ICDAR subset (Colter et al., 2021).

On a diverse benchmark of over $0 \le i \le 3$5 text-based PDF tables from various domains, Tablext reported recall $0 \le i \le 3$6, precision $0 \le i \le 3$7, and F1 $0 \le i \le 3$8, while Tabula reported recall $0 \le i \le 3$9, precision $[8i,\; 8i+7]$0, and F1 $[8i,\; 8i+7]$1. The comparison is particularly notable because Tabula was given the original text-based PDFs and their metadata, whereas Tablext was given only scanned images of those pages and used OCR (Colter et al., 2021).

The reported strengths are therefore specific. Tablext works from images rather than PDF or HTML internals; it handles partial borders, borderless regions, and spanning cells; it can process multi-column pages and multiple tables; and its hybrid design yields competitive benchmark accuracy with smaller, more focused learning components than a purely neural end-to-end parser (Colter et al., 2021).

The paper and supporting synthesis also make its limitations clear. OCR remains a bottleneck; one qualitative example notes that Tesseract failed to recognize the plus-minus symbol. The custom detector can over-propose line-rich non-table regions. There is a rare identification failure case when two adjacent tables in separate columns have very different heights, causing extra non-table area to be included. The structural heuristic assumes a dominant regular layout with local irregularities repaired afterward, so extremely unconventional layouts are less directly modeled. The paper does not discuss rotated or heavily skewed tables, so robustness there is unclear, and the system appears optimized for mostly horizontal text orientation (Colter et al., 2021).

6. Position in the broader table-understanding literature

Tablext belongs to the extraction branch of table research rather than the generation or transformation branches. In the extraction literature, it is closely related to other hybrid or structured systems. GTE formulates table detection and cell structure recognition jointly, using a containment-based penalty in GTE-Table and a hierarchical style-aware cell detector in GTE-Cell, and reports a significant $[8i,\; 8i+7]$2 improvement in the full table extraction system on ICDAR benchmarks together with greater than $[8i,\; 8i+7]$3 improvement in cell structure recognition over a vanilla RetinaNet model on FinTabNet (Zheng et al., 2020). TableZa represents a classical OCR-first post-detection alternative: it assumes the tabular area is already detected, then reconstructs rows and columns using morphology, contour reasoning, Tesseract OCR, and the “Least Collision method” for estimating separators in borderless tables (Banthia et al., 2021). The “Flexible Table Recognition and Semantic Interpretation System” goes further into semantic interpretation, combining two rule-based algorithms for complete table recognition with a graph-based interpretation method and reporting a complete information extraction F1 score of $[8i,\; 8i+7]$4 (Namysl et al., 2021).

Subsequent benchmark work broadened the evaluation environment that systems like Tablext inhabit. TabRecSet introduced a bilingual end-to-end table recognition dataset with $[8i,\; 8i+7]$5 images and $[8i,\; 8i+7]$6 tables, polygon annotations, and support for table detection, table structure recognition, and table content recognition in wild scenarios (Yang et al., 2023). PubTables-v2 extended the benchmark frontier to contextualized extraction with $[8i,\; 8i+7]$7 cropped tables, $[8i,\; 8i+7]$8 single pages, and $[8i,\; 8i+7]$9 full documents, including $400 \times 400$0 multi-page tables, and used that setting to motivate POTATR, an image-to-graph extension of the Table Transformer for comprehensive page-level table extraction (Smock et al., 11 Dec 2025). This suggests that Tablext’s staged design anticipated a continuing concern of the field: explicit structural reasoning remains competitive even as benchmarks move from cropped tables to full pages and full documents.

The name should also be distinguished from other table-centered research programs. Tablext is not a table-to-text generator: TabT5 is an encoder-decoder model for table-and-text-to-text generation with row and column embeddings and table-specific pre-training (Andrejczuk et al., 2022), whereas PixT3 recasts table-to-text as pixel-based image-to-text generation and is strongest in loosely controlled and open-ended settings (Alonso et al., 2023). It is not a dataset unification toolkit like TabGenie, which represents diverse data-to-text inputs as tables plus metadata and exposes that representation through a browser UI, CLI, and Python API (Kasner et al., 2023). Nor is it a column-transformation framework like TabulaX, which classifies source-target mappings into string-based, numerical, algorithmic, and general classes to synthesize interpretable transformation functions for heterogeneous joins (Nobari et al., 2024).

Within that broader landscape, Tablext is best understood as a hybrid extractor: an image-rendered document is transformed into structured CSV through localized neural detection, explicit geometric structure inference, learned cell repair, and OCR. Its significance lies less in any single component than in the architecture as a whole: the system assigns neural models and heuristics different responsibilities and thereby offers a concrete blueprint for general-format table extraction from images (Colter et al., 2021).