Markdown TEDS: Tree-Edit Distance for Tables

• The paper introduces Markdown TEDS, adapting tree-edit-distance techniques to evaluate Markdown tables with a focus on holistic structural and content fidelity.
• It employs Markdown-specific preprocessing including table isolation, fuzzy merging, and optimal multi-table matching to repair fragmented outputs.
• Empirical results in financial table conversion show Markdown TEDS effectively discriminates model performance and provides a quantifiable accuracy metric.

Searching arXiv for the cited papers to ground the article in current records. Markdown Tree-Edit-Distance-based Similarity (Markdown TEDS) is a custom evaluation metric for comparing model-generated and reference tables encoded in Markdown. It was introduced to obtain a single, holistic score for structural and content fidelity in financial table conversion, and it adapts the Tree-Edit-Distance-based Similarity framework—originally designed for tables represented as HTML trees—to Markdown output through Markdown-aware parsing, table isolation, fuzzy table merging, and optimal multi-table matching (Tan et al., 4 Aug 2025).

1. Definition and formal basis

Markdown TEDS evaluates the similarity between model-generated and ground truth tables encoded in Markdown, capturing table structure, table content, and the holistic document-level alignment between reference and predicted tables (Tan et al., 4 Aug 2025). Its immediate purpose is evaluation rather than transformation: it scores outputs of Markdown-generating systems by comparing the tree structures induced by Markdown tables.

The underlying primitive is tree edit distance. In the classical setting, tree edit distance is defined for rooted, ordered trees whose nodes are labeled from an alphabet, and it is the minimum cost of transforming one tree into another by elementary edit operations consisting of deleting and relabeling existing nodes, as well as inserting new nodes (Bringmann et al., 2017). More generally, tree edit distance is the minimum-cost sequence of insertions, deletions, and replacements required to transform one tree into another, and it can be computed by dynamic programming over trees and forests (Paaßen, 2018).

Markdown TEDS uses the normalized similarity computed as in standard TEDS:

$$\mathrm{TEDS}(T_{pred}, T_{gt}) = 1 - \frac{\mathrm{TED}(T_{pred}, T_{gt})}{\max(|T_{pred}|, |T_{gt}|)}.$$

Here, $T_{pred}$ and $T_{gt}$ are the tree representations of the predicted and ground-truth tables, $\mathrm{TED}$ is the tree edit distance, and $|T|$ is the node count in tree $T$ (Tan et al., 4 Aug 2025). This normalization makes the score a similarity measure rather than a raw edit cost.

A common misunderstanding is to treat Markdown TEDS as a token-overlap metric over Markdown strings. The metric is instead tree-based: Markdown tables are parsed into tree representations, and the score is driven by the edit distance between those trees rather than by surface $n$-gram overlap or cellwise string matching alone (Tan et al., 4 Aug 2025).

2. Evaluation workflow and Markdown-specific preprocessing

Markdown TEDS proceeds in three stages. First, it performs table structure isolation. Raw output Markdown is parsed and only the table constructs are extracted; non-table text such as headings and explanations is ignored, making the metric agnostic to model performance on document-level prose (Tan et al., 4 Aug 2025). This design makes the metric narrowly focused on tabular fidelity.

Second, it performs fuzzy table merging. In Markdown generation, a single logical table may be emitted as several smaller Markdown tables. Markdown TEDS merges tables when their headers are textually similar above a defined threshold, exemplified in the description as $>80\%$ average cell-wise similarity (Tan et al., 4 Aug 2025). This step is explicitly intended to avoid penalizing format splitting when the predicted fragments still correspond to one logical table.

Third, it performs optimal multi-table matching. Because a document can contain several tables, the method computes similarity scores for every pairing of output and ground-truth tables and then uses the Hungarian algorithm to compute the optimal one-to-one matching maximizing total similarity (Tan et al., 4 Aug 2025). The final document-level score reflects the aggregate quality of this matching.

These preprocessing stages are not peripheral heuristics; they are part of what distinguishes Markdown TEDS from a direct application of HTML-oriented TEDS. They encode failure modes that are specific to Markdown generation, especially fragmentation and mixed document content (Tan et al., 4 Aug 2025).

3. Tree representation and relation to traditional TEDS

Each extracted Markdown table is parsed into a tree representation, analogous to the tree representation used by TEDS for HTML tables (Tan et al., 4 Aug 2025). The tree edit distance is then calculated between the prediction and the ground truth, quantifying the minimal series of node insertions, deletions, or substitutions required to transform one structure into the other (Tan et al., 4 Aug 2025).

Traditional TEDS was introduced for tables represented as HTML trees; Markdown TEDS retains the same tree-edit-distance core while changing the representation and augmenting the evaluation pipeline for Markdown-specific behavior (Zhong et al., 2019). The key distinctions are structural rather than merely syntactic.

Aspect	Traditional TEDS	Markdown TEDS
Input representation	HTML tables	Markdown tables
Preprocessing	None/HTML focus	Table isolation, fuzzy merging, multi-table matching
Target behavior	HTML table comparison	Markdown table comparison with fragmentation handling

A broader computational point is that Markdown TEDS inherits a nontrivial algorithmic primitive. Historically, the fastest known general algorithm for tree edit distance ran in cubic $O(n^3)$ time (Bringmann et al., 2017). More recent work establishes fine-grained equivalence between tree edit distance and APSP and gives subcubic algorithms for weighted and unweighted ordered-tree settings, including $O(n^3/2^{\Omega(\sqrt{\log n})})$ for weighted TED and $T_{pred}$0 for unweighted TED (Nogler et al., 2024). This suggests that the efficiency of Markdown TEDS depends not only on Markdown parsing and matching heuristics but also on the complexity of the underlying tree comparison routine.

4. Empirical use in financial table conversion

Markdown TEDS was introduced in the evaluation of a fine-tuned vision-LLM for Markdown conversion of financial tables in Malaysian audited financial reports (Tan et al., 4 Aug 2025). In that study, the metric was paired with a criteria-based LLM-as-a-judge framework: the former supplied a holistic structural score, while the latter supplied fine-grained accuracy judgments.

The reported benchmark involved 100 out-of-sample tables. Markdown TEDS was used to compare fine-tuned and untuned models, larger and smaller vision-LLMs, and proprietary versus self-hosted systems (Tan et al., 4 Aug 2025). In the reported results, the fine-tuned Qwen2.5-VL-7B model achieved a 96.53 Markdown TEDS score and 92.20% overall accuracy, substantially above the untuned base model and above several larger or proprietary systems (Tan et al., 4 Aug 2025).

Model	Overall Accuracy (%)	TEDS (%)
Qwen2.5 VL 7B (Ours, FT)	92.20	96.53
Qwen2.5 VL 7B (base)	32.80	52.08
Gemini 2.5 Flash	82.40	79.19
GPT-4o	65.20	74.41

Within that evaluation, Markdown TEDS was described as providing a clear, quantifiable, and holistic measure of output quality and as effectively discriminating improvements in structural reproduction beyond what token-by-token metrics provide (Tan et al., 4 Aug 2025). The intended contrast is with metrics such as BLEU, accuracy, or F1, which primarily reflect token-level correctness or semantic similarity and can miss major structural defects (Tan et al., 4 Aug 2025).

5. Place within the broader tree-edit-distance literature

Markdown TEDS belongs to a wider family of domain-adapted tree similarity measures in which the basic edit-distance machinery is retained while the representation, cost model, or matching regime is specialized to a target structure.

In topological data analysis, local tree edit distance was introduced for merge trees in order to compare all pairs of subtrees and support finer, multiscale analysis; it enables local similarity detection that global tree-edit-distance formulations do not directly provide (Sridharamurthy et al., 2021). In cyber-security, a modified Zhang-Shasha-based approach incorporated semantic similarity between node labels using sentence embeddings, replacing exact label equality with thresholded semantic equivalence (Schiele et al., 4 Mar 2025). In code analysis, AST-based Tree Similarity of Edit Distance normalized edit distance by maximum node count and was presented as effective across multiple programming languages, with the authors explicitly noting adaptability to structured data beyond code (Song et al., 2024).

Markdown TEDS differs from these variants in a specific way. It does not redefine node substitution costs by semantics, nor does it target local subtree discovery. Its distinctive contribution is the combination of Markdown-aware parsing with table-only isolation, fragmentation repair by fuzzy merging, and document-level optimal matching across multiple tables (Tan et al., 4 Aug 2025). In that sense, it is a task-level adaptation of TEDS rather than a new general theory of edit costs.

6. Advantages, limitations, and interpretive issues

Several advantages are stated explicitly. Markdown TEDS is holistic in that it accounts for both table structure and cell content rather than only string overlap; it is Markdown-native rather than HTML-native; it is robust to common generative quirks such as table splitting and header shifts; and it is suitable for large-scale, automatic benchmarking with fine granularity (Tan et al., 4 Aug 2025). These properties explain why it was used alongside, rather than replaced by, criteria-based LLM judging.

Its limitations are equally important. Markdown TEDS focuses on tables only and ignores errors outside table regions; it requires accurate parsing, since the parser and merging heuristic must reliably reconstruct the intended table logic; and it is domain-specific, being tuned for Markdown as used in financial documents and potentially requiring adaptation for other domains or markup styles (Tan et al., 4 Aug 2025). A frequent misconception is therefore to interpret a high Markdown TEDS score as a complete document-understanding score. The metric does not evaluate surrounding narrative text, explanations, or document-level prose (Tan et al., 4 Aug 2025).

Another interpretive issue concerns what is actually being rewarded. Because non-table content is discarded and fragmented tables may be merged before scoring, Markdown TEDS is designed to privilege logical tabular fidelity over literal formatting fidelity (Tan et al., 4 Aug 2025). This is a methodological choice rather than a universal property of tree-edit-distance-based metrics.

This suggests a broader implication: when outputs are hierarchical and structurally meaningful, a useful evaluation metric may require not only a tree distance but also domain-specific preprocessing that determines what counts as comparable structure. Markdown TEDS instantiates that principle for Markdown financial tables, using tree edit distance as the scoring core and Markdown-specific normalization around it (Tan et al., 4 Aug 2025).