---
title: 'TabStruct Benchmark: Synthetic Data Structure'
url: https://www.emergentmind.com/topics/tabstruct
type: topic
---

# TabStruct Benchmark: Synthetic Data Structure

Searching arXiv for the exact topic and closely related work to ground the article in the latest papers.
TabStruct is a benchmark and evaluation framework for synthetic tabular data that treats **structural fidelity** as a first-class evaluation dimension. In this line of work, the central question is not only whether a generator matches marginals, preserves privacy, or supports downstream prediction, but whether it preserves the **causal or conditional-independence structure** of the reference data. The initial TabStruct benchmark introduced structural fidelity through conditional-independence testing on datasets with expert-validated causal graphical structures, while a later extension broadened the framework to larger-scale benchmarking and introduced **global utility** as an SCM-free proxy for global structural fidelity on real-world datasets [2503.09453][2509.11950].

## 1. Conceptual basis and motivation

TabStruct was introduced in response to a recurring limitation in tabular generative modeling evaluation: standard benchmarks typically emphasize **density estimation**, **downstream utility**, and **privacy preservation**, but do not directly test whether synthetic data preserves the inter-feature structure of the real data. The motivating claim is that tabular data is heterogeneous and that its organization is often better characterized by **causal structure** and **conditional-independence relations** than by the kinds of modality-specific priors commonly used for images or text [2503.09453].

The framework therefore distinguishes between a generator that is merely useful for a particular predictive task and a generator that captures the broader organization of the data-generating process. This distinction is operationalized through the contrast between **local** and **global** structure. Local structure concerns relations involving a designated target variable \(y\); global structure concerns the full dependency pattern among variables. A generator can therefore perform well under task-specific utility metrics while still failing to reproduce the global structure of the reference data. This distinction is explicit in both TabStruct papers and is central to the benchmark’s rationale [2503.09453][2509.11950].

A closely related motivation is evaluator bias. Downstream utility depends on the choice of target and predictor, and the framework argues that this can obscure structural failures. In the later benchmark, this concern is extended into a broader claim that conventional evaluation dimensions are **complementary, not interchangeable**, and that structural fidelity must be assessed alongside them rather than inferred from them [2509.11950].

## 2. Structural fidelity and its formalization

In TabStruct, **structural fidelity** is defined as the alignment between the structural relations in the reference data \(\mathcal{D}_{\text{ref}}\) and those in the synthetic data \(\mathcal{D}_{\text{syn}}\). The original formulation evaluates this alignment at the level of the **Markov equivalence class**, represented by a **CPDAG**, rather than at the full DAG level. The stated reason is methodological: exact edge-direction recovery is difficult and highly dependent on the causal discovery method, whereas equivalence-class-level conditional independences are more robust [2503.09453].

The original benchmark decomposes structural fidelity into two measures. **Global independence** evaluates all conditional-independence relations in the dataset. **Local independence** restricts attention to relations involving the target variable \(y\). Each conditional-independence test is treated as a binary classification problem, where \(1\) denotes independence and \(0\) denotes dependence, and the score is computed using **balanced accuracy**. This makes structural fidelity a direct property of the agreement between the synthetic data and the graph-implied independence structure, rather than a side effect of predictive performance [2503.09453].

The later paper retains the same structural concern but presents a more explicit conditional-independence formalization. When the true SCM is known, it defines a CI set \(\mathcal{C}_{\text{global}}\) that includes both d-separation-based independencies and d-connections obtained by removing variables from separating sets:

$$
\mathcal{C}_{\text{global}} \coloneqq \bigl\{(x_j \perp x_k \mid S_{j,k}) \mid  S_{j,k}\subseteq\mathcal{X}\setminus\{x_j,x_k\}\bigr\} \cup \bigl\{(x_j \not\!\perp x_k \mid \widehat{S}_{j,k}) \mid \widehat{S}_{j,k}\subsetneq S_{j,k}\bigr\}.
$$

The CI score for a dataset \(\mathcal{D}\) is then defined as

$$
\text{CI}(\mathcal{C}, \mathcal{D}) = \frac{1}{|\mathcal{C}|} \sum_{\mathcal{C}} \mathbbm{1}\Bigl[ \widehat{\mathcal{I}_{\alpha}(x_j, x_k \mid S_{j,k}, \widehat{S}_{j,k};\mathcal{D}) = 1 \Bigr],
$$

with significance level \(\alpha=0.01\). The same local/global distinction is preserved: **local CI** focuses on relations involving the prediction target, while **global CI** measures fidelity to the full causal organization [2509.11950].

## 3. Benchmark construction and evaluation scope

The original TabStruct benchmark is built around **seven benchmark datasets** drawn from **bnlearn**, each paired with an **expert-validated structural causal model**. These datasets span classification and regression settings and a range of structure sizes, including small, medium, and large graphs. The benchmark includes Sangiovese, Insurance, Hailfinder, ANDES, Healthcare, MEHRA, and ARTH150, and the reference datasets are created by **prior sampling from SCMs** [2503.09453].

That original benchmark evaluates **nine concrete methods across eight generator categories**, plus the reference dataset itself. The categories include interpolation, Bayesian networks, VAE-based models, GAN-based models, normalizing flows, diffusion models, tree-based models, and LLM-based generators. The explicit method list is SMOTE, BN, TVAE, GOGGLE, CTGAN, Neural Spline Flows, TabDDPM, ARF, and GReaT [2503.09453].

The later TabStruct benchmark expands substantially. It evaluates **13 tabular generators from nine distinct categories across 29 datasets**. The dataset inventory is split between **SCM datasets**, where causal structure is available, and **real-world datasets**, where it is not. The real-world portion includes **14 classification datasets** and **9 regression datasets**, while the SCM portion includes six benchmark datasets in the main experiments and five additional large SCM datasets in the appendix. The evaluation pipeline uses an **80\% train / 20\% test** split, then divides train into **90\% reference** and **10\% validation**, repeating this **10 times** [2509.11950].

The following table summarizes the scope of the two TabStruct benchmark formulations.

| Aspect | Original TabStruct | Extended TabStruct |
|---|---|---|
| Core papers | [2503.09453] | [2509.11950] |
| Datasets | 7 bnlearn SCM datasets | 29 datasets total |
| Structural supervision | Expert-validated SCMs | SCM and real-world settings |
| Generator coverage | 9 methods, 8 categories | 13 generators, 9 categories |
| Structural metrics | Local/global independence | Local/global CI and global utility |

This progression suggests a shift from a benchmark centered on expert-validated causal graphs toward a broader evaluation suite that can also operate when such graphs are unavailable.

## 4. Metrics beyond structure

TabStruct does not replace conventional metrics; it places them alongside structural fidelity. The original benchmark reports **Shape**, **Trend**, **\(\alpha\)-precision**, **\(\beta\)-recall**, **Accuracy**, **RMSE**, **DCR**, and **Authenticity**, with downstream utility computed using the standard **train-on-synthetic, test-on-real** protocol and averaged across six downstream predictors: Logistic Regression, KNN, MLP, Random Forest, XGBoost, and TabPFN [2503.09453].

The later benchmark uses a closely related but broader evaluation layout. It groups metrics into **density estimation**, **privacy preservation**, **ML efficacy**, and **structural fidelity**. Its density-estimation metrics are **Shape**, **Trend**, **\(\alpha\)-precision**, and **\(\beta\)-recall**; privacy metrics are **DCR** and **\(\delta\)-Presence**; ML efficacy is represented by local utility; and structural fidelity includes **local CI**, **global CI**, and **global utility**. The downstream predictor ensemble for utility is expanded to **nine models**: LR, KNN, MLP, RF, Extra Trees, LightGBM, CatBoost, XGBoost, and TabPFN [2509.11950].

A central empirical claim of the benchmark line is that these dimensions are not interchangeable. In the original study, the reported rank correlations show that downstream utility aligns much more strongly with **local** structure than with **global** structure. The reported values are **0.90** for accuracy versus local independence and **0.57** for accuracy versus global independence, while **RMSE versus local independence** is **0.77** and **RMSE versus global independence** is **0.33**. The stated interpretation is that good predictive performance can coexist with weak preservation of the overall data graph [2503.09453].

This is reinforced in the later work, which states that density estimation and privacy preservation do not strongly correlate with global CI, and that ML efficacy is often biased toward local target-specific structure. The benchmark therefore treats structure as an independent dimension that must be measured directly [2509.11950].

## 5. Global utility and the SCM-free extension

The principal methodological addition in the later TabStruct paper is **global utility**, introduced to assess global structural fidelity **without access to ground-truth causal structures**. The motivation is that CI-based structural evaluation requires known SCMs, which are rarely available for real-world tabular datasets [2509.11950].

For each variable \(x_j\), the framework treats that variable as a prediction target and measures the relative predictive performance of a dataset \(\mathcal{D}\) against the reference dataset \(\mathcal{D}_{\text{ref}}\). The per-variable utility is defined as

$$
\text{Utility}_j(\mathcal{D}) \coloneqq \begin{cases} \text{Perf}_j(\mathcal{D}_{\text{ref}})^{-1}\text{Perf}_j(\mathcal{D}), & \text{if } x_j \text{ is categorical},\\[8pt] \text{Perf}_j(\mathcal{D})^{-1}\text{Perf}_j(\mathcal{D}_{\text{ref}}), & \text{if } x_j \text{ is numerical}. \end{cases}
$$

The paper uses **balanced accuracy** for categorical targets and **RMSE** for numerical targets. These per-variable scores are then averaged:

$$
\text{Global Utility}(\mathcal{D}) \coloneqq \frac{1}{D+1}\sum_{j=1}^{D+1} \text{Utility}_j(\mathcal{D}).
$$

The intended interpretation is that if the global dependency structure is preserved, every variable should remain predictable from the others in a manner comparable to the reference data. The paper connects this intuition to the Markov blanket perspective and positions global utility as a task-independent and domain-agnostic lens on structure preservation [2509.11950].

The reported quantitative justification is strong. The paper gives a **Spearman correlation between global utility and global CI of 0.84**, with \(p<0.001\). It also reports that normalization is important: **global utility with normalized performance** has Spearman correlation **0.84** with global CI, whereas **global utility with absolute performance** drops to **0.57**. The paper further reports a runtime example of **0.64s per 1000 samples** under a small predictor setup, compared with about **1.21s per 1000 samples** for local utility at comparable stability [2509.11950].

A plausible implication is that the later TabStruct formulation is not merely an enlarged benchmark, but a methodological shift from “evaluate structure when graphs are known” to “approximate structure faithfully when graphs are unknown.”

## 6. Empirical findings, model rankings, and disambiguation

Across both TabStruct papers, the dominant empirical result is that **existing generators still struggle to preserve structure**. In the original benchmark, the best local-independence score among generators in classification is reported as **74.02\%** for SMOTE, whereas the reference dataset is **100\%**. For **global independence**, the gap from reference is described as around **35 percentage points**. The paper uses these gaps to argue that structure is substantially harder to preserve than conventional task-level or distributional behavior [2503.09453].

The two studies also identify recurring model-family tradeoffs. The original paper reports that **SMOTE** performs strongly on downstream utility and local independence but poorly on global independence; **BN** performs relatively well on structural fidelity but degrades on larger datasets; **TVAE** is competitive on global independence; **TabDDPM** performs well on privacy but not on structure; and **GOGGLE**, despite its relational-structure design, performs poorly in structural fidelity in those experiments [2503.09453].

The later paper sharpens these conclusions at larger scale. It reports that diffusion-based models—specifically **TabDDPM**, **TabSyn**, and **TabDiff**—are consistently the Top-3 performers in **global CI** and **global utility**, while **SMOTE** remains strong for **local utility** and **local CI**. It also states that **BN** and **GOGGLE** do not dominate despite their structural inductive bias, and that **GReaT** performs weakly on structural fidelity, which the paper attributes to feature-order sensitivity and the mismatch between autoregressive factorization and permutation-invariant tabular structure [2509.11950].

Because the term “TabStruct” can be confused with table-structure-recognition work in document AI, a disambiguation is useful. In the standalone title usage covered here, **TabStruct** denotes a benchmark and evaluation framework for **synthetic tabular data** and **structural fidelity** [2503.09453][2509.11950]. By contrast, **TabStruct-Net** is a separate image-based **table structure recognition** system that combines top-down cell detection with bottom-up graph reasoning to recover table layouts from document images [2010.04565]. The shared string “TabStruct” therefore spans two different research contexts: one in **tabular data generation evaluation**, the other in **document table parsing**.

This suggests that the most stable encyclopedia-level definition of TabStruct is the benchmark line concerned with **measuring whether synthetic tabular data preserves causal and conditional-independence structure**, while acknowledging that the same lexical root also appears in table-structure-recognition nomenclature.

Source: https://www.emergentmind.com/topics/tabstruct