TabStruct Benchmark: Synthetic Data Structure

• TabStruct is a benchmark and evaluation framework for synthetic tabular data that measures structural fidelity through causal and conditional-independence tests.
• It distinguishes local and global structures by evaluating target-specific and full dependency patterns using balanced accuracy and RMSE.
• The framework complements conventional density, privacy, and utility metrics with structure-specific scores such as global utility.

Searching arXiv for the exact topic and closely related work to ground the article in the latest papers. TabStruct is a benchmark and evaluation framework for synthetic tabular data that treats structural fidelity as a first-class evaluation dimension. In this line of work, the central question is not only whether a generator matches marginals, preserves privacy, or supports downstream prediction, but whether it preserves the causal or conditional-independence structure of the reference data. The initial TabStruct benchmark introduced structural fidelity through conditional-independence testing on datasets with expert-validated causal graphical structures, while a later extension broadened the framework to larger-scale benchmarking and introduced global utility as an SCM-free proxy for global structural fidelity on real-world datasets (Jiang et al., 12 Mar 2025, Jiang et al., 15 Sep 2025).

1. Conceptual basis and motivation

TabStruct was introduced in response to a recurring limitation in tabular generative modeling evaluation: standard benchmarks typically emphasize density estimation, downstream utility, and privacy preservation, but do not directly test whether synthetic data preserves the inter-feature structure of the real data. The motivating claim is that tabular data is heterogeneous and that its organization is often better characterized by causal structure and conditional-independence relations than by the kinds of modality-specific priors commonly used for images or text (Jiang et al., 12 Mar 2025).

The framework therefore distinguishes between a generator that is merely useful for a particular predictive task and a generator that captures the broader organization of the data-generating process. This distinction is operationalized through the contrast between local and global structure. Local structure concerns relations involving a designated target variable $y$; global structure concerns the full dependency pattern among variables. A generator can therefore perform well under task-specific utility metrics while still failing to reproduce the global structure of the reference data. This distinction is explicit in both TabStruct papers and is central to the benchmark’s rationale (Jiang et al., 12 Mar 2025, Jiang et al., 15 Sep 2025).

A closely related motivation is evaluator bias. Downstream utility depends on the choice of target and predictor, and the framework argues that this can obscure structural failures. In the later benchmark, this concern is extended into a broader claim that conventional evaluation dimensions are complementary, not interchangeable, and that structural fidelity must be assessed alongside them rather than inferred from them (Jiang et al., 15 Sep 2025).

2. Structural fidelity and its formalization

In TabStruct, structural fidelity is defined as the alignment between the structural relations in the reference data $\mathcal{D}_{\text{ref}}$ and those in the synthetic data $\mathcal{D}_{\text{syn}}$. The original formulation evaluates this alignment at the level of the Markov equivalence class, represented by a CPDAG, rather than at the full DAG level. The stated reason is methodological: exact edge-direction recovery is difficult and highly dependent on the causal discovery method, whereas equivalence-class-level conditional independences are more robust (Jiang et al., 12 Mar 2025).

The original benchmark decomposes structural fidelity into two measures. Global independence evaluates all conditional-independence relations in the dataset. Local independence restricts attention to relations involving the target variable $y$. Each conditional-independence test is treated as a binary classification problem, where $1$ denotes independence and $0$ denotes dependence, and the score is computed using balanced accuracy. This makes structural fidelity a direct property of the agreement between the synthetic data and the graph-implied independence structure, rather than a side effect of predictive performance (Jiang et al., 12 Mar 2025).

The later paper retains the same structural concern but presents a more explicit conditional-independence formalization. When the true SCM is known, it defines a CI set $\mathcal{C}_{\text{global}}$ that includes both d-separation-based independencies and d-connections obtained by removing variables from separating sets:

$$\mathcal{C}_{\text{global}} \coloneqq \bigl\{(x_j \perp x_k \mid S_{j,k}) \mid S_{j,k}\subseteq\mathcal{X}\setminus\{x_j,x_k\}\bigr\} \cup \bigl\{(x_j \not\!\perp x_k \mid \widehat{S}_{j,k}) \mid \widehat{S}_{j,k}\subsetneq S_{j,k}\bigr\}.$$

The CI score for a dataset $\mathcal{D}$ is then defined as

$\text{CI}(\mathcal{C}, \mathcal{D}) = \frac{1}{|\mathcal{C}|} \sum_{\mathcal{C}} \mathbbm{1}\Bigl[ \widehat{\mathcal{I}_{\alpha}(x_j, x_k \mid S_{j,k}, \widehat{S}_{j,k};\mathcal{D}) = 1 \Bigr],$

with significance level $\mathcal{D}_{\text{ref}}$0. The same local/global distinction is preserved: local CI focuses on relations involving the prediction target, while global CI measures fidelity to the full causal organization (Jiang et al., 15 Sep 2025).

3. Benchmark construction and evaluation scope

The original TabStruct benchmark is built around seven benchmark datasets drawn from bnlearn, each paired with an expert-validated structural causal model. These datasets span classification and regression settings and a range of structure sizes, including small, medium, and large graphs. The benchmark includes Sangiovese, Insurance, Hailfinder, ANDES, Healthcare, MEHRA, and ARTH150, and the reference datasets are created by prior sampling from SCMs (Jiang et al., 12 Mar 2025).

That original benchmark evaluates nine concrete methods across eight generator categories, plus the reference dataset itself. The categories include interpolation, Bayesian networks, VAE-based models, GAN-based models, normalizing flows, diffusion models, tree-based models, and LLM-based generators. The explicit method list is SMOTE, BN, TVAE, GOGGLE, CTGAN, Neural Spline Flows, TabDDPM, ARF, and GReaT (Jiang et al., 12 Mar 2025).

The later TabStruct benchmark expands substantially. It evaluates 13 tabular generators from nine distinct categories across 29 datasets. The dataset inventory is split between SCM datasets, where causal structure is available, and real-world datasets, where it is not. The real-world portion includes 14 classification datasets and 9 regression datasets, while the SCM portion includes six benchmark datasets in the main experiments and five additional large SCM datasets in the appendix. The evaluation pipeline uses an 80\% train / 20\% test split, then divides train into 90\% reference and 10\% validation, repeating this 10 times (Jiang et al., 15 Sep 2025).

The following table summarizes the scope of the two TabStruct benchmark formulations.

Aspect	Original TabStruct	Extended TabStruct
Core papers	(Jiang et al., 12 Mar 2025)	(Jiang et al., 15 Sep 2025)
Datasets	7 bnlearn SCM datasets	29 datasets total
Structural supervision	Expert-validated SCMs	SCM and real-world settings
Generator coverage	9 methods, 8 categories	13 generators, 9 categories
Structural metrics	Local/global independence	Local/global CI and global utility

This progression suggests a shift from a benchmark centered on expert-validated causal graphs toward a broader evaluation suite that can also operate when such graphs are unavailable.

4. Metrics beyond structure

TabStruct does not replace conventional metrics; it places them alongside structural fidelity. The original benchmark reports Shape, Trend, $\mathcal{D}_{\text{ref}}$1-precision, $\mathcal{D}_{\text{ref}}$2-recall, Accuracy, RMSE, DCR, and Authenticity, with downstream utility computed using the standard train-on-synthetic, test-on-real protocol and averaged across six downstream predictors: Logistic Regression, KNN, MLP, Random Forest, XGBoost, and TabPFN (Jiang et al., 12 Mar 2025).

The later benchmark uses a closely related but broader evaluation layout. It groups metrics into density estimation, privacy preservation, ML efficacy, and structural fidelity. Its density-estimation metrics are Shape, Trend, $\mathcal{D}_{\text{ref}}$3-precision, and $\mathcal{D}_{\text{ref}}$4-recall; privacy metrics are DCR and $\mathcal{D}_{\text{ref}}$5-Presence; ML efficacy is represented by local utility; and structural fidelity includes local CI, global CI, and global utility. The downstream predictor ensemble for utility is expanded to nine models: LR, KNN, MLP, RF, Extra Trees, LightGBM, CatBoost, XGBoost, and TabPFN (Jiang et al., 15 Sep 2025).

A central empirical claim of the benchmark line is that these dimensions are not interchangeable. In the original study, the reported rank correlations show that downstream utility aligns much more strongly with local structure than with global structure. The reported values are 0.90 for accuracy versus local independence and 0.57 for accuracy versus global independence, while RMSE versus local independence is 0.77 and RMSE versus global independence is 0.33. The stated interpretation is that good predictive performance can coexist with weak preservation of the overall data graph (Jiang et al., 12 Mar 2025).

This is reinforced in the later work, which states that density estimation and privacy preservation do not strongly correlate with global CI, and that ML efficacy is often biased toward local target-specific structure. The benchmark therefore treats structure as an independent dimension that must be measured directly (Jiang et al., 15 Sep 2025).

5. Global utility and the SCM-free extension

The principal methodological addition in the later TabStruct paper is global utility, introduced to assess global structural fidelity without access to ground-truth causal structures. The motivation is that CI-based structural evaluation requires known SCMs, which are rarely available for real-world tabular datasets (Jiang et al., 15 Sep 2025).

For each variable $\mathcal{D}_{\text{ref}}$6, the framework treats that variable as a prediction target and measures the relative predictive performance of a dataset $\mathcal{D}_{\text{ref}}$7 against the reference dataset $\mathcal{D}_{\text{ref}}$8. The per-variable utility is defined as

$\mathcal{D}_{\text{ref}}$9

The paper uses balanced accuracy for categorical targets and RMSE for numerical targets. These per-variable scores are then averaged:

$\mathcal{D}_{\text{syn}}$0

The intended interpretation is that if the global dependency structure is preserved, every variable should remain predictable from the others in a manner comparable to the reference data. The paper connects this intuition to the Markov blanket perspective and positions global utility as a task-independent and domain-agnostic lens on structure preservation (Jiang et al., 15 Sep 2025).

The reported quantitative justification is strong. The paper gives a Spearman correlation between global utility and global CI of 0.84, with $\mathcal{D}_{\text{syn}}$1. It also reports that normalization is important: global utility with normalized performance has Spearman correlation 0.84 with global CI, whereas global utility with absolute performance drops to 0.57. The paper further reports a runtime example of 0.64s per 1000 samples under a small predictor setup, compared with about 1.21s per 1000 samples for local utility at comparable stability (Jiang et al., 15 Sep 2025).

A plausible implication is that the later TabStruct formulation is not merely an enlarged benchmark, but a methodological shift from “evaluate structure when graphs are known” to “approximate structure faithfully when graphs are unknown.”

6. Empirical findings, model rankings, and disambiguation

Across both TabStruct papers, the dominant empirical result is that existing generators still struggle to preserve structure. In the original benchmark, the best local-independence score among generators in classification is reported as 74.02\% for SMOTE, whereas the reference dataset is 100\%. For global independence, the gap from reference is described as around 35 percentage points. The paper uses these gaps to argue that structure is substantially harder to preserve than conventional task-level or distributional behavior (Jiang et al., 12 Mar 2025).

The two studies also identify recurring model-family tradeoffs. The original paper reports that SMOTE performs strongly on downstream utility and local independence but poorly on global independence; BN performs relatively well on structural fidelity but degrades on larger datasets; TVAE is competitive on global independence; TabDDPM performs well on privacy but not on structure; and GOGGLE, despite its relational-structure design, performs poorly in structural fidelity in those experiments (Jiang et al., 12 Mar 2025).

The later paper sharpens these conclusions at larger scale. It reports that diffusion-based models—specifically TabDDPM, TabSyn, and TabDiff—are consistently the Top-3 performers in global CI and global utility, while SMOTE remains strong for local utility and local CI. It also states that BN and GOGGLE do not dominate despite their structural inductive bias, and that GReaT performs weakly on structural fidelity, which the paper attributes to feature-order sensitivity and the mismatch between autoregressive factorization and permutation-invariant tabular structure (Jiang et al., 15 Sep 2025).

Because the term “TabStruct” can be confused with table-structure-recognition work in document AI, a disambiguation is useful. In the standalone title usage covered here, TabStruct denotes a benchmark and evaluation framework for synthetic tabular data and structural fidelity (Jiang et al., 12 Mar 2025, Jiang et al., 15 Sep 2025). By contrast, TabStruct-Net is a separate image-based table structure recognition system that combines top-down cell detection with bottom-up graph reasoning to recover table layouts from document images (Raja et al., 2020). The shared string “TabStruct” therefore spans two different research contexts: one in tabular data generation evaluation, the other in document table parsing.

This suggests that the most stable encyclopedia-level definition of TabStruct is the benchmark line concerned with measuring whether synthetic tabular data preserves causal and conditional-independence structure, while acknowledging that the same lexical root also appears in table-structure-recognition nomenclature.