Relational Database Distillation (RDD)

• Relational Database Distillation (RDD) is a framework that compresses multi-table relational data into a condensed heterogeneous graph while preserving key predictive information.
• It transforms structured tables into a Relational Entity Graph using multi-modal tokenizers and primary-foreign key relationships, enabling efficient graph synthesis.
• Experimental results show that RDD achieves over 500× compression and robust generalization across various graph-based models, validated on real-world databases.

Relational Database Distillation (RDD) denotes a data-centric framework for compressing a large relational database into a much smaller heterogeneous graph while preserving the predictive information needed by graph-based models. In the formulation introduced in "Relational Database Distillation: From Structured Tables to Condensed Graph Data" (Gao et al., 8 Oct 2025), the original multi-table database is converted into a Relational Entity Graph (REG), multi-modal column information is preserved through node features, primary-foreign key relations are encoded via heterogeneous edges, and a kernel ridge regression-guided objective with pseudo-labels is used to learn synthetic features and labels without the traditional, inefficient bi-level distillation framework.

1. Problem setting and formal definition

The underlying database is written as

$\mathcal{D} = (\mathcal{T}, \mathcal{L}),$

where $\mathcal{T} = \{T_1, \ldots, T_n\}$ is the set of tables, and $$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$ is the set of table links induced by foreign keys. Each table $T \in \mathcal{T}$ is a set of entities or rows,

$T = \{v_1, \ldots, v_{n_T}\},$

and each entity is a triplet

$v = (p_v, \mathcal{K}_v, x_v),$

with primary key $p_v$, foreign keys $\mathcal{K}_v$, and multi-modal attributes $x_v$.

The attributes are partitioned into numerical and categorical components,

$x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$

One table $\mathcal{T} = \{T_1, \ldots, T_n\}$0 is designated as the target table with labels $\mathcal{T} = \{T_1, \ldots, T_n\}$1, while the remaining tables provide auxiliary relational context. The central RDD problem is: given $\mathcal{T} = \{T_1, \ldots, T_n\}$2 and task labels $\mathcal{T} = \{T_1, \ldots, T_n\}$3, construct a compact synthetic database $\mathcal{T} = \{T_1, \ldots, T_n\}$4 with far fewer entities but comparable predictive utility under graph-based models (Gao et al., 8 Oct 2025).

This formulation arises from the cost profile of relational deep learning. Recent graph-based approaches can capture complex inter-table relations as multi-hop dependencies, but they remain hindered by prohibitive storage overhead and excessive training time, due to the massive scale of the database and the computational burden of intensive message passing across interconnected tables. RDD addresses those concerns by synthesizing a much smaller database representation rather than merely accelerating training on the original database.

2. Relational Entity Graph representation

RDD is built on a relational-to-graph transformation. From the table links $\mathcal{T} = \{T_1, \ldots, T_n\}$5, the framework defines inverse links

$\mathcal{T} = \{T_1, \ldots, T_n\}$6

and forms a schema graph $\mathcal{T} = \{T_1, \ldots, T_n\}$7 with edge set $\mathcal{T} = \{T_1, \ldots, T_n\}$8. The resulting Relational Entity Graph is

$\mathcal{T} = \{T_1, \ldots, T_n\}$9

where $$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$0 is the set of nodes, $$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$1 is the set of edges, $$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$2 maps each node to its table type, and $$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$3 maps each edge to its relation type.

Multi-modal column information is encoded through lightweight tokenizers. For numerical attributes, the framework uses per-column linear tokenizers,

$$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$4

with $$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$5. For categorical attributes, it uses per-column embedding tables,

$$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$6

with $$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$7. Column embeddings are aggregated into an entity embedding,

$$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$8

which serves as the initial node feature for the REG.

A heterogeneous GNN $$\mathcal{L} \subseteq \mathcal{T} \times \mathcal{T}$$9 then processes $T \in \mathcal{T}$0 through heterogeneous message passing to produce higher-level embeddings $T \in \mathcal{T}$1. The tokenizers $T \in \mathcal{T}$2 are explicitly part of the distilled database parameterization,

$T \in \mathcal{T}$3

This makes the tokenization stage itself part of the compressed representation rather than an external preprocessing artifact.

3. T2G: clustering, SBM structure synthesis, and KRR feature distillation

The paper instantiates RDD through Table-to-Graph (T2G), a three-stage pipeline combining tokenization, clustering, stochastic block modeling, and kernel ridge regression. The first stage performs clustering-based pretraining. After computing node embeddings

$T \in \mathcal{T}$4

the method applies online clustering on $T \in \mathcal{T}$5 for each table $T \in \mathcal{T}$6 to obtain pseudo-labels

$T \in \mathcal{T}$7

The clustering objective is

$T \in \mathcal{T}$8

For classification tasks, clustering is performed within each class; for regression tasks, clustering is done in label space. Each cluster corresponds to one synthetic entity, so the number of clusters $T \in \mathcal{T}$9 determines the synthetic table size. The global compression ratio is

$T = \{v_1, \ldots, v_{n_T}\},$0

The second stage synthesizes graph structure with a heterogeneous stochastic block model. For each relation type $T = \{v_1, \ldots, v_{n_T}\},$1, the method estimates a cluster-to-cluster connectivity matrix $T = \{v_1, \ldots, v_{n_T}\},$2. A global sparsity ratio $T = \{v_1, \ldots, v_{n_T}\},$3 induces a type-specific threshold

$T = \{v_1, \ldots, v_{n_T}\},$4

and synthetic adjacency is generated by

$T = \{v_1, \ldots, v_{n_T}\},$5

This produces the edge set $T = \{v_1, \ldots, v_{n_T}\},$6 of the synthetic REG $T = \{v_1, \ldots, v_{n_T}\},$7. The construction is not simple subset selection: nodes correspond to clusters, and edges are synthesized from estimated inter-cluster connectivity.

The third stage distills features and labels through a kernel ridge regression objective. Let $T = \{v_1, \ldots, v_{n_T}\},$8 be the embeddings of synthetic nodes. On the target table, the synthetic KRR predictor is

$T = \{v_1, \ldots, v_{n_T}\},$9

The task loss is

$v = (p_v, \mathcal{K}_v, x_v),$0

and a pseudo-label KRR term is added through

$v = (p_v, \mathcal{K}_v, x_v),$1

The optimization target is

$v = (p_v, \mathcal{K}_v, x_v),$2

Because the predictor has a closed-form KRR solution, the method avoids the inner-loop neural training characteristic of standard bi-level dataset distillation.

4. Empirical results, compression behavior, and generalization

The empirical study uses three real-world relational databases from SyntheRela: Rossmann, Walmart, and Airbnb. Rossmann has 1,015,159 rows, 16 columns, and 2 tables; Walmart has 310,707 rows, 17 columns, and 3 tables; Airbnb has 1,020,723 rows, 20 columns, and 2 tables. The evaluated baselines are Random selection, Herding, K-Center, Graph coarsening, FreeHGC, HGCond, and a Whole Database baseline trained on the original REG (Gao et al., 8 Oct 2025).

At very small compression ratios, T2G consistently outperforms the competing condensation baselines. On Rossmann, the Whole Database baseline reaches MAE $v = (p_v, \mathcal{K}_v, x_v),$3. At $v = (p_v, \mathcal{K}_v, x_v),$4, the best baseline HGCond gives $v = (p_v, \mathcal{K}_v, x_v),$5, whereas T2G gives $v = (p_v, \mathcal{K}_v, x_v),$6; at $v = (p_v, \mathcal{K}_v, x_v),$7, HGCond gives $v = (p_v, \mathcal{K}_v, x_v),$8 and T2G gives $v = (p_v, \mathcal{K}_v, x_v),$9. On Walmart, the Whole Database baseline is $p_v$0 MAE. At $p_v$1, HGCond gives $p_v$2, while T2G gives $p_v$3; at $p_v$4, T2G gives $p_v$5, essentially equal to the full database. On Airbnb, the Whole Database baseline reaches $p_v$6 AUC. At $p_v$7, the best baseline HGCond gives $p_v$8, whereas T2G gives $p_v$9; at $\mathcal{K}_v$0, HGCond gives $\mathcal{K}_v$1 and T2G gives $\mathcal{K}_v$2 (Gao et al., 8 Oct 2025).

The distilled data also generalize across downstream architectures. With $\mathcal{K}_v$3, the synthetic graphs are used to train SAGE, GIN, EDGE, HGT, HAN, and MLP. On Rossmann, T2G achieves the best average MAE, $\mathcal{K}_v$4, versus HGCond’s $\mathcal{K}_v$5; on Airbnb, T2G achieves the highest average AUC, $\mathcal{K}_v$6, versus HGCond’s $\mathcal{K}_v$7. This demonstrates that the distilled graph is not overfitted to a single HGNN architecture.

Storage reduction is a central empirical result. Rossmann compresses from 1,015,159 rows and 35.0 MB to 147 synthetic rows at $\mathcal{K}_v$8, with 59.1 KB synthetic data and 5 KB tokenizers, for 61.1 KB total, which is reported as $\mathcal{K}_v$9 compression. Walmart compresses from 310,707 rows and 13.5 MB to 192 synthetic rows at $x_v$0, with about 89.5 KB total. Airbnb compresses from 1,020,723 rows and 350 MB to 119 synthetic rows at $x_v$1, with about 69.8 KB total. The experiments therefore operate in the regime of training on a $x_v$2-$x_v$3 sized dataset.

Ablation results attribute much of this behavior to structure modeling and pseudo-label guidance. At $x_v$4 on Rossmann, full T2G obtains MAE $x_v$5; removing SBM gives $x_v$6; removing $x_v$7 gives $x_v$8; removing pseudo-labels entirely gives $x_v$9. Similar trends are reported on Walmart and Airbnb, indicating that both heterogeneous SBM and pseudo-label supervision are materially important.

5. Related meanings of “relational distillation”

The literature uses closely related terminology for several distinct operations.

Paradigm	Distilled object	Relational object
"Relational Database Distillation: From Structured Tables to Condensed Graph Data" (Gao et al., 8 Oct 2025)	compact heterogeneous graph from a relational database	primary-foreign key structure, node features, pseudo-label-guided graph utility
"Relational Knowledge Distillation" (Park et al., 2019)	student model	pairwise distances and triplet angles in teacher embedding space
"Relational Representation Distillation" (Giakoumoglou et al., 2024)	student representation	neighbor distributions over a memory buffer
"Relative Difficulty Distillation for Semantic Segmentation" (Liang et al., 2024)	segmentation student	pixel-level relative difficulty maps
"Quantum Relational Knowledge Distillation" (Liu et al., 18 Aug 2025)	classical student guided by quantum kernels	pairwise quantum similarities in Hilbert space
"Relational Deep Dive: Error-Aware Queries Over Unstructured Data" (Chao et al., 4 Nov 2025)	query-specific relational schema and populated database	schema sufficiency, table assignment, and error-aware extracted cells

The 2019 RKD paper does not mention “Relational Database Distillation” by name, but it explicitly states that, if “Relational Database Distillation” is understood as distilling the relational structure encoded in a dataset, then RKD is exactly this kind of idea, although its notion of relation is geometric rather than schema-based. In RKD, the student is trained to reproduce pairwise distances or triplet angles among examples in the teacher’s representation space, rather than to copy teacher outputs point-wise.

Relational Representation Distillation shifts the relational object from geometry to similarity-based neighborhood distributions. For each anchor, the teacher and student define probability distributions over a memory buffer, and the student matches the teacher through a KL objective. This preserves a soft neighborhood structure rather than the hard positive-versus-negative partition typical of InfoNCE-style contrastive learning.

Relative Difficulty Distillation uses the acronym RDD for a different purpose. There, the distilled signal is a pixel-level map of relative learning difficulty between teacher and student in semantic segmentation, implemented as a reweighting of the task loss rather than as condensation of a relational database.

Two later developments widen the scope still further. Quantum Relational Knowledge Distillation extends RKD by mapping classical features into a Hilbert space and using quantum kernels as pairwise relational measures, while keeping teacher and student fully classical at inference. ReDD, by contrast, starts from an unstructured corpus and, for a given query, distills a minimal joinable schema plus populated relational tables, with SCAPE and SCAPE-HYB providing coverage guarantees for error detection. The common thread across these lines is the transfer or synthesis of relational structure, but the objects being distilled—student parameters, embedding geometry, pixel difficulty maps, query-specific schemas, or compact heterogeneous graphs—are not interchangeable.

6. Misconceptions, limitations, and research directions

RDD in the database sense is neither standard model compression nor ordinary graph sampling. Existing efficiency approaches such as subgraph sampling, neighborhood sampling, distributed GNN training systems, and graph coarsening do not reduce the dataset itself. By contrast, RDD synthesizes a smaller relational or graph dataset that preserves both multi-modal attributes and inter-table dependencies. It is also not the same as generic vision or NLP dataset distillation: those methods are mostly designed for homogeneous tensors and classification tasks, often rely on expensive bi-level optimization, and do not directly handle multi-table relational structure or heterogeneous graphs (Gao et al., 8 Oct 2025).

The method also has clear assumptions. The synthetic graph preserves the original schema graph $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$0, so it assumes a well-defined primary-foreign key structure and relational integrity. The quality of clustering and pseudo-labels affects both the SBM and the KRR-based feature distillation, and the hyperparameters $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$1 and $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$2 matter: the reported sensitivity curves show that moderate $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$3 improves performance, while very large $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$4 may overfit pseudo-labels, and moderate sparsity $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$5 gives the best trade-off between redundant or noisy edges and information loss. Distillation uses an HGNN feature extractor, so although the resulting synthetic data generalize across SAGE, GIN, EDGE, HGT, HAN, and MLP, the framework is conceptually centered on graph-based downstream models. Scalability is improved but not eliminated, because pretraining and clustering still operate on the full dataset; the paper therefore points to FAISS acceleration, neighbor sampling, and clustering-based GCN techniques such as Cluster-GCN.

The complexity analysis makes the computational structure explicit. Column tokenization costs $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$6, a $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$7-layer HGNN costs $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$8, online clustering costs $x_v = (x_v^{\text{num}}, x_v^{\text{cat}}).$9, SBM estimation costs $\mathcal{T} = \{T_1, \ldots, T_n\}$00, adjacency sampling costs $\mathcal{T} = \{T_1, \ldots, T_n\}$01, and the closed-form KRR step costs $\mathcal{T} = \{T_1, \ldots, T_n\}$02. Pretraining and feature distillation dominate the total cost, but the KRR cubic term applies to the small synthetic set because $\mathcal{T} = \{T_1, \ldots, T_n\}$03.

Future directions identified around this formulation include extending RDD to dynamic relational databases, supporting workloads beyond per-entity prediction, integrating richer domain knowledge such as business rules into SBM and feature distillation, and handling noisy or missing foreign-key relations through additional structure learning. A plausible implication is that RDD is best viewed not as a single compression heuristic, but as a task-aware synthesis framework for relational data: it compresses database size, preserves relational semantics, and produces a synthetic graph that remains usable across multiple heterogeneous learning architectures.