Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures (2506.16654v1)

Abstract: Graph machine learning has led to a significant increase in the capabilities of models that learn on arbitrary graph-structured data and has been applied to molecules, social networks, recommendation systems, and transportation, among other domains. Data in multi-tabular relational databases can also be constructed as 'relational entity graphs' for Relational Deep Learning (RDL) - a new blueprint that enables end-to-end representation learning without traditional feature engineering. Compared to arbitrary graph-structured data, relational entity graphs have key properties: (i) their structure is defined by primary-foreign key relationships between entities in different tables, (ii) the structural connectivity is a function of the relational schema defining a database, and (iii) the graph connectivity is temporal and heterogeneous in nature. In this paper, we provide a comprehensive review of RDL by first introducing the representation of relational databases as relational entity graphs, and then reviewing public benchmark datasets that have been used to develop and evaluate recent GNN-based RDL models. We discuss key challenges including large-scale multi-table integration and the complexities of modeling temporal dynamics and heterogeneous data, while also surveying foundational neural network methods and recent architectural advances specialized for relational entity graphs. Finally, we explore opportunities to unify these distinct modeling challenges, highlighting how RDL converges multiple sub-fields in graph machine learning towards the design of foundation models that can transform the processing of relational data.

• The paper formalizes relational databases as heterogeneous temporal graphs to enable end-to-end neural learning on large-scale data.
• It systematically benchmarks diverse real-world datasets and analyzes methodologies including GNNs, temporal models, and transformer adaptations.
• The study introduces next-generation architectures and highlights open challenges for developing unified and scalable relational deep learning systems.

Relational Deep Learning: A Formal Overview of Challenges, Foundations, and Next-Generation Architectures

The surveyed paper provides a comprehensive analysis of Relational Deep Learning (RDL), a paradigm at the intersection of graph machine learning and relational databases. The authors systematically review the conceptual mapping between multi-table relational databases and heterogeneous temporal graphs, as well as the methodological advancements making direct end-to-end neural learning on large-scale relational data feasible.

Formalization: From Relational Databases to Entity Graphs

Relational databases, ubiquitous in industry, consist of entities (table rows) linked via primary-foreign key relations. The paper formalizes this structure as heterogeneous temporal graphs—nodes are database entries annotated by their types (origin tables), and edges represent inter-table reference links encapsulating both type heterogeneity and temporal information (e.g., transaction times). This formalization enables the direct use of graph neural network (GNN) techniques on relational data, bypassing traditional two-stage pipelines of feature engineering and downstream tree-based/tabular models.

Benchmarking and Datasets

The authors review public benchmarks essential to RDL research, notably RelBench, which encapsulates diverse real-world relational databases across e-commerce, social, sports, and medical domains. These benchmarks define a broad spectrum of predictive tasks (classification, regression, recommendation), incorporate temporal splitting protocols to prevent time leakage, and support evaluation at scale (millions of entities). Additional benchmarks from the CTU Relational Dataset Repository extend this diversity, though they often lack standardized splits and multi-task settings.

Key Methodological Building Blocks

The methodological review uncovers the layered structure of RDL, summarizing developments along several axes:

• Tabular Models: Legacy methods such as XGBoost and LightGBM provide efficient baselines but do not exploit relational connectivity. Transformer-based advances (TabTransformer, TabPFN, XTab) offer single-table advances but lack generalization to multi-table environments unless explicitly extended.
• Graph Neural Networks: Standard message-passing GNNs (GCNs, GraphSAGE, GIN) are foundational, supporting end-to-end learning on graph-structured relational data. Importantly, handling attribute heterogeneity and scalability is critical due to database scale and diversity.
• Temporal Graph Learning: Temporal GNNs (TGAT, TGN, CAWN, ROLAND) extend static GNNs to accommodate evolving graphs, modeling event sequences with precise timestamps via modules like temporal attention or memory, applicable directly to temporal relational data.
• Graph Transformers: Both local (GAT, Graph Transformer) and global attention architectures are discussed, with the latter encumbered by quadratic computational costs on large graphs. Hence, hybrid architectures and scalable positional encodings are central to practical deployment in RDL.
• Heterogeneous Graph Architectures: To encode multi-type nodes and edges, extensions like RGCN, HAN, and heterogeneous graph transformers are reviewed.

Next-Generation RDL Architectures

The exposition highlights advanced architecture tailored for RDL:

• Heterogeneous GraphSAGE-based Systems: Early RDL implementations, combined with temporal-aware sampling, outperform or match traditional tabular techniques while reducing model development time by over 95%.
• RelGNN: Introduces composite message passing along "atomic routes" to address the inefficiency of standard aggregation over hub and bridge nodes, better aligning model inductive bias with relational schemas.
• ContextGNN: Hybridizes two-tower and pairwise modeling to achieve scalability and contextual fidelity in recommendations.
• TREeLGNN: Merges single-table pretrained models with graph-based static models for efficient, high-throughput learning in relational contexts.
• Transformers for Relational Data: DBFormer and Relational Graph Transformer (RelGT) adapt transformer architectures to multi-table settings via inter-table and cross-attention layers, robust positional encodings, and flexible tokenization schemes incorporating node type, topology, and temporal attributes.

Integration of LLMs

The potential for LLMs to facilitate direct, in-context learning on relational prediction tasks is noted. Recent baselines demonstrate LLMs can process serialized database snapshots and task descriptions, particularly when augmented with metric-aware inference or shallow neural heads for regression.

Temporal and Structural Encoding

Time encoding is treated with rigor—both learnable (Time2Vec) and fixed (GraphMixer) encodings are described, with an emphasis on their theoretical foundations (e.g., Bochner's theorem) and empirical suitability for capturing temporal patterns in event-driven relational graphs. The discussion also addresses the challenge of variable time granularities and motivates granular, unified frameworks such as UTG for addressing diverse forecasting needs.

Open Challenges and Research Directions

The authors articulate several unsolved issues in the field:

• Unified GNNs: There is a strong call for the development of pipeline architectures unifying column encoding, relational message passing, temporal subgraph sampling, and multi-modal integration—essential for robust, transferable, and scalable RDL systems across diverse enterprise-grade databases.
• Foundation Models for RDL: Establishing universal, large-scale pretrained models (akin to LLMs for text) that encode rich relational structures for zero-shot or few-shot adaptation remains an open challenge. Progress is exemplified by emerging efforts (e.g., Kumo Relational Foundation Model) but demands scalable ingest, universal schema embeddings, and effective self-supervision with minimal task-specific tuning.

Implications and Impact

This work provides a blueprint for the future of learning on structured enterprise data. The fusion of graph and temporal deep learning, combined with advances in transformers and LLMs, positions RDL as a pathway toward foundation models capable of unifying information from disparate data verticals natively. Practically, this promises to dramatically reduce the cost and expertise required to build predictive models on relational databases, leading to higher productivity and broader AI adoption in sectors built around large-scale tabular data.

At the theoretical level, RDL is driving progress in heterogeneity-aware message passing, expressivity of temporal and structural graph modules, and scalable training on graphs with rich attribute and relational diversity. The pronounced focus on realistic benchmarks, rigorous time-aware evaluation, and real-world obstacles grounds this research program in urgent industrial needs while spurring cross-pollination among the fields of graph learning, tabular deep learning, temporal data modeling, and foundation model pretraining. Improvements along these axes will further inform model design for multi-modal, dynamic, and large-graph settings, strengthening the broader AI research ecosystem.