---
title: 'ExtGraph: Direct Graph Extraction'
url: https://www.emergentmind.com/topics/extgraph
type: topic
---

# ExtGraph: Direct Graph Extraction

ExtGraph is a graph extraction method for relational databases that is designed to produce **user-intended graphs** directly from SQL join workloads rather than from only foreign-key structure. Its central systems idea is to optimize graph extraction by **hybrid query processing of outer join and materialized view**, so that shared joins across multiple edge-definition queries are reused without introducing the **virtual vertices** and **virtual edges** characteristic of some earlier approaches [2509.18534].

## 1. Problem setting and the notion of a user-intended graph

ExtGraph addresses a recurrent mismatch between data storage and graph analytics. In many organizations, important data remains in relational DBMSs, while graph analytics is used to study recommendation, fraud detection, social analysis, bioinformatics, and AI or knowledge-graph workloads. In this setting, the graph is not stored explicitly; it must first be extracted from relational data. The paper distinguishes two broad approaches: translating graph queries into relational queries, or extracting a graph first and then executing graph analytics on the extracted graph. ExtGraph targets the second approach [2509.18534].

The method is organized around the idea of a **user-intended graph**. In this usage, graph edges are not restricted to simple foreign-key links. Instead, they are defined by relational join queries that express the relationships users actually want to analyze. The examples given include edges between customers who bought the same product (`Co-pur`), customers who saw the same promotion (`Same-pro`), a customer and an item purchased with a promotion (`Get-disc`), a store and products it sells (`Sell`), and a customer and products it bought (`Buy`). These relationships may require **chain joins**, **star** joins, or **cyclic** joins rather than a single schema-derived edge pattern [2509.18534].

This focus places ExtGraph between two prior families of systems. **Schema-based extraction methods** are efficient, but they cannot express many graph edges users actually care about. **Join workload-based methods** are expressive, but they often become slow because each edge definition may require expensive joins, including many-way joins and \(N\)-to-\(N\) joins. Prior systems such as **GraphGen** and **R2GSync** reduce some extraction cost by decomposing complex edge queries into smaller pieces, but the result is not the original intended graph: one conceptual edge becomes a path through virtual graph elements, and recovering the actual graph requires post-processing. ExtGraph is intended to preserve join-workload expressiveness while extracting the intended graph directly [2509.18534].

## 2. Formal graph model and relational abstraction

The method formalizes graph extraction from a relational database instance \(D=\{R_i\}\) by defining a graph model

$$
M=(M_v, M_e),
$$

where \(M_v\) is a set of vertex definitions and \(M_e\) is a set of edge definitions. A vertex definition is

$$
m_v=(l_v, R_v)\in M_v,
$$

where \(l_v\) is the vertex label and \(R_v\) is a table whose tuples define vertices of that label. An edge definition is

$$
m_e=(l_e, m_{src}, m_{dst}, Q)\in M_e,
$$

where \(l_e\) is the edge label, \(m_{src}\) and \(m_{dst}\) are the source and destination vertex definitions, and \(Q\) is a query representing the join relationship between two vertices. Each tuple in the result of \(Q\) defines one edge of label \(l_e\) [2509.18534].

On this basis, graph extraction produces a directed multigraph \(G=(V,E)\). ExtGraph revises the ordinary extraction workflow by inserting an optimization phase. The resulting process is: define a graph model \(M\), **optimize edge definitions using join sharing techniques**, extract a set of vertices and a set of edges from \(D\), and convert them into a graph. The method is therefore not a graph query language or a graph store in its own right; it is an extraction-time optimization framework [2509.18534].

A concrete example is the `RetailG` graph. It defines `Customer` vertices from table `C`, `Item` vertices from table `I`, a `GetDisc` edge from a cyclic query over `C`, `SS`, `P`, and `I`, and a `CoPur` edge from a chain-style query over `C1`, `SS1`, `I`, `SS2`, and `C2`. This example is used to emphasize that ExtGraph supports **arbitrary join-form edge definitions**, not only chain joins [2509.18534].

To optimize such workloads, each edge-definition query is converted into a **join graph**. In this representation, a vertex indicates a table in the query, an edge indicates a join between tables, \(f(e)\) denotes the join type, and \(g(e)\) denotes the join condition. ExtGraph then identifies a **shared subgraph** between two join graphs as a connected component of common joins with matching join type and join condition. This abstraction allows the optimizer to reason structurally about shared join fragments across edge definitions [2509.18534].

## 3. Join sharing by outer join and by materialized view

ExtGraph’s main technical contribution is the combination of two join-sharing techniques: **JS-OJ** and **JS-MV**. Both seek to exploit overlap among edge-definition queries, but they do so differently and under different cost conditions [2509.18534].

**JS-OJ** merges related edge-definition queries into a single query using **outer joins**. If two join graphs share a common subgraph \(S\), each query is decomposed into \(S\) plus its non-shared subgraphs. The merged query retains the shared part once and attaches the non-shared parts via outer joins. The paper’s rationale is semantic: if the query-specific parts were attached with inner joins, one side could filter tuples needed by the other side. Outer joins prevent this interference because they preserve tuples from the designated outer side. ExtGraph enumerates possible decompositions, constructs merged join graphs, estimates their costs, and chooses the cheapest merged plan [2509.18534].

The correctness claim for JS-OJ is explicit. **Theorem 1** states that the result of the original queries can be retrieved **without loss or error** by performing the selected merged plan \(\mathtt{G_M^*}\). The proof idea is that outer joins preserve tuples from the shared side, and extra tuples introduced by the outer-join structure contain `null`s; tuples with non-`null` values correspond to valid results from the original queries [2509.18534].

**JS-MV** is used when direct merging would create a harmful increase in join complexity, especially through large \(N\)-to-\(N\) interactions. In that case, ExtGraph computes the shared join once, stores it as a **materialized view**, and rewrites the original edge-definition queries to use that view. JS-MV avoids repeated execution of the shared join without forcing unrelated query-specific components into a single monolithic outer-join plan. Its drawback is the materialization overhead: computing the view costs time and storing it incurs I/O [2509.18534].

The paper positions this dual design against GraphGen and R2GSync. ExtGraph’s novelty is not only join reuse, but **direct extraction of user-intended graphs without virtual edges/vertices and without post-processing**. In this sense, its primary concern is semantic fidelity during extraction, not merely lower intermediate cost [2509.18534].

## 4. Cost model and hybrid optimization

The optimizer chooses among baseline execution, JS-OJ, and JS-MV by means of an explicit cost model. For \(N\) edge-definition queries \(Q_i\), the baseline cost is

$$
Cost(P_{base})=\sum_{i=1}^{N}Join(Q_i).
$$

The paper assumes only **left-deep binary join plans** and **hash joins**. Under that assumption,

$$
Join(Q)=\sum_{i=2}^{M}Build(T_i)+Probe(T_1).
$$

For a merged query \(Q_M\) under JS-OJ, the cost is

$$
Join(Q_M)=Join(SQ_S)+\sum_{i=1}^{|U|}Join(SQ_i)+Outer(O),
$$

with outer-join combination cost

$$
Outer(O)=\sum_{i=1}^{|U|}Build(SQ_i)+Probe(SQ_S).
$$

For JS-MV, if \(K\) materialized views are created, the cost is

$$
Cost(P_{MV})= \sum_{i=1}^{K}(Join(V_i)+A_D \cdot N_P(V_i)) +\sum_{i=1}^{N}Join(Q_i').
$$

Here \(Q_i'\) denotes the rewritten edge-definition query, \(A_D\) the cost of accessing a disk page, and \(N_P(V_i)\) the number of disk pages containing the view [2509.18534].

The optimizer itself is a **greedy heuristic optimizer**. It starts from the baseline plan, enumerates candidate transformations obtainable by one application of JS-OJ or JS-MV, estimates their costs, adopts the lowest-cost improvement, and repeats until no cheaper plan is found. The paper gives a worked example with three queries \(\{Q_1,Q_2,Q_3\}\) and an initial baseline cost of \(800\). Candidate transformations are evaluated at costs \(2940\), \(760\), \(700\), and \(770\); the optimizer first selects JS-MV at cost \(700\), then JS-OJ at cost \(650\), yielding a hybrid final plan. This example is used to show that different shared joins may be best optimized in different ways [2509.18534].

## 5. Experimental evaluation

The evaluation uses **TPC-DS** with scale factors **SF=10, 30, 100**, plus **DBLP** and **IMDB**. In TPC-DS, two scenarios are defined: a **recommendation scenario** and a **fraud detection scenario**. Since TPC-DS includes three sales channels—**Store, Catalog, Web**—the experiments use six graph models in total. The real-data models include an `Auth-Edit` relationship in DBLP, and `Wri-Dir` and `Act-Dir` relationships in IMDB. Experiments run on one server with two 16-core 3.0 GHz CPUs, 1 TB memory, 14 TB hard disk, Ubuntu 18.04.4, and PostgreSQL 14.4 as the base system [2509.18534].

ExtGraph is compared with **Ringo**, **GraphGen**, and **R2GSync**. For fairness, the measured time for GraphGen and R2GSync includes the conversion needed to recover the same user-intended graph produced directly by Ringo and ExtGraph. This matters because GraphGen and R2GSync natively output structures containing virtual graph elements [2509.18534].

The main reported result is that ExtGraph outperforms the state-of-the-art methods **up to by 2.78x in terms of graph extraction time**. Representative results are as follows:

| Setting | Reported result | Comparison |
|---|---:|---|
| TPC-DS recommendation | **2.34X** | ExtGraph over Ringo at SF 10 |
| TPC-DS fraud detection | **2.78X** | ExtGraph over R2GSync at SF 30 |
| DBLP | **669.9 sec** | Ringo 769.2 sec; GraphGen 1866.6 sec; R2GSync 1882.1 sec |
| IMDB | **71.2 sec** | Ringo 88.4 sec; GraphGen 116.4 sec; R2GSync 388.8 sec |

The ablation study isolates the two join-sharing techniques on a TPC-DS graph model containing `Sell`, `Buy`, `Co-pur`, and `Same-pro`. Four configurations are compared: no join sharing, JS-OJ only, JS-MV only, and the hybrid plan. The **hybrid plan performs best**. The explanation is specific: **JS-MV alone** is better than **JS-OJ alone** for `Co-pur` and `Same-pro`, where direct query merging is less favorable, but the hybrid plan outperforms JS-MV alone because `Sell` and `Buy` can use **JS-OJ** and thereby avoid materializing those query results [2509.18534].

## 6. Scope, limitations, and relation to adjacent graph systems

ExtGraph is not presented as universally optimal under all workloads. Its own analysis identifies several trade-offs. **JS-OJ can backfire** when merging queries creates expensive additional \(N\)-to-\(N\) joins. **JS-MV incurs materialization I/O cost**, so it is not always advantageous to materialize a shared subquery. The hybrid planner is a **greedy heuristic optimizer**, not an exhaustive global search. The cost model is simplified by its assumptions of left-deep plans, hash joins, and disk-backed materialized views. The paper also characterizes ExtGraph’s extraction cost as **medium**: it is more efficient than independent query execution, but unlike GraphGen and R2GSync it does not reduce extraction cost by changing the graph semantics [2509.18534].

Within the broader graph-systems landscape, ExtGraph occupies a distinct position. **GraphPaper** is an environment for graph creation, visualization, and transformation specialized to interaction nets and coupled to TULIP [1003.4326]. **ELK** is a collection of graph drawing algorithms that supports compound graph layout and ports as explicit anchor points of edges [2311.00533]. **EGML** is an XML application for representing evolving graphs as timestamped `graph-instance` sequences [1006.4608]. **GSS** is a graph stream sketch with linear space \(O(|E|)\), practical \(O(1)\) update, and topology-aware approximate queries [1809.01246]. ExtGraph differs from these systems in that its primary problem is neither layout, nor interactive rewriting, nor temporal representation, nor graph-stream summarization. It is concerned with the **pre-analytics construction of graphs from relational workloads**.

This suggests that ExtGraph belongs to a specific infrastructural layer of graph processing: the layer that turns relational data into a graph faithful to the relationships users actually intend to analyze. Its contribution is therefore best understood not as a new graph model, but as a cost-based extraction architecture that treats join sharing, semantic fidelity, and direct graph materialization as a unified systems problem [2509.18534].

Source: https://www.emergentmind.com/topics/extgraph