---
title: Simple Relation Decoupling (SRD)
url: https://www.emergentmind.com/topics/simple-relation-decoupling-srd
type: topic
---

# Simple Relation Decoupling (SRD)

Searching arXiv for the specified paper to ground the article and verify bibliographic details.
Simple Relation Decoupling (SRD) is a relation-extraction approach that separates feature design from model architecture in large-scale extraction from unstructured text. In the formulation described in "Simple Large-scale Relation Extraction from Unstructured Text" [1803.09091], the task is to learn a function $f : E \times E \times S^* \to R \cup \{\bot\}$ that takes a candidate entity pair $(x,y) \in E \times E$ together with all sentences in which both entities co-occur and predicts either a relation $r \in R$ or “no relation” $(\bot)$. The central claim of SRD is that a much simpler classifier trained on similar features performs on par with a highly complex neural network system, at a $75\times$ reduction to the training time, which suggests that the features are a bigger contributor to the final performance [1803.09091].

## 1. Formalization of the extraction task

SRD assumes a knowledge base with entity set $E = \{e_1, e_2, \dots\}$, relation set $R = \{r_1, r_2, \dots\}$, and a corpus of tokenized, dependency-parsed sentences $S = \{s_1, s_2, \dots\}$. For a pair $(x,y)$, the support set is defined as

$$
S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.
$$

Given a large KB of known triples $\mathrm{KB} \subseteq E \times R \times E$, SRD adopts the distant-supervision assumption: if $(x,r,y) \in \mathrm{KB}$ and $x,y$ both appear in sentence $s$, then $s$ is noisily a positive example of $r$ [1803.09091].

The resulting noisy training set is

$$
D = \{ ((x,y), r, S(x,y)) : (x,r,y) \in \mathrm{KB} \wedge S(x,y) \neq \emptyset \}.
$$

Negative examples are sampled from pairs $(x,y) \notin \mathrm{KB}$, ensuring ontological constraints, at a fixed ratio such as $4{:}1$. In this formulation, relation extraction is not defined sentence-by-sentence in isolation; rather, the prediction consumes all supports in which the entity pair co-occurs. This suggests that SRD treats aggregation across mentions as a first-class design choice rather than as a secondary postprocessing step.

## 2. Distant supervision and page-specific gazetteers

A defining element of SRD is the use of page-specific gazetteers to improve entity-resolution precision during distant supervision [1803.09091]. For each Wikipedia page $P$ whose main entity is $x$, SRD gathers all $1$-hop neighbors

$$
N_1(x) = \{ y : \exists r \, ((x,r,y) \in \mathrm{KB}) \}
$$

and collects their name-variants in a page-specific lexicon. Entity mentions are then identified by greedy longest-match string lookup on each sentence $s \in S(P)$ to produce $\mathrm{ents}(s)$.

The pipeline can be summarized in four stages. First, page-specific gazetteers are built from KB-linked page entities and their $1$-hop neighbors. Second, distant supervision and entity resolution produce labeled examples by matching entities in sentences and assigning either a KB relation or $\bot$. Third, shortest dependency paths are converted into sparse symbolic features. Fourth, a fastText classifier is trained with cross-entropy and $\ell_2$ regularization [1803.09091].

The stated effect of page-specific gazetteers is twofold. They reduce the number of candidate entities per sentence from $O(|\mathrm{KB}|)$ to $O(1\text{-hop degree of page entity})$, making distant-supervision label generation approximately $10\times$ faster and more precise. The trade-off is that the system depends on high-precision entity linkings and a careful distant-supervision pipeline, including page-specific gazetteers and Bloom filters for KB lookup [1803.09091].

## 3. Feature design and representation

For each candidate $(x,y)$ and each supporting sentence $s \in S(x,y)$, SRD extracts the shortest dependency path $p = [t_0,\dots,t_L]$ connecting $x$ to $y$. Each token $t_i$ on the path is annotated with the following attributes [1803.09091]:

- lemma $\ell_i \in V$
- Brown-cluster prefix $b_i \in \{0,1\}^4$
- POS tag $p_i \in P$
- dependency relation $d_i \in \mathrm{DepRel}$
- direction $\delta_i \in \{\rightarrow,\leftarrow\}$

The representation also includes features for the entities themselves: $x_{\text{str}}$, $y_{\text{str}}$, and the $4$-bit Brown prefixes of their names, $b_x$ and $b_y$.

If $S(x,y) = \{s^{(1)},\dots,s^{(K)}\}$, the feature multiset is

$$
F(x,y) = \biguplus_{k=1..K} \phi(p^{(k)};x,y),
$$

where the atomic feature vector for one support is the set of symbolic strings

$$
\{ \ell_i \Vert "/" \Vert b_i \Vert "/" \Vert p_i \Vert "/" \Vert d_i \Vert "/" \Vert \delta_i : i=0\dots L \}
\cup
\{ x_{\text{str}} \Vert "/" \Vert b_x,\; y_{\text{str}} \Vert "/" \Vert b_y \}.
$$

In practice, $\phi$ is treated as a sparse indicator vector over all such strings. SRD then collapses the multiset $F(x,y)$ into a single sparse feature vector $v \in \{0,1\}^d$, explicitly characterized as a bag-of-features representation [1803.09091].

The feature inventory is syntacto-semantic: it preserves dependency-path structure through path extraction, but the final classifier consumes only sparse symbolic indicators. The paper’s interpretation is explicit: rich features such as dependency paths and Brown clusters drive most of the performance, not sophisticated recurrent architectures. A plausible implication is that SRD relocates representational burden from the classifier into the feature extractor.

## 4. Linear classification and decision rule

SRD trains a linear multiclass classifier, implemented with fastText in the reported experiments, over the sparse feature vector $v$ [1803.09091]. The score for relation $r$ is

$$
\mathrm{score}(r \mid v) = W_r^\top v + b_r,
$$

where $W \in \mathbb{R}^{|R| \times d}$ and $b \in \mathbb{R}^{|R|}$. Scores are converted to probabilities by softmax:

$$
P(r \mid v) = \frac{\exp(\mathrm{score}(r \mid v))}{\sum_{r' \in R \cup \{\bot\}} \exp(\mathrm{score}(r' \mid v))}.
$$

Learning minimizes the negative log-likelihood with $\ell_2$ regularization,

$$
L(W,b) = - \sum_{(v,y)\in D} \log P(y \mid v) + \lambda \lVert W \rVert^2,
$$

with $\lambda$ a small $\ell_2$ regularizer. At inference time, prediction is made by

$$
\hat{y} = \arg\max_{r \in R \cup \{\bot\}} \mathrm{score}(r \mid v).
$$

The pseudocode in the source description makes the decoupling explicit: the system first constructs sparse path-based features, then applies a standard linear classifier trained by SGD on cross-entropy plus $\ell_2$. This suggests that SRD is less a new classifier family than a design argument about where relation-extraction performance originates.

## 5. Relation to HypeNET and empirical results

SRD is presented through a direct comparison with HypeNET, which is structurally more complex [1803.09091]. HypeNET includes embedding layers for lemmas, POS, Brown clusters, dependency relations, and direction; a uni- or bi-directional LSTM over the token sequence of each path; average-pooling over the $K$ supports to obtain a path representation $h_{\text{path}} \in \mathbb{R}^D$; and concatenation of $h_{\text{path}}$ with learned embeddings of $x$ and $y$ before a softmax layer.

Its parameter count is described as on the order of a few million, approximately $1$–$3$M weights, and training on $50$K examples takes approximately $75$ min on a single GPU. By contrast, the SRD fastText model has only $O(d \cdot |R|) \approx O(10^7)$ parameters but trains in $\lesssim 1$ min on the same hardware [1803.09091].

| Relation | HypeNET $F_1$ | fastText $F_1$ |
|---|---:|---:|
| Wikidata “instance of” | $93.9 \pm 0.2$ | $96.4 \pm 0.0$ |
| Wikidata “birthplace of” | $92.1 \pm 0.9$ | $93.1 \pm 0.1$ |
| Wikidata “part of” | $48.7 \pm 2.6$ | $72.9 \pm 0.2$ |
| Alexa KB “instance of” | $94.3 \pm 0.2$ | $94.3 \pm 0.0$ |
| Alexa KB “birthplace of” | $85.6 \pm 0.3$ | $87.6 \pm 0.0$ |
| Alexa KB “applies to” | $82.0 \pm 1.8$ | $86.2 \pm 0.0$ |

The same comparison also reports MaxEnt baselines of $58.5$, $66.7$, and $45.1$ on Wikidata “instance of,” “birthplace of,” and “part of,” and $83.9$, $80.8$, and $65.3$ on Alexa KB “instance of,” “birthplace of,” and “applies to,” respectively [1803.09091]. The reported conclusion is that SRD’s fastText matches or beats HypeNET while cutting training time by approximately $75\times$.

A common misconception in discussions of this result is to treat it as a claim that neural architectures are ineffective for relation extraction. The actual claim is narrower: once the feature design of a state-of-the-art neural network system is decoupled from the model architecture, a simpler classifier trained on similar features performs on par with the more complex system [1803.09091].

## 6. Scalability, trade-offs, and extensibility

The computational profile of SRD is stated in simple asymptotic terms [1803.09091]. Feature extraction is linear in total tokens and dependency-path length, $O(\sum_s |s|)$. fastText training is $O(N \cdot d \cdot E)$, where $N$ is the number of training examples, $d$ is the average number of nonzeros per $v$, and $E$ is the number of epochs. Empirically, training is reported as $\lesssim 1$ min on $50$K examples. HypeNET’s per-example cost includes an LSTM over each support path and gradient computation through all embedding layers, yielding $O(N \cdot \mathrm{LSTM\_cost})$ and approximately $75\times$ slower training.

The paper’s stated trade-off is that SRD requires a well-engineered feature extractor, including dependency parses and Brown clustering, together with a careful distant-supervision pipeline. However, once that infrastructure is in place, it is described as trivially extended to hundreds of relations: simply generate more $(x,r,y)$ triples and re-train the same fastText model [1803.09091].

The broader significance of SRD lies in its decomposition of the relation-extraction stack. By assigning most of the burden to entity resolution, dependency-path extraction, and symbolic feature construction, it argues that model simplicity need not imply representational weakness. This suggests a methodological lesson for large-scale RE: when supervision is weak and throughput matters, improving the precision of label generation and the quality of path-based features may yield larger gains than increasing architectural complexity.

Source: https://www.emergentmind.com/topics/simple-relation-decoupling-srd