Simple Relation Decoupling (SRD)

• The paper demonstrates that decoupling feature design from model architecture allows a simple fastText classifier to match complex neural networks, reducing training time by 75×.
• SRD leverages page-specific gazetteers and dependency-path extraction to boost entity resolution precision and accelerate distant supervision by up to 10×.
• The approach scales efficiently to hundreds of relations by shifting the representational burden to robust feature extraction rather than complex modeling.

Searching arXiv for the specified paper to ground the article and verify bibliographic details. Simple Relation Decoupling (SRD) is a relation-extraction approach that separates feature design from model architecture in large-scale extraction from unstructured text. In the formulation described in "Simple Large-scale Relation Extraction from Unstructured Text" (Christodoulopoulos et al., 2018), the task is to learn a function $f : E \times E \times S^* \to R \cup \{\bot\}$ that takes a candidate entity pair $(x,y) \in E \times E$ together with all sentences in which both entities co-occur and predicts either a relation $r \in R$ or “no relation” $(\bot)$. The central claim of SRD is that a much simpler classifier trained on similar features performs on par with a highly complex neural network system, at a $75\times$ reduction to the training time, which suggests that the features are a bigger contributor to the final performance (Christodoulopoulos et al., 2018).

1. Formalization of the extraction task

SRD assumes a knowledge base with entity set $E = \{e_1, e_2, \dots\}$, relation set $R = \{r_1, r_2, \dots\}$, and a corpus of tokenized, dependency-parsed sentences $S = \{s_1, s_2, \dots\}$. For a pair $(x,y)$, the support set is defined as

$$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$

Given a large KB of known triples $(x,y) \in E \times E$0, SRD adopts the distant-supervision assumption: if $(x,y) \in E \times E$1 and $(x,y) \in E \times E$2 both appear in sentence $(x,y) \in E \times E$3, then $(x,y) \in E \times E$4 is noisily a positive example of $(x,y) \in E \times E$5 (Christodoulopoulos et al., 2018).

The resulting noisy training set is

$(x,y) \in E \times E$6

Negative examples are sampled from pairs $(x,y) \in E \times E$7, ensuring ontological constraints, at a fixed ratio such as $(x,y) \in E \times E$8. In this formulation, relation extraction is not defined sentence-by-sentence in isolation; rather, the prediction consumes all supports in which the entity pair co-occurs. This suggests that SRD treats aggregation across mentions as a first-class design choice rather than as a secondary postprocessing step.

2. Distant supervision and page-specific gazetteers

A defining element of SRD is the use of page-specific gazetteers to improve entity-resolution precision during distant supervision (Christodoulopoulos et al., 2018). For each Wikipedia page $(x,y) \in E \times E$9 whose main entity is $r \in R$0, SRD gathers all $r \in R$1-hop neighbors

$r \in R$2

and collects their name-variants in a page-specific lexicon. Entity mentions are then identified by greedy longest-match string lookup on each sentence $r \in R$3 to produce $r \in R$4.

The pipeline can be summarized in four stages. First, page-specific gazetteers are built from KB-linked page entities and their $r \in R$5-hop neighbors. Second, distant supervision and entity resolution produce labeled examples by matching entities in sentences and assigning either a KB relation or $r \in R$6. Third, shortest dependency paths are converted into sparse symbolic features. Fourth, a fastText classifier is trained with cross-entropy and $r \in R$7 regularization (Christodoulopoulos et al., 2018).

The stated effect of page-specific gazetteers is twofold. They reduce the number of candidate entities per sentence from $r \in R$8 to $r \in R$9, making distant-supervision label generation approximately $(\bot)$0 faster and more precise. The trade-off is that the system depends on high-precision entity linkings and a careful distant-supervision pipeline, including page-specific gazetteers and Bloom filters for KB lookup (Christodoulopoulos et al., 2018).

3. Feature design and representation

For each candidate $(\bot)$1 and each supporting sentence $(\bot)$2, SRD extracts the shortest dependency path $(\bot)$3 connecting $(\bot)$4 to $(\bot)$5. Each token $(\bot)$6 on the path is annotated with the following attributes (Christodoulopoulos et al., 2018):

• lemma $(\bot)$7
• Brown-cluster prefix $(\bot)$8
• POS tag $(\bot)$9
• dependency relation $75\times$0
• direction $75\times$1

The representation also includes features for the entities themselves: $75\times$2, $75\times$3, and the $75\times$4-bit Brown prefixes of their names, $75\times$5 and $75\times$6.

If $75\times$7, the feature multiset is

$75\times$8

where the atomic feature vector for one support is the set of symbolic strings

$75\times$9

In practice, $E = \{e_1, e_2, \dots\}$0 is treated as a sparse indicator vector over all such strings. SRD then collapses the multiset $E = \{e_1, e_2, \dots\}$1 into a single sparse feature vector $E = \{e_1, e_2, \dots\}$2, explicitly characterized as a bag-of-features representation (Christodoulopoulos et al., 2018).

The feature inventory is syntacto-semantic: it preserves dependency-path structure through path extraction, but the final classifier consumes only sparse symbolic indicators. The paper’s interpretation is explicit: rich features such as dependency paths and Brown clusters drive most of the performance, not sophisticated recurrent architectures. A plausible implication is that SRD relocates representational burden from the classifier into the feature extractor.

4. Linear classification and decision rule

SRD trains a linear multiclass classifier, implemented with fastText in the reported experiments, over the sparse feature vector $E = \{e_1, e_2, \dots\}$3 (Christodoulopoulos et al., 2018). The score for relation $E = \{e_1, e_2, \dots\}$4 is

$E = \{e_1, e_2, \dots\}$5

where $E = \{e_1, e_2, \dots\}$6 and $E = \{e_1, e_2, \dots\}$7. Scores are converted to probabilities by softmax:

$E = \{e_1, e_2, \dots\}$8

Learning minimizes the negative log-likelihood with $E = \{e_1, e_2, \dots\}$9 regularization,

$R = \{r_1, r_2, \dots\}$0

with $R = \{r_1, r_2, \dots\}$1 a small $R = \{r_1, r_2, \dots\}$2 regularizer. At inference time, prediction is made by

$R = \{r_1, r_2, \dots\}$3

The pseudocode in the source description makes the decoupling explicit: the system first constructs sparse path-based features, then applies a standard linear classifier trained by SGD on cross-entropy plus $R = \{r_1, r_2, \dots\}$4. This suggests that SRD is less a new classifier family than a design argument about where relation-extraction performance originates.

5. Relation to HypeNET and empirical results

SRD is presented through a direct comparison with HypeNET, which is structurally more complex (Christodoulopoulos et al., 2018). HypeNET includes embedding layers for lemmas, POS, Brown clusters, dependency relations, and direction; a uni- or bi-directional LSTM over the token sequence of each path; average-pooling over the $R = \{r_1, r_2, \dots\}$5 supports to obtain a path representation $R = \{r_1, r_2, \dots\}$6; and concatenation of $R = \{r_1, r_2, \dots\}$7 with learned embeddings of $R = \{r_1, r_2, \dots\}$8 and $R = \{r_1, r_2, \dots\}$9 before a softmax layer.

Its parameter count is described as on the order of a few million, approximately $S = \{s_1, s_2, \dots\}$0–$S = \{s_1, s_2, \dots\}$1M weights, and training on $S = \{s_1, s_2, \dots\}$2K examples takes approximately $S = \{s_1, s_2, \dots\}$3 min on a single GPU. By contrast, the SRD fastText model has only $S = \{s_1, s_2, \dots\}$4 parameters but trains in $S = \{s_1, s_2, \dots\}$5 min on the same hardware (Christodoulopoulos et al., 2018).

Relation	HypeNET $S = \{s_1, s_2, \dots\}$6	fastText $S = \{s_1, s_2, \dots\}$7
Wikidata “instance of”	$S = \{s_1, s_2, \dots\}$8	$S = \{s_1, s_2, \dots\}$9
Wikidata “birthplace of”	$(x,y)$0	$(x,y)$1
Wikidata “part of”	$(x,y)$2	$(x,y)$3
Alexa KB “instance of”	$(x,y)$4	$(x,y)$5
Alexa KB “birthplace of”	$(x,y)$6	$(x,y)$7
Alexa KB “applies to”	$(x,y)$8	$(x,y)$9

The same comparison also reports MaxEnt baselines of $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$0, $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$1, and $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$2 on Wikidata “instance of,” “birthplace of,” and “part of,” and $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$3, $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$4, and $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$5 on Alexa KB “instance of,” “birthplace of,” and “applies to,” respectively (Christodoulopoulos et al., 2018). The reported conclusion is that SRD’s fastText matches or beats HypeNET while cutting training time by approximately $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$6.

A common misconception in discussions of this result is to treat it as a claim that neural architectures are ineffective for relation extraction. The actual claim is narrower: once the feature design of a state-of-the-art neural network system is decoupled from the model architecture, a simpler classifier trained on similar features performs on par with the more complex system (Christodoulopoulos et al., 2018).

6. Scalability, trade-offs, and extensibility

The computational profile of SRD is stated in simple asymptotic terms (Christodoulopoulos et al., 2018). Feature extraction is linear in total tokens and dependency-path length, $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$7. fastText training is $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$8, where $$S(x,y) = \{ s \in S : x \in \mathrm{ents}(s) \wedge y \in \mathrm{ents}(s) \}.$$9 is the number of training examples, $(x,y) \in E \times E$00 is the average number of nonzeros per $(x,y) \in E \times E$01, and $(x,y) \in E \times E$02 is the number of epochs. Empirically, training is reported as $(x,y) \in E \times E$03 min on $(x,y) \in E \times E$04K examples. HypeNET’s per-example cost includes an LSTM over each support path and gradient computation through all embedding layers, yielding $(x,y) \in E \times E$05 and approximately $(x,y) \in E \times E$06 slower training.

The paper’s stated trade-off is that SRD requires a well-engineered feature extractor, including dependency parses and Brown clustering, together with a careful distant-supervision pipeline. However, once that infrastructure is in place, it is described as trivially extended to hundreds of relations: simply generate more $(x,y) \in E \times E$07 triples and re-train the same fastText model (Christodoulopoulos et al., 2018).

The broader significance of SRD lies in its decomposition of the relation-extraction stack. By assigning most of the burden to entity resolution, dependency-path extraction, and symbolic feature construction, it argues that model simplicity need not imply representational weakness. This suggests a methodological lesson for large-scale RE: when supervision is weak and throughput matters, improving the precision of label generation and the quality of path-based features may yield larger gains than increasing architectural complexity.