Maximizing Relation Diversity in FSRC

• Maximum relation diversity is defined as the number of distinct relation types in training, emphasizing breadth of semantic coverage over sheer data quantity.
• Empirical results on the REBEL-FS benchmark indicate that increasing relation types can improve accuracy from approximately 80.9% to 91.4% in 5-shot settings.
• A fixed-budget sampling strategy, which allocates minimal examples per relation, demonstrates that targeted data curation reduces overfitting and enhances transfer performance.

Searching arXiv for the specified topic and paper. Maximum relation diversity, in few-shot relation classification (FSRC), denotes the objective of exposing a model to as many distinct relation types as possible during training, rather than concentrating a fixed labeling budget on many examples of a small relation set. In "Diversity Over Quantity: A Lesson From Few-Shot Relation Classification" (Cohen et al., 2024), this idea is formalized through the cardinality of the training relation label set and evaluated through controlled sampling experiments that hold the total number of training instances fixed. The central claim is that, for FSRC, diversity in relation types is more crucial than raw dataset size for generalization to unseen relations, and that targeted data curation focused on diversity can substantially reduce the need for large-scale datasets (Cohen et al., 2024).

1. Formalization of relation diversity

The paper defines relation diversity on a training set $D_{\mathrm{train}}=\{(x_i,y_i)\}$ by first introducing the set of unique relation labels

$R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$

and then setting

$\delta(D_{\mathrm{train}})=|R|.$

Under this definition, relation diversity is simply the number of distinct relation types present in the training data (Cohen et al., 2024).

The work does not introduce a Shannon-entropy or coverage index for relations. Instead, it tracks how many relations occur at or above a given frequency threshold. For $K\in\mathbb{N}$,

$$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$

where

$$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$

This quantity is plotted on a log-log scale and is used to form relation subsets of size $29$, $79$, $233$, $461$, and $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$0, corresponding to $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$1, $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$2, $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$3, $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$4, and $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$5, respectively (Cohen et al., 2024).

In this framework, “maximum” relation diversity is not introduced as a separate optimization theorem. Rather, it is operationalized by increasing $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$6 under a fixed total example budget $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$7, so that the direct effect of relation breadth can be separated from the effect of dataset size.

2. REBEL-FS as a high-diversity benchmark

The empirical study centers on REBEL-FS, a benchmark carved from the REBEL relation-extraction corpus, which contains $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$8 relations and $R=\{y_i\mid (x_i,y_i)\in D_{\mathrm{train}}\},$9 million sentences (Cohen et al., 2024). The construction protocol is frequency-based: all relations with at least $\delta(D_{\mathrm{train}})=|R|.$0 instances are retained for training, yielding $\delta(D_{\mathrm{train}})=|R|.$1 train relations; the remaining relations are split into $\delta(D_{\mathrm{train}})=|R|.$2 dev relations and $\delta(D_{\mathrm{train}})=|R|.$3 test relations. The resulting benchmark contains $\delta(D_{\mathrm{train}})=|R|.$4 distinct relations in total (Cohen et al., 2024).

The paper emphasizes that REBEL-FS incorporates an order of magnitude more relation types than existing datasets. This matters because the benchmark is intended to stress generalization to novel relations under few-shot conditions rather than memorization of a narrow label space.

Component	Relation count
Train	576
Dev	100
Test	278
Total distinct relations	954

Few-shot samples are generated as sentence pairs. Positive pairs consist of two sentences carrying the same REBEL relation, while negative pairs consist of two sentences carrying different relations. In the main experiments, the pairing process maintains $\delta(D_{\mathrm{train}})=|R|.$5 positive and $\delta(D_{\mathrm{train}})=|R|.$6 negative examples; additional experiments use $\delta(D_{\mathrm{train}})=|R|.$7 and $\delta(D_{\mathrm{train}})=|R|.$8 negative settings (Cohen et al., 2024).

The frequency-threshold structure used for ablation is also explicit:

Minimum frequency $\delta(D_{\mathrm{train}})=|R|.$9	Relations with $K\in\mathbb{N}$0
5000	29
1000	79
500	233
100	461
40	576

This organization makes it possible to vary relation diversity systematically while controlling for data budget and minimum per-relation support.

3. Fixed-budget sampling and diversity maximization

No new “diversity-maximization algorithm” is formally presented. Instead, the paper studies an effective controlled-sampling strategy for maximizing relation diversity subject to a fixed number of training instances $K\in\mathbb{N}$1 (Cohen et al., 2024). The procedure is: fix $K\in\mathbb{N}$2, choose a target number of relations $K\in\mathbb{N}$3, and uniformly sample $K\in\mathbb{N}$4 examples per selected relation, or as close as possible. If some relations do not contain enough examples, only relations above the relevant minimum frequency threshold are considered.

This design is used in two principal ways. In one setting, $K\in\mathbb{N}$5 is varied from $K\in\mathbb{N}$6 to $K\in\mathbb{N}$7 while holding $K\in\mathbb{N}$8 constant, as in the fixed-budget experiments of Figure 1. In another, the ablation over $K\in\mathbb{N}$9, $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$0, $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$1, $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$2, and $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$3 relations keeps total dataset size fixed at values such as $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$4K, $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$5K, and $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$6K, again isolating the effect of $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$7 (Cohen et al., 2024).

Methodologically, this matters because it changes only the breadth of relation exposure while suppressing the confounding effect of total sample count. The experiments therefore target the specific question of whether a model benefits more from more examples or from more relation types.

4. Generalization effects under increasing relation diversity

The fixed-budget diversity sweep provides the paper’s clearest evidence. With $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$8K training examples and $$N_{\mathrm{rel}}(K)=\left|\{r\in R:\mathrm{freq}(r)\ge K\}\right|,$$9 relations, performance on FewRel, CORE, and TACRED-FS test sets rises monotonically with $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$0 for $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$1-way $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$2-shot evaluation at $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$3 and $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$4 (Cohen et al., 2024). One reported example is that, at $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$5, accuracy on FewRel climbs from approximately $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$6 when $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$7 to approximately $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$8 when $$\mathrm{freq}(r)=\left|\{(x_i,y_i)\in D_{\mathrm{train}}:y_i=r\}\right|.$$9.

The cross-dataset generalization results point in the same direction. Models trained on REBEL-FS with $29$0 relations outperform or match models trained on FewRel and CORE when evaluated on the other datasets, although they fall short on out-of-domain TACRED-FS (Cohen et al., 2024). This indicates that large relation inventories improve transfer across relation-classification benchmarks, but domain shift remains a limiting factor.

The paper frames these findings as a challenge to the common assumption that more data alone leads to better performance. Under a controlled budget, the decisive variable is not merely the number of examples, but the diversity of the relation types from which those examples are drawn.

5. High-negative regimes, data efficiency, and overfitting

The siamese-pair ablation in § 5.5 studies five relation-set sizes, three NOTA rates ($29$1, $29$2, $29$3), and three dataset sizes ($29$4K, $29$5K, $29$6K) (Cohen et al., 2024). At $29$7K examples, F1 at $29$8 NOTA rises from $29$9 for $79$0 relations to $79$1 for $79$2 relations. In the $79$3 NOTA setting, F1 rises from $79$4 to $79$5, a gain of $79$6 points and $79$7 relative improvement. In the $79$8 NOTA setting, it rises from $79$9 to $233$0, a gain of $233$1 points and $233$2 relative improvement (Cohen et al., 2024).

Setting	F1 change from 29 to 576 relations
50% NOTA, 100K examples	76.98 $233$3 91.23
90% NOTA	27.24 $233$4 55.38
99% NOTA	3.32 $233$5 7.71

A further result is that very small datasets, such as $233$6K examples, achieve near-identical performance to $233$7K examples when $233$8 is high, specifically at $233$9 relations (Cohen et al., 2024). The paper interprets this as evidence that relation diversity can replace sheer data quantity.

The overfitting curves in § 5.6 reinforce that interpretation. Models trained on only $461$0 or $461$1 relations overfit after one epoch; with $461$2 relations, overfitting appears after approximately two epochs; with $461$3 or $461$4 relations, no overfitting is observed over four epochs (Cohen et al., 2024). This establishes a direct link between low relation diversity and rapid memorization of narrow semantic, syntactic, and lexical patterns.

6. Interpretation, practical prescriptions, and scope

The paper does not provide formal theorems or proofs for why relation diversity helps. Its justification is intuitive: different relations invoke different semantic, syntactic, and lexical patterns; training on a small set of relations overfits the model to those patterns; and unseen relations then fall outside that manifold. Exposure to a broader “spanning set” of relations helps the model learn more general relation-recognition mechanisms and reduces overfitting (Cohen et al., 2024).

From this argument and the empirical study, the paper derives explicit practical guidelines. It recommends “breadth first” sampling: given a labeling budget $461$5, sample the minimal number of examples per relation across as many distinct relations as possible, rather than allocating many examples to a small relation inventory. It further recommends aiming for at least $461$6–$461$7 distinct relations in a few-shot training corpus to move beyond the main point of diminishing returns; including rare relations in evaluation by reserving a separate dev/test split for relations with fewer than $461$8 examples; diversifying aggressively to $461$9 relation types in high-NOTA settings; monitoring overfitting behavior, especially degradation within one to two epochs; and, when annotation is expensive, preferring one example each for many relations over ten examples for few relations (Cohen et al., 2024).

A common misconception addressed by the study is that the primary route to better FSRC is simply scaling up the number of labeled examples. The reported results do not deny that quantity matters, but they show that, under fixed budgets and in few-shot generalization settings, the number of distinct training relations is the more consequential variable. In that sense, maximum relation diversity is best understood as a data-curation principle for FSRC: maximize the breadth of relation supervision first, and only then increase per-relation depth.