---
title: 'DOREMI: Optimizing Tail Relations in DocRE'
url: https://www.emergentmind.com/topics/document-level-relation-extraction-optimizing-the-long-tail-doremi
type: topic
---

# DOREMI: Optimizing Tail Relations in DocRE

Document-level Relation Extraction optimizing the long tail (DOREMI) refers to a modern framework designed to address the pronounced class imbalance inherent in document-level relation extraction (DocRE) tasks. In DocRE, entity-relation pairs within long-form documents often follow a heavy-tailed distribution: a small number of relation types occur frequently (head relations), while the majority appear infrequently (tail relations), leading to poor generalization on rare relation types. DOREMI systematically improves model performance for tail relations with minimal annotation effort by leveraging active instance selection and targeted manual annotation, resulting in substantial improvements on rare relation classes while maintaining or enhancing overall model robustness [2601.11190].

## 1. Characterization of the Long-Tail Problem in DocRE

Document-level RE requires predicting all relevant semantic relations between ordered entity pairs $(s, o)$ from a predefined set $\mathcal{R}$, aggregating evidence across the entire document. In major benchmarks such as DocRED, the relation label distribution is sharply skewed: only a few relations have thousands of training examples, while the majority of relations (“tail relations”) have $O(10-100)$ instances. Models trained on standard distant supervision (DS) corpora achieve near-zero recall on these tail relations due to insufficient supervision and overwhelming gradient signal from head relations [2205.10511, 2212.10171, 2311.07314].

Traditional denoising and augmentation strategies—such as uncertainty-guided label denoising (UGDRE) [2305.11029], generative augmentation via VAEs or LLMs [2412.13503, 2311.07314], or relation co-occurrence modeling [2212.10171]—improve tail performance by either filtering noisy pseudo-labels or generating additional synthetic data. However, these approaches are constrained by the inherent limitations of data quality or the scalability of naive manual annotation.

## 2. DOREMI Framework: Iterative Active Human-in-the-Loop Enhancement

DOREMI (DOcument-level Relation Extraction optiMizing the long taIl) [2601.11190] frames the improvement of tail relation coverage as an iterative, budget-constrained optimization—explicitly leveraging annotation resources to maximize improvements on underrepresented relations.

- **Joint Optimization Objective**
  \[
  \min_{\theta}\; \mathcal{L}(\theta; \mathrm{HA} \cup T)\quad\text{s.t.}\quad |T| \le B
  \]
  where $\theta$ denotes DocRE model parameters, $\mathrm{HA}$ a small human-annotated seed, $T$ the set of selected examples for annotation, and $B$ the annotation budget.

- **Iterative Process**
  At each round $t$, a set $S_t$ of $k$ candidate entity pairs is sampled (from $DS \setminus (\mathrm{HA} \cup T)$) according to an informativeness score $I(x; \theta_{t-1})$. The batch $S_t$ is annotated, models are finetuned, and the selection process is repeated, prioritizing examples with maximum model ensemble disagreement. Aggregated predictions with thresholding produce DDS (Denoised DS) for downstream training.

- **Sampling Criteria**
  - **Per-relation disagreement**:
    \[
    \phi_{\Gamma}(r|(s,o)) = 1 -\Bigl[\prod_{i=1}^n p_{\gamma_i}(r|(s,o)) + \prod_{i=1}^n(1-p_{\gamma_i}(r|(s,o)))\Bigr].
    \]
  - **Aggregate multi-label disagreement**:
    \[
    \psi_{\Gamma}(s,o) = \sum_{r \in \mathcal{R}} \log[\phi_{\Gamma}(r|(s,o))]
    \]
  $S_t$ is selected to maximize $\sum_{x \in S} I(x; \theta_{t-1})$ where $I(x;\theta)$ is instantiated as $\psi_{\Gamma}(s,o)$.

- **Scalability**
  Empirical configurations annotate only 0.001–0.003% of DS, e.g., $B=400$ suffices for DocRED ($\sim$6 expert hours).

## 3. DOREMI Integration with Existing DocRE Models

DOREMI is model-agnostic; any DocRE model—such as ATLOP, DREEAM, or those employing advanced denoising [2305.11029], generative augmentation [2412.13503, 2311.07314], or relation correlation modeling [2212.10171]—can serve as the “core” classifier. At each annotation step, an ensemble of core models is trained and used for informativeness scoring. Once DDS is constructed, the downstream DocRE model is (re-)trained from scratch on $\mathrm{HA} \cup \mathrm{DDS}$, benefitting from reduced noise and improved tail coverage.

Comparisons with UGDRE [2305.11029] and generative augmentation frameworks [2412.13503, 2311.07314] indicate that DOREMI's active sample selection is particularly effective for tail relations, as it directly exploits differences in model uncertainty and disagreement where labeled data is most scarce.

## 4. Quantitative Impact and Empirical Results

DOREMI achieves marked improvements for rare relation classes on DocRED and its cleaned counterpart Re-DocRED. In head-to-head evaluation against label denoising and hybrid annotation approaches, DOREMI delivers:

| Dataset         | Metric        | UGDRE | DOREMI | $\Delta$ (DOREMI - UGDRE) |
|-----------------|--------------|-------|--------|----------------------------|
| DocRED-dev      | Tail F1      | —     | +5.0   | +5.0                       |
| DocRED-dev      | Tail-ignF1   | —     | +28.7  | +28.7                      |
| Re-DocRED-test  | Tail F1      | —     | +16.2  | +16.2                      |
| Re-DocRED-test  | Extreme-tail ignF1 | — | +33.5 | +33.5                      |

Additional precision and recall gains are observed, with overall F1 improvements for tails up to $+5$ points and for extreme-tail relations ($<$100 instances) ignPrecision up to $+207.9$ points [2601.11190]. These results substantially surpass gains reported for uncertainty-driven denoising (UGDRE: +2.28 IgnF1) [2305.11029], correlation-guided augmentation (Correl: +1.54 Macro@100 F1) [2212.10171], and VaeDiff-DocRE generative augmentation (+0.86 LTail F1) [2412.13503].

## 5. Comparison with Alternative Long-Tail Mitigation Strategies

Several orthogonal strategies have been introduced for the long-tail in DocRE, all demonstrating complementary or additive gains:

- **Uncertainty-guided denoising (UGDRE):** Instance-level MC Dropout uncertainty with dynamic class thresholds $\theta_c$ for selective relabeling; large gains for rare classes, but limited by initial DS coverage [2305.11029].
- **Representation-level augmentation (ERA/ERACL):** Attention-weight masking for entity-pair representations; explicitly boosts tail gradients via targeted context perturbation and contrastive learning [2205.10511].
- **Generative data augmentation (VaeDiff-DocRE, LLM+NLI):** Embedding-space VAE+diffusion sampling (class-conditional) for multi-label relation-specific synthetic examples [2412.13503]; LLM in-context generation + NLI mapping, effective for dataset expansion and particularly boosting sparse relations [2311.07314].
- **Correlation modeling:** Explicit co-occurrence tasks (CRCP/FRCP) to transfer representation robustness from head to tail via relation embedding structure [2212.10171].

While denoising and augmentation are effective, DOREMI's budgeted, active annotation is unique in directly targeting the set of instances with maximal expected informational value for tail relation generalization.

## 6. Analysis, Limitations, and Future Directions

DOREMI's performance is dependent on the diversity and reliability of its core model ensemble; if the models share the same architectural backbone, ensemble disagreement can be limited, reducing selection informativeness. Potential extensions include richer cost models for annotation (variable per relation), integration of density/representativeness metrics to avoid outlier oversampling, and incorporation into multi-task RE with supporting evidence prediction.

While the annotation overhead is minimal, further improvements may be realized by integrating LLM-based denoisers or direct generative augmentation to further flatten the long-tail. Scalability to large-scale real-world corpora and highly domain-specific relation sets remains an open area, as does the joint optimization of model and annotation strategies [2601.11190].

## 7. Bibliography and Related Work

Key references for DOREMI and related long-tail DocRE strategies include:
- "DOREMI: Optimizing Long Tail Predictions in Document-Level Relation Extraction" [2601.11190]
- "Uncertainty Guided Label Denoising for Document-level Distant Relation Extraction" [2305.11029]
- "VaeDiff-DocRE: End-to-end Data Augmentation Framework for Document-level Relation Extraction" [2412.13503]
- "Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models" [2311.07314]
- "Document-level Relation Extraction with Relation Correlations" [2212.10171]
- "Improving Long Tailed Document-Level Relation Extraction via Easy Relation Augmentation and Contrastive Learning" [2205.10511]

Source: https://www.emergentmind.com/topics/document-level-relation-extraction-optimizing-the-long-tail-doremi