---
title: Difficulty-Influence Quadrant (DIQ)
url: https://www.emergentmind.com/topics/difficulty-influence-quadrant-diq
type: topic
---

# Difficulty-Influence Quadrant (DIQ)

The Difficulty-Influence Quadrant (DIQ), as introduced in the curriculum-learning literature, is a four-quadrant categorisation of difficulty signals used to analyse whether the properties that humans regard as difficult are the same properties that neural models find difficult to learn. The framework is defined by two axes—**Source** and **Scope**—and partitions signals into **Human** versus **Model** and **Task-agnostic** versus **Task-dependent** categories. In this formulation, DIQ functions both as a taxonomy for existing difficulty estimators and as a diagnostic tool for testing a central curriculum-learning assumption: that human linguistic difficulty, model learning difficulty, and useful curriculum orderings are closely aligned [2601.01488].

## 1. Axes and quadrant structure

DIQ splits difficulty signals along two dimensions. The first is **Source**: a signal is either **Human**, meaning that it is based on human judgment or human-derived linguistic properties, or **Model**, meaning that it is derived from a model’s own behaviour during training or inference. The second is **Scope**: a signal is either **Task-agnostic**, meaning that it is intended to measure difficulty independently of a downstream task, or **Task-dependent**, meaning that it is defined relative to a specific task or label space [2601.01488].

These two axes yield four quadrants.

| Quadrant | Expansion | Definition |
|---|---|---|
| TA-H | Task-agnostic Human | Human-derived linguistic difficulty proxies independent of the task label |
| TA-M | Task-agnostic Model | Model-based but not task-specific difficulty signals |
| TD-H | Task-dependent Human | Human difficulty signals tied to task ambiguity or disagreement |
| TD-M | Task-dependent Model | Model difficulty signals derived from training dynamics on a specific task |

The framework is useful because curriculum learning often mixes such signals implicitly while assuming that they reflect the same underlying notion of difficulty. DIQ makes those assumptions explicit and testable. Within the paper’s formulation, the issue is not merely terminological. The distinction between source and scope determines whether a signal is expected to track lexical or syntactic complexity, task ambiguity, or model-specific learnability, and therefore whether it is plausible to use that signal as a curriculum variable [2601.01488].

## 2. Signal families represented in the four quadrants

In **Task-agnostic Human Difficulty (TA-H)**, the paper places linguistic difficulty proxies that do not depend on the task label. The listed examples are **sentence length**, **word rarity**, **Flesch Reading Ease (FRE)**, **age-of-acquisition (AOA)**, **concreteness**, **prevalence**, **syntactic diversity**, **syntactic complexity**, and **SLE, a learned reference-less metric of sentence difficulty**. The stated intuition is that these measures reflect how hard the input language is for people to process, regardless of whether the downstream task is NLI, sentiment, or another supervised objective [2601.01488].

In **Task-agnostic Model Difficulty (TA-M)**, the paper uses **perplexity** from a pretrained language model, computed before finetuning. More specifically, it uses the **average perplexity over masked tokens** in the input. The intended intuition is that if a pretrained language model finds a sequence surprising or low-probability, the input may be hard in a general linguistic sense. The empirical analysis later tests whether this intuition is actually borne out [2601.01488].

In **Task-dependent Human Difficulty (TD-H)**, the paper uses **inter-annotator disagreement**, operationalised as **annotation entropy** over multiple labels for each instance. The directionality is explicit: higher entropy means more disagreement and therefore more task-dependent human uncertainty. This places ambiguity, underspecification, and subjectivity inside the formal representation of difficulty rather than treating them as annotation noise [2601.01488].

In **Task-dependent Model Difficulty (TD-M)**, the framework relies on **training dynamics**. The tracked quantities are **confidence** in the correct label, **variability** of confidence across epochs, **correctness** across training, and **loss** and its variability. The paper explicitly connects this quadrant to the **dataset cartography** idea: examples that are consistently low-confidence or high-loss are harder for a model to learn. This quadrant therefore captures difficulty as a property of optimisation behaviour on a particular task rather than as a surface property of the input [2601.01488].

## 3. Hypotheses and analytical design

The paper formulates five hypotheses about how these quadrants should interact. **H1** states that TA-H measures should not all collapse into one factor, because linguistic difficulty is multi-dimensional. **H2** states that perplexity should correlate at least moderately with some human linguistic features, since both reflect lexical or syntactic expectations. **H3** states that linguistic difficulty might contribute to annotator disagreement. **H4** states that if linguistic heuristics are useful for curriculum learning, they should relate to model learning difficulty. **H5** states that annotation disagreement should correlate with model difficulty, as in dataset cartography [2601.01488].

The empirical study is conducted on **SNLI**. The dataset choice is motivated by the fact that SNLI provides **four independent annotator labels per instance** in the training split, which makes it possible to compute **annotation entropy** as a TD-H signal. The study computes **TA-H** in preprocessing from sentence-level features, **TA-M** as perplexity from pretrained **BERT-base**, **RoBERTa-base**, and **GPT-2-base**, and **TD-M** as training dynamics collected on **3,647 data points** at **12 evenly spaced checkpoints** [2601.01488].

The training details are specified separately for model families. For BERT and RoBERTa, the paper uses **batch size = 64** and **learning rate = \(3 \times 10^{-5}\)**. For GPT-2, it uses **batch size = 16** and **learning rate = \(10^{-5}\)**. For all models, it uses **5 epochs**, **AdamW with weight decay 0.01**, a **linear learning-rate schedule**, and **6% warm-up** [2601.01488].

The analysis proceeds in four ways: **Pearson correlations** within TA-H and between TA-H and TD-H or TD-M; **multivariate regression** predicting TD-H or TD-M from all TA features jointly using linear and tree-based models; **dataset cartography-style grouping** comparing the top and bottom 25% of examples, including “easy-to-learn” versus “ambiguous”; and a **sanity check** that replicates the known relation between annotation entropy and model ambiguity [2601.01488].

## 4. Empirical relationships among the quadrants

The first finding is that **TA-H signals are internally diverse**. The paper reports that TA-H features are only weakly or moderately correlated, with two notable exceptions: **length–complexity: \(r = 0.71\)** and **AOA–FRE: \(r = -0.58\)**. Most other pairs have low correlations. The paper’s interpretation is that linguistic difficulty is multi-dimensional and that no single surface notion captures it all [2601.01488].

The second finding is that **TA-M does not align with TA-H as expected**. Perplexity shows **no meaningful correlation** with the TA-H features. The expected relation
$$
\text{TA-H} \leftrightarrow \text{TA-M}
$$
is therefore not supported. The paper interprets this as evidence that pretrained language-model surprisal is not a simple proxy for the human-style linguistic difficulty measures used in the study [2601.01488].

The third finding is the strongest: **TA-H does not predict human disagreement or model difficulty**. The reported correlations between TA-H and TD-H are essentially absent, and the correlations between TA-H and TD-M are also essentially absent. The regression analyses yield very low predictive power, with **\(R^2 < 0.05\)** for annotation entropy and **\(R^2 < 0.1\)** for TD-M metrics. The histogram analysis further shows that “easy” versus “ambiguous” examples overlap almost completely in terms of task-agnostic linguistic features. The paper interprets this as showing that surface linguistic complexity does not meaningfully distinguish either what humans disagree on or what models find hard to learn [2601.01488].

The fourth finding is that **TD-H and TD-M do align**. Annotation entropy correlates with **lower correctness**, **lower confidence**, **higher loss**, and **higher variability**. The paper treats this as confirmation that human label disagreement captures a kind of task-specific ambiguity that models also struggle with [2601.01488].

Taken together, the reported pattern is:
$$
\text{TA-H} \not\approx \text{TD-H}, \quad \text{TA-H} \not\approx \text{TD-M}, \quad \text{TD-H} \approx \text{TD-M}.
$$
This implies, in the paper’s own framing, that **task-agnostic and task-dependent difficulty signals are largely independent**, and that only **task-dependent features align** [2601.01488].

## 5. Consequences for curriculum learning

The framework matters because curriculum learning is described as having two components: a **difficulty estimator** and a **scheduler** that decides when each example is shown. The paper’s core claim is that curriculum-learning research often assumes a simple chain from human linguistic difficulty to model learning difficulty to a useful curriculum ordering. DIQ is used to test that assumption directly rather than accepting it as an implicit premise [2601.01488].

On the reported evidence, the common use of surface heuristics in curriculum learning becomes difficult to justify as a claim about genuine model difficulty. The paper states that **surface heuristics are cheap but unreliable as model-difficulty proxies**: length, readability, lexical rarity, and similar measures are easy to compute, but they do **not** track model learning difficulty well in this study. By contrast, **annotation entropy is informative but expensive**, because it requires multiple annotations, and **training-dynamics signals are useful but post hoc**, because they require training the model first and are therefore not directly usable for pre-training curricula [2601.01488].

The practical recommendation that follows is explicit: the main open problem is to develop **lightweight, task-dependent estimators** that can be computed before full training but still reflect actual model learning behaviour. The paper also argues that if task-agnostic heuristics still help in practice, their success may not come from accurately estimating difficulty. Instead, success may come from the way the **scheduler** reshapes the training distribution over time, how exposure order changes optimisation dynamics, or other task-specific effects unrelated to true difficulty estimation. This suggests a shift in emphasis from surface difficulty proxies to the interaction between estimator design, task dependence, and scheduler behaviour [2601.01488].

## 6. Related terminology, adjacent frameworks, and common confusions

A recurrent source of confusion is that **DIQ is not the same as DQI**. In "DQI: A Guide to Benchmark Evaluation," **DQI** stands for **Data Quality Index**, a benchmark-evaluation metric proposed to quantify the quality of datasets or benchmarks by measuring how much they are affected by bias or artifacts and how much they support generalization. It is a **generic, empirical, composite metric** built from **7 components**, with the overall form
$$
DQI = f(DQI_1, DQI_2, DQI_3, DQI_4, DQI_5, DQI_6, DQI_7),
$$
where \(f\) is **task- and dataset-dependent** and must be tuned experimentally. That paper explicitly states that it does **not** define a quadrant-based framework called DIQ or “Difficulty-Influence Quadrant” [2008.03964].

A second confusion arises from **DIQ-H**, which stands for **“Degraded Image Quality leading to Hallucinations”**. DIQ-H is a benchmark for evaluating VLM robustness under dynamic visual degradation in temporal sequences. It introduces metrics such as **Hallucination Rate**, **Recovery Rate**, and **Temporal Consistency**, and an adaptive **Difficulty Calibrator** with
$$
\lambda_t = \alpha \cdot \text{EPI}_{t-1} + \beta \cdot \text{HR}_{t-1}.
$$
However, the paper explicitly states that this is **not** a formal “Difficulty-Influence Quadrant” or a quadrant-based DIQ analysis; “DIQ” there is part of the benchmark name rather than a four-quadrant curriculum-learning taxonomy [2512.03992].

A third nearby but distinct framework is the **Inward and Outward Network Influence (IONI)** model. IONI introduces two influence parameters for network nodes—**outward influence** and **inward influence**—and classifies nodes into four quadrants: high inward and high outward, low inward and high outward, low inward and low outward, and high inward and low outward. Its main model is
$$
\mathbb{Y}=\Gamma W\Lambda \mathbb{Y}+\mathbb{X}\eta+\varepsilon,
$$
with \(\gamma_i\) as an inward influence index and \(\lambda_j\) as an outward influence index. Although this is a genuine four-quadrant framework, its axes concern **receptivity** and **influence magnitude** in network analysis rather than **source** and **scope** of difficulty signals in curriculum learning [2205.07297].

These distinctions matter because the acronym overlap can obscure substantial conceptual differences. In the curriculum-learning usage, DIQ is a quadrant-based categorisation of difficulty estimators. In DQI, the object of analysis is benchmark quality. In DIQ-H, the object of analysis is hallucination persistence under temporal visual degradation. In IONI, the object of analysis is heterogeneous nodal influence in networks. The shared quadrant or acronym vocabulary does not imply a shared formalism.

## 7. Position within the study of difficulty estimation

Within the literature represented here, DIQ occupies a specific methodological role. It is not presented as a new optimisation algorithm or scheduler. Rather, it is a framework for organising difficulty signals and for testing whether widely used estimators are commensurate. The central result is negative in an important sense: **task-agnostic human linguistic heuristics are internally varied, but do not align well with model learning difficulty**, whereas **task-dependent human disagreement and task-dependent model dynamics do align** [2601.01488].

For researchers working on curriculum learning, this repositions the status of many common heuristics. A signal such as sentence length or readability may still be operationally convenient, but the reported evidence does not support treating it as a faithful surrogate for what the model actually struggles to learn. By contrast, signals grounded in annotation entropy or training dynamics appear closer to the relevant notion of difficulty, though they are more expensive or only available after partial training. The framework therefore sharpens an existing methodological divide: the easiest difficulty estimators to compute are not necessarily the most informative, while the most informative estimators may be costly or post hoc.

In that sense, DIQ is best understood as a formal device for separating kinds of difficulty claims that curriculum-learning practice has often conflated. It makes explicit whether a proposed difficulty measure is human- or model-derived, whether it is task-agnostic or task-dependent, and whether it has any demonstrated relation to model learning behaviour. That clarification is the framework’s main contribution.

Source: https://www.emergentmind.com/topics/difficulty-influence-quadrant-diq