Tiered Distractors in Assessment and Learning

• Tiered distractors are a structured arrangement of incorrect options, ordered by confusability or difficulty to specifically target misconceptions and cognitive levels.
• They are operationalized through distinct methods in high-school physics MCQs, identity representation learning, and cloze distractor generation using concept maps, contrastive objectives, and ensemble ranking.
• By enforcing a graded hierarchy, these techniques improve assessment validity, reduce guessing, and enhance model discrimination in both educational and representation learning settings.

Tiered distractors are distractors or negative examples arranged in an explicit hierarchy of confusability, difficulty, or conceptual depth. In educational assessment, they are used to align answer options with prerequisite confusions, documented misconceptions, and Bloom-level targets; in representation learning, they are used to enforce an ordering between true matches, near-identity confounders, and unrelated negatives. Recent work instantiates this idea in concept-map–guided multiple-choice question generation for high-school physics, matched-context identity representation learning, and difficulty-controllable cloze distractor generation, showing that distractor tiering can be operationalized through structured knowledge retrieval, contrastive objectives, or explicit difficulty conditioning (Scaria et al., 2 May 2025, Cvejic et al., 2 Apr 2026, Kang et al., 3 Nov 2025).

1. Core concept and operational forms

Tiered distractors are defined by ordered relation, not merely by incorrectness. In the concept-map framework for physics MCQs, the ordering is tied to Bloom’s levels: “Remember” distractors draw from simple prerequisite confusions, “Understand” distractors use shallow misconceptions such as incorrect formulas, and “Apply” and above distractors address deeper conceptual mis-applications or combine two misconceptions. In NearID, the ordering is explicitly contrastive: same identity must rank above a NearID distractor, which in turn must rank above a random negative. In DCDG, distractors are sorted by ensemble QA confidence and split into “Easy” and “Hard,” with the middle third discarded (Scaria et al., 2 May 2025, Cvejic et al., 2 Apr 2026, Kang et al., 3 Nov 2025).

Domain	Tiers or ordering	Operational target
High-school physics MCQs	Remember, Understand, Apply, Analyze, Evaluate; distractors drawn from prerequisites, misconceptions, or deeper misapplication	Assessment across cognitive levels
Identity representation learning	same identity $>$ NearID distractor $>$ random negative	Identity discrimination under matched context
Cloze distractor generation	Easy vs. Hard after ensemble ranking; middle third discarded	Difficulty-controllable distractor generation

This suggests that tiering functions as a graded control mechanism over confusability. The common design principle is that distractors are selected or learned so that they challenge the intended inference process rather than merely supplying alternative wrong answers.

2. Knowledge sources and construction pipelines

In the physics MCQ framework, the source of tiering is a hierarchical concept map of high-school classical physics stored in a PostgreSQL database with hybrid JSON fields. The hierarchy has four levels: Unit, Topic, Subtopic, and JSON-encoded subtopic attributes. These attributes include prerequisites, mathematical formulations with LaTeX-encoded equations and variable definitions, common misconceptions, engineering applications, cross-cutting connections, analogies, and curriculum alignments with NCERT references. The structure can be represented as a directed acyclic graph,

$$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$

where edges from $S$ to $S_{pre}$ capture prerequisite relationships. Topic matching is performed by a zero-temperature, zero-shot LLM prompt that returns the closest matching topic name deterministically, after which SQL queries retrieve all related subtopics and their JSON attributes for prompt construction (Scaria et al., 2 May 2025).

In DCDG, tiering is built through dataset construction rather than a hand-authored concept graph. The framework uses two-way distractor generation. A fine-tuned seq2seq PLM, Gemma 2 9B, generates on average 29.7 candidates per question from the masked passage plus the ground-truth answer. A second route, Information-Restriction generation, deletes the top $r\cdot|P|$ passage words according to attention scores computed with the final answer token as query, using $r \in \{0.1, 0.2, 0.4\}$ while preserving words immediately around the blank; this yields on average 19.3 IR candidates per question. The two methods have semantic diversity of 66.84% and 69.28%, respectively, with exact-match Jaccard overlap of 12.8%. Candidates are filtered by LanguageTool and by GPT-4o mini with a 2-shot prompt; 21.3% of generated candidates are removed. Difficulty annotation is then produced by an ensemble of 18 small encoder-only QA models, after Box–Cox normalization and averaging across a selected high-stability subset of six models. Candidates are sorted by ensemble score, the bottom third is labeled “Easy,” the top third “Hard,” and pairwise STS $< 0.8$ is enforced within each question (Kang et al., 3 Nov 2025).

In NearID, tiering is induced by dataset design. The NearID dataset contains 19,386 rigid objects, up to 3 views per identity for 45,215 multi-view positive images, and 316,505 NearID distractors. The distractors are created by extracting the anchor’s background mask, choosing $K$ other objects with similar 3D shape or category, and inpainting them into the anchor’s background using SDXL-inpaint, FLUX.1-Fill, FLUX.1-Canny, Qwen-Image ControlNet, and PowerPaint (BrushNet), followed by standardization to $512^2$ resolution. Because the distractor shares the exact same background as the anchor, only the identity signal differs (Cvejic et al., 2 Apr 2026).

3. Mechanisms for injecting tier information into models

The physics MCQ system injects tier information through prompting. The prompt template includes the variables topic, grade, Bloom_level, and definition_of_level. The in-prompt instructions require the model to use only the provided structured context items, generate one question stem aligned to the Bloom’s level, produce exactly four options with one correct key and three distractors, ensure that each distractor maps to a particular misconception or prerequisite, keep language appropriate for grade level, and output strict JSON for automated validation. The example “Kinetic Energy” prompt for Bloom’s level Apply explicitly asks for one distractor from prerequisites, one from simple misconception, and one deeper conceptual misapplication (Scaria et al., 2 May 2025).

DCDG injects tier information through conditional generation and multitask learning. The backbone is Gemma 2 9B, a Transformer-based encoder–decoder with 32 layers, 16 heads, and 4,096-dimensional hidden states, fine-tuned with LoRA using rank $>$0 and $>$1. The main input template is [GEN] [DIFF=EASY/HARD] [K=4] [[PASSAGE](https://www.emergentmind.com/topics/parallel-application-of-slitless-spectroscopy-to-analyze-galaxy-evolution-passage): …] [ANSWER: …], with $>$2 distractors separated by ⟨sep⟩. Two auxiliary tasks are added in the same seq2seq format: Answer Selection & Distractor Difficulty Estimation (ASDE), which predicts the correct answer and the difficulty labels of distractors, and Distractor Detection & Difficulty Estimation (DDDE), which detects an inserted distractor and predicts its difficulty. The special token [DIFF=EASY] or [DIFF=HARD] has its own learned embedding and conditions the entire generation (Kang et al., 3 Nov 2025).

NearID injects tiering through the training objective rather than natural-language prompting. The intended hierarchy is

$>$3

Training uses a frozen backbone with a trainable MAP head. The method combines a discrimination term that pulls anchors and true positives together while pushing away all negatives, including NearID negatives, with a ranking regularizer that enforces NearID distractors to lie closer to the anchor than random negatives. The reported default hyperparameters are $>$4 for ranking weight, $>$5, $>$6–$>$7 positives per identity, $>$8–$>$9 NearID distractors per identity, global batch size $$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$0, and AdamW with learning rate $$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$1 and weight decay $$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$2; only the MAP head is trained (Cvejic et al., 2 Apr 2026).

4. Mathematical formulations and validation criteria

The physics MCQ work evaluates tiered distractors through student and expert metrics. The reported formulas are

$$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$3

$$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$4

and

$$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$5

where $$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$6 is correctness, $$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$7 attempted, $$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$8 guessed, and $$\mathcal{G} = (V, E, \ell)\quad \begin{cases} V = U \cup T \cup S,\ E \subseteq (U\times T)\cup(T\times S)\cup(S\times S_{pre}),\ \ell:V\to\{\text{Unit,Topic,Subtopic}\}, \end{cases}$$9 is a difficulty weight. Inter-rater reliability is measured with Cohen’s $S$0 for binary criteria and quadratic-weighted $S$1 for Bloom’s-level agreement (Scaria et al., 2 May 2025).

NearID defines evaluation through directed similarity margins. For a test identity with positives $S$2 and $S$3 and matched distractors $S$4 and $S$5,

$S$6

where $S$7. Sample Success Rate is strict:

$S$8

and Pairwise Accuracy treats each directed margin as an independent trial. This protocol requires true cross-background positives to outrank matched-context distractors for a sample to count as successful (Cvejic et al., 2 Apr 2026).

DCDG formalizes tier control both in annotation and in training. The Box–Cox normalization used for candidate scoring is

$S$9

with $S_{pre}$0 chosen by maximizing the likelihood of normality per passage and model. The multitask training objective is

$S_{pre}$1

with $S_{pre}$2 in practice. This makes tier assignment part of both data construction and parameter optimization (Kang et al., 3 Nov 2025).

5. Representative examples

The physics MCQ framework provides explicit examples of annotated distractor tiers. For the Apply-level question “A 2 kg object moves at 3 m/s. What is its kinetic energy?”, the distractor 2 kg × 9.8 m/s² × 3 m is labeled an easy tier because it uses $S_{pre}$3; 2 kg × 3 m/s is a moderate tier because it confuses kinetic energy with momentum $S_{pre}$4; and $S_{pre}$5 is a difficult tier because it forgets to square $S_{pre}$6. A second example at Analyze level asks for the ratio of kinetic energies of two equal-mass balls moving at $S_{pre}$7 and $S_{pre}$8; the correct answer is $S_{pre}$9, while distractors include $r\cdot|P|$0 as an easy linear-proportional misconception, $r\cdot|P|$1 as a moderate reversed-ratio confusion, and $r\cdot|P|$2 as a difficult combination of linear and squared mistakes (Scaria et al., 2 May 2025).

NearID illustrates the same principle in metric space rather than option space. For identity CoffeeCup–123, the anchor is a side view of Cup 123 on a wooden table, the positive is a top-down shot of the same cup on a different table, the NearID distractor is Cup 456 inpainted into the original wooden-table scene, and the random negative is an image of a cat on grass. The reported distances are $r\cdot|P|$3, $r\cdot|P|$4, and $r\cdot|P|$5. With hinge margins $r\cdot|P|$6 and $r\cdot|P|$7, the simplified loss evaluates to $r\cdot|P|$8. The example makes explicit that tiered distractors can be represented as ordered negatives with controlled semantic proximity (Cvejic et al., 2 Apr 2026).

These examples show that tiering is not identical to generic hardness. In the educational setting, tiers are attached to particular misconceptions or prerequisite failures. In the identity setting, tiers are attached to controlled context matching and semantic similarity. This suggests that the technical meaning of a “tier” is domain-dependent, while the design goal remains the same: confusability must be structured rather than accidental.

6. Empirical findings, limitations of simpler baselines, and extensions

In the concept-map MCQ study, expert evaluation covered 50 topics $r\cdot|P|$9 5 levels for 750 MCQs. Two physics experts rated each question on eight criteria: Relevance, Correctness, GradeLevel, Similarity, Bloom’sLevel, Plausibility, Misconceptions, and Independence. A question was counted as high-quality only if all criteria were “Yes” from both raters. Inter-annotator agreement $r \in \{0.1, 0.2, 0.4\}$0 ranged from $r \in \{0.1, 0.2, 0.4\}$1 to $r \in \{0.1, 0.2, 0.4\}$2. Overall high-quality success was 37.6% for the Base LLM, 37.2% for RAG, and 75.2% for the concept map method. In learner-centered evaluation with Grade 9 students ($r \in \{0.1, 0.2, 0.4\}$3), the Accuracy values were 44.47% for LLM, 40.66% for RAG, and 37.25% for concept map; Difficulty-weighted accuracy was 41.08%, 37.67%, and 33.60%; Guessing Success Rate was 37.10%, 33.16%, and 28.05%. Chi-square and $r \in \{0.1, 0.2, 0.4\}$4-tests showed that the concept map approach significantly reduced guessing success compared to the base LLM with $r \in \{0.1, 0.2, 0.4\}$5 and Bonferroni-corrected $r \in \{0.1, 0.2, 0.4\}$6 (Scaria et al., 2 May 2025).

In NearID, pre-trained encoders under matched-context evaluation can rank distractors above true cross-view matches. The reported object-level SSR improves from 30.7% for frozen SigLIP2 to 99.2% with NearID training. On part-level discrimination (MTG), SSR rises from 0% to 35% and PA from 0% to 46.5%. Pearson correlation with the MTG oracle improves from 0.180 to 0.465, and human alignment on DreamBench++ rises from 0.516 to 0.545. The ablations are particularly important: simply adding NearID distractors to the softmax denominator yields high SSR but collapses alignment, whereas the two-tier objective preserves both discrimination and human/oracle correlation (Cvejic et al., 2 Apr 2026).

In DCDG, the central claim is controllable difficulty aligned with human perception. Under GPT-4o relative-difficulty evaluation, DCDG + ASDE + DDDE places 64.2% of easy distractors in the “easiest” slot with 0.2% invalid distractors, and 73.3% of hard distractors in the “hardest” slot with 5.1% invalid distractors. The corresponding GPT-4o baselines are lower: for 0-shot GPT-4o, 33.5% of easy distractors are placed in “easiest” and 56.8% of hard distractors in “hardest”; for 5-shot GPT-4o, the values are 46.4% and 53.8%. On Exact-Match F1@10, the combined DCDG + ASDE + DDDE model achieves 13.05 on original CLOTH, 26.64 on augmented Easy, and 41.98 on augmented Hard. Human evaluation involved five ESL learners with less than five years of experience and ten expert English users with more than ten years of experience. On ESL learners over 50 questions, model-generated easy distractors were ranked easiest 72.8% of the time with 0.0% invalid rate, and model-generated hard distractors were ranked hardest 45.6% of the time with 1.6% invalid rate (Kang et al., 3 Nov 2025).

The three lines of work jointly reject a weak view of distractors as interchangeable wrong alternatives. In the physics system, effectiveness depends on mapping distractors to misconceptions and prerequisites. In NearID, effectiveness depends on matched-background control and an explicit negative ordering. In DCDG, effectiveness depends on filtering, QA-based difficulty annotation, and explicit easy/hard conditioning. A plausible implication is that future extensions will move toward finer-grained tiers or continuous control: DCDG explicitly proposes more than two difficulty bins or continuous control by pairing the technique with Item-Response Theory, generalizing Information Restriction to other question formats, and integrating readability or blank-position features into the difficulty encoder (Kang et al., 3 Nov 2025).