FINER: Fine-Grained Negative Queries

Updated 4 July 2026

FINER is a framework that constructs queries with a minimal, precise contradiction embedded in otherwise accurate descriptions to challenge compositional grounding.
It evaluates models through multi-object, multi-attribute, multi-relation, and false-premise 'What' questions using scene-graph and dynamic vocabulary techniques.
FINER-Tuning leverages Direct Preference Optimization on curated positive-negative pairs, significantly improving hallucination detection and attribution accuracy.

Fine-grained Negative Queries (FINER) denotes query constructions in which a description is mostly consistent with the underlying instance or image but contains a single precise contradiction, so that successful prediction requires rejecting a near-miss rather than solving a coarse presence–absence test. In multimodal LLMs (MLLMs), FINER was introduced as a benchmark framework for hallucination analysis under multi-object, multi-attribute, multi-relation, and false-premise “What” questions (Xiao et al., 18 Mar 2026). Closely related formulations appear in open-vocabulary object detection (OVD), where dynamic vocabularies contain hard-negative classes such as “red shirt” versus “blue shirt,” and in ultra-fine-grained entity set expansion, where negative seed entities encode unwanted semantics alongside positive seeds (Bianchi et al., 2023, Li et al., 2024).

1. Conceptual scope and defining properties

FINER is defined against a background of coarse negative queries that mainly test binary object existence, such as asking whether a dog is present. By contrast, a fine-grained negative query injects one subtle contradiction into a composite description that otherwise matches the evidence, forcing the model to reject a sentence containing multiple correct elements and one incorrect element (Xiao et al., 18 Mar 2026). The central failure mode is false-positive hallucination: the model accepts the query because most of its content is true.

The 2026 FINER benchmark studies four settings. In multi-object, one object mention in a set of otherwise correct objects is replaced by a plausible but incorrect object. In multi-attribute, one attribute bound to an object is replaced by an incorrect attribute. In multi-relation, one relation among object pairs is replaced by an incorrect relation. In “What” questions, a Wh-style question contains an incorrect premise, and the correct behavior is not merely to fill the Wh slot but to deny the false premise and assert the true one (Xiao et al., 18 Mar 2026).

A related formulation appears in OVD. There, the relevant distinction is not yes/no answering but whether a detector can localize the correct box and assign the correct phrase among semantically adjacent alternatives that share the same head noun and differ only in modifiers such as color, pattern, or material. Examples include “red shirt” versus “blue shirt,” “striped shirt” versus “polka-dot shirt,” and “metal chair” versus “plastic chair” (Bianchi et al., 2023). This makes FINER-style evaluation a test of attribution under hard-negative classes rather than of category recognition alone.

A common misconception is that FINER is simply another name for hard negatives in the generic sense. The literature is narrower. The negative alternative is intended to be semantically local, minimally contradictory, and usually confusable because the remaining description is correct. HNC’s “hard negative captions,” for example, are defined as captions obtained by altering exactly one piece of information in a positive caption, preserving locality and plausibility (Dönmez et al., 6 May 2026). This suggests that FINER is best understood as a controlled stress test of compositional grounding rather than as arbitrary negative sampling.

2. Formalizations and query construction

In the original FINER benchmark for MLLMs, construction is scene-graph based. Let an image scene graph contain objects $\mathrm{OBJ}$ , attributes $\mathrm{ATTR}$ bound to objects, and relations $\mathrm{REL}$ among object pairs. Positive and negative question pairs are then generated by preserving all but one component of a scene-graph description. For multi-object queries, the negative version replaces one object from a set of $k$ true objects with a plausible incorrect counterpart sampled from a pool of four candidates. For multi-attribute and multi-relation queries, one attribute or one relation is replaced while the rest of the description remains unchanged. For Wh questions, the negative version inserts a wrong attribute into the premise, so the only correct answer is a correction sentence that denies the false attribute and restores the true one (Xiao et al., 18 Mar 2026).

OVD adopts an aligned but differently parameterized construction. The paper notation organizes instances into groups such as $G_1=\{1\}$ and $G_2=\{2,3\}$ , with group-specific vocabularies $V^{G_1}$ and $V^{G_2}$ (Bianchi et al., 2023). In the FINER interpretation provided for that paper, each group collects instances that share a base object class but differ in fine-grained attributes, and the dynamic vocabulary $V^{G_k}$ is composed of positives $P^{G_k}$ and hard negatives $\mathrm{ATTR}$ 0. Phrases have the form $\mathrm{ATTR}$ 1, where $\mathrm{ATTR}$ 2 is the head noun and $\mathrm{ATTR}$ 3 is one or more attribute modifiers. Hard negatives are generated by altering attributes while preserving the head noun, optionally flipping one attribute at a time or multiple attributes jointly in compound phrases such as “red striped shirt” versus “blue striped shirt,” “red polka-dot shirt,” or “blue polka-dot shirt” (Bianchi et al., 2023).

UltraWiki formalizes the same logic for entity-centric retrieval. An Ultra-ESE query is a union of positive and negative seed entities, $\mathrm{ATTR}$ 4, where the positive and negative seeds belong to the same fine-grained semantic class but diverge on specific attribute values (Li et al., 2024). The retrieval objective is explicitly contrastive: $\mathrm{ATTR}$ 5. Here the negative component does not denote random irrelevance; it encodes unwanted attribute constraints, such as “mobile phone brands using Android” versus “mobile phone brands using iOS” (Li et al., 2024).

A further generalization appears in first-stage ranking for set-compositional retrieval. There, FINER corresponds to negated subconstraints that should penalize only the negated parts of an information need. “Disentangled Negation” implements this by subtracting only the coordinates of the negated representation that do not overlap with the positive representation, thereby preserving shared positive signal while penalizing purely negative dimensions (Krasakis et al., 13 Jan 2025).

3. Benchmarks and datasets

The benchmark infrastructure most directly associated with the FINER acronym comprises FINER-CompreCap and FINER-DOCCI (Xiao et al., 18 Mar 2026). FINER-CompreCap builds on CompreCap’s 560 images with human-annotated scene graphs. FINER-DOCCI builds scene graphs from DOCCI’s 5k images paired with long, dense human captions, using conservative extraction and additional visual checks (Xiao et al., 18 Mar 2026).

Benchmark	Source and scope	MCQ counts
FINER-CompreCap	560 images; human-annotated scene graphs; MSCOCO-style images/classes	Multi-obj 6,300; Multi-attr 3,338; Multi-rel 4,280; Wh 3,166
FINER-DOCCI	5k images; dense human captions; open-set evaluation beyond COCO classes	Multi-obj 10,000; Multi-attr 28,630; Multi-rel 11,542; Wh 20,944

For CompreCap, the maximum entities per question are up to 6 objects, 3 attributes, and 3 relations; for DOCCI, the maxima are up to 6 objects, 5 attributes, and 3 relations (Xiao et al., 18 Mar 2026). DOCCI scene graphs are extracted in two stages with Gemini-2.0-Flash and validated jointly with Qwen2.5-VL-72B, caption re-checking, and human verification on sampled data; in total, 1,771 relations are removed in DOCCI (Xiao et al., 18 Mar 2026). Quality assessment using InternVL3.5-8B reports comparable positive-scene-graph accuracies for CompreCap and DOCCI: objects 96.4% versus 96.1%, attributes 91.5% versus 88.3%, and relations 82.8% versus 85.1% (Xiao et al., 18 Mar 2026).

Negative candidate generation is also controlled. Each object, attribute, and relation is assigned four negative counterparts, and ambiguous or false negatives are filtered by entropy-based visual discrimination with Qwen2.5-VL-72B (Xiao et al., 18 Mar 2026). Entropy thresholds differ by dataset and category, for example $\mathrm{ATTR}$ 6, $\mathrm{ATTR}$ 7, and $\mathrm{ATTR}$ 8 for CompreCap, and $\mathrm{ATTR}$ 9, $\mathrm{REL}$ 0, and $\mathrm{REL}$ 1 for DOCCI (Xiao et al., 18 Mar 2026).

The OVD benchmark in (Bianchi et al., 2023) is less fully specified in the excerpt, but its conceptual suite is organized around increasing difficulty and properties such as color, pattern, and material. Difficulty tiers range from one attribute family with few hard negatives per positive to compound multi-attribute phrases with larger negative sets. The excerpt explicitly notes that exact datasets, counts, and splits are not specified there (Bianchi et al., 2023).

Outside the original FINER benchmark, UltraWiki provides a large-scale entity-centric analogue with 50,973 entities, 394,097 entity-labeled sentences, and 236 ultra-fine-grained semantic classes in the abstract, while a later dataset analysis section states 261 ultra-fine-grained classes (Li et al., 2024). HNC provides a vision–language training corpus with 16,416,392 clean-strict training captions and 2,314,832 validation captions derived from GQA scene graphs (Dönmez et al., 6 May 2026).

4. Evaluation protocols and empirical behavior

The primary metric in FINER is paired accuracy, designed to require correctness on both the positive query and its matched negative counterpart:

$\mathrm{REL}$ 2

This guards against yes/no bias and label imbalance (Xiao et al., 18 Mar 2026). For negative filtering during benchmark construction, the entropy over five multiple-choice options is

$\mathrm{REL}$ 3

Random guess baselines are correspondingly low: 4% paired accuracy for uniform 5-way guessing on both halves of the pair, and 6.25% for a polarity-aware random guesser (Xiao et al., 18 Mar 2026).

Empirically, FINER exposes strong granularity sensitivity. In a motivational study across seven granularity levels, InternVL3.5-14B drops on FINER-CompreCap from about 80% at level 1 to about 20% by levels 5–7, and on FINER-DOCCI from about 58% at level 1 to about 15% by levels 6–7 (Xiao et al., 18 Mar 2026). As the number of entities increases, paired accuracy drops for all base models, especially for attributes and relations, and positional bias remains visible even after tuning (Xiao et al., 18 Mar 2026). The dominant error pattern is the co-occurrence trap: models accept a finely wrong description when most of the description matches genuine image content.

Representative category-level gains from FINER-Tuning quantify the same phenomenon. On CompreCap, InternVL-3.5-14B improves from 47.0 to 71.2 on Multi-rel, a gain of 24.2 points, while on DOCCI it improves from 41.4 to 57.0 on the same category (Xiao et al., 18 Mar 2026). The largest improvements tend to occur in relations and false-premise settings, which require rejecting a single incorrect component embedded within otherwise correct structure.

For OVD, the aligned FINER protocol evaluates joint localization and attribution under dynamic vocabularies. A detection is a true positive only if the predicted box matches the ground truth under an IoU threshold and the phrase matches the ground-truth attributes; if IoU is correct but attributes differ, the event is treated as a misattribution, simultaneously producing a false positive for the predicted phrase and a false negative for the correct one (Bianchi et al., 2023). Phrase-level AP, attribute-level AP, and grouped aggregation over vocabularies $\mathrm{REL}$ 4 provide a direct decomposition of localization versus attribution performance. The excerpt further states that HNCR and FINER-mAP formulas are a mapping consistent with FINER goals rather than formulas fixed by the paper (Bianchi et al., 2023). This is significant because standard OVD benchmarks without hard negatives can overestimate fine-grained capability.

5. Training methods and mitigation strategies

FINER-Tuning is the principal mitigation method associated with the FINER benchmark. It post-trains frontier MLLMs with Direct Preference Optimization (DPO) on FINER-inspired preference tuples, ranking accepted responses above rejected responses with $\mathrm{REL}$ 5 and a frozen reference policy (Xiao et al., 18 Mar 2026). The training data are generated from Pixmo-caption using Phi-4-14B, with positive and negative phrases for object, attribute, relation, and Wh categories; the pipeline produces more than 1.6M tuples, and actual DPO training subsamples at most 160k per model (Xiao et al., 18 Mar 2026).

The reported implementation uses LLaMA-Factory with LoRA of rank 32 on projection layers q_proj and v_proj, AdamW, learning rate $\mathrm{REL}$ 6, global batch size 64, one epoch, warmup 0.1, cosine decay, and 2–4 H100 GPUs depending on model size (Xiao et al., 18 Mar 2026). Four backbones are tuned: LLaVA-1.6-7B, Qwen2.5-VL-7B-Instruct, InternVL-3.5-8B, and InternVL-3.5-14B (Xiao et al., 18 Mar 2026).

The gains are not confined to FINER’s own benchmarks. The same tuning improves performance on eight hallucination suites, including DASH, POPE, RePOPE, HallusionBench, AMBER, CRPE_R, MMHal-Bench, and HaloQuest, and improves or maintains performance on six general multimodal benchmarks including MMStar, TextVQA, ChartQA, MMVP, NaturalBench, and V* (Xiao et al., 18 Mar 2026). The paper explicitly describes this as occurring without an alignment tax.

Adjacent literature develops analogous negative-aware training principles. HNC trains image–text matching systems on hard negative captions derived from scene graphs, using within-scene sampling, semantic class constraints, co-occurrence matching, and spatial-noise filtering (Dönmez et al., 6 May 2026). The resulting corpus covers 12 caption types grouped into attribute-based, relation-based, counting-based, existence-based, and reasoning-based categories (Dönmez et al., 6 May 2026). This is not the same benchmark as FINER, but it operationalizes the same idea: minimally contradictory negatives that force token–region grounding and compositional verification.

UltraWiki contributes a retrieval-side counterpart. RetExpan uses positive-only candidate generation followed by negative-aware segmented re-ranking, while GenExpan uses constrained generation, chain-of-thought reasoning, and negative-aware re-ranking (Li et al., 2024). Their improvement patterns show that negative seeds are not merely auxiliary labels; they directly shape the target semantics of ultra-fine-grained expansion.

Related work shows that the FINER principle extends beyond MLLM hallucination analysis into detection, retrieval, and vision–language pretraining.

Domain	Fine-grained negative mechanism	Representative paper
MLLM evaluation	Positive/negative query pairs with one wrong object, attribute, relation, or false premise	(Xiao et al., 18 Mar 2026)
Open-vocabulary detection	Group-specific vocabularies $\mathrm{REL}$ 7 with same head noun and conflicting modifiers	(Bianchi et al., 2023)
Ultra-fine-grained entity expansion	Positive and negative seed entities encoding desired and unwanted attributes	(Li et al., 2024)
Image–text matching	Minimally contradictory hard negative captions sampled from the same scene graph	(Dönmez et al., 6 May 2026)
First-stage ranking and clinical IR	Disentangled negation, signed sparse weights, or implicit negative feedback from negated spans	(Krasakis et al., 13 Jan 2025, Kuhn et al., 2016)

In first-stage ranking, the major technical issue is interference: naive subtraction of a negated constraint can suppress the shared positive signal. “Disentangled Negation” addresses this by penalizing only purely negative coordinates, and on QUEST it achieves NDCG@10 of 0.258 and Recall@100 of 0.398 for set difference, outperforming plain subtraction and several baselines (Krasakis et al., 13 Jan 2025). When sparse retrieval is modified to allow negative term scores, pairwise accuracy on NevIR rises from 23.07% for standard SPLADE to 42.73% with signed weights under the best aggregation scheme (Krasakis et al., 13 Jan 2025). This suggests that explicit negative attribution is important when negation is semantically central.

An earlier precursor appears in clinical IR. There, negated content in clinical narratives is treated as implicit negative feedback, with query decomposition into positive and negated terms and ranking functions such as $\mathrm{REL}$ 8 (Kuhn et al., 2016). On TREC Clinical Decision Support data, untreated negations reduce performance, and the proposed methods reduce the negative impact on early precision by approximately 65.4% (Kuhn et al., 2016). Although this work predates the FINER acronym, it already embodies the principle that fine-grained negative intent must shape ranking directly rather than be ignored.

The main open problems are consistent across the literature. The FINER benchmark notes that large-scale DOCCI scene graphs and negatives rely partly on LLM extraction and filtering rather than full human curation, that rule-based MCQ templates trade flexibility for control, that multi-rel is capped at three relations, and that residual positional bias and challenging Wh false-premise detection remain unresolved (Xiao et al., 18 Mar 2026). The OVD formulation highlights limited attribute coverage, synonym handling, missing part annotations, and the need for part-aware cues and multi-region text–vision alignment (Bianchi et al., 2023). UltraWiki reports sensitivity to negative seed quality, long-tail entities, and semantic drift in iterative generation (Li et al., 2024). HNC identifies residual world priors, sparse scene-graph predicates, and weak coverage of coreference and temporal reasoning (Dönmez et al., 6 May 2026).

Taken together, these lines of work establish FINER not as a single benchmark artifact but as a general evaluation and training principle: the negative example should be semantically adjacent, minimally contradictory, and entangled with otherwise correct evidence. Under that condition, failure reveals whether a system truly grounds attributes, relations, and exclusions, or merely matches the dominant positive content.