CHAIR-DPO: Fine-Tuning to Reduce Object Hallucinations

• CHAIR-DPO is a method that combines the CHAIR hallucination metric with Direct Preference Optimization to tackle object hallucinations in multimodal models.
• It constructs pairwise preferences from visual and textual data, favoring responses with fewer unsupported object mentions.
• The approach achieves notable reductions in hallucination scores across benchmarks while balancing trade-offs with object mention coverage.

CHAIR-DPO is a preference-construction and fine-tuning method for reducing object hallucinations in multimodal LLMs (MLLMs). It combines the CHAIR hallucination metric with Direct Preference Optimization (DPO): for a fixed image $x_I$ and textual prompt $x_T$, it samples two answers, scores both with $CHAIR_i$, labels the less hallucinatory answer as the winner $y_w$ and the more hallucinatory one as the loser $y_l$, and then fine-tunes the MLLM with a standard DPO objective so that the model assigns higher relative probability to $y_w$ than to $y_l$ (Compagnoni et al., 27 Aug 2025). In the paper’s framing, the method addresses hallucination as an alignment problem rather than as a decoding-only or post hoc correction problem: the optimization signal is made explicitly object-aware by depending on whether generated object mentions are supported by the visual input.

1. Problem setting and definition

The target failure mode is object hallucination: the model mentions objects that are not actually present in the image. The paper treats this as especially problematic in visually grounded generation because the output can remain fluent and plausible while being factually ungrounded. Examples in the paper include descriptions that mention a chair, bench, person, or vehicle that does not appear in the image (Compagnoni et al., 27 Aug 2025).

The method’s central premise is that hallucination mitigation can be cast as pairwise preference learning. Given two responses to the same multimodal input, the preferred response is the one with fewer hallucinated objects. This yields a specific interpretation of “object-aware”: the preference signal depends on object presence or absence in the image, not on explicit region grounding, token-level grounding, or a general-purpose learned reward model (Compagnoni et al., 27 Aug 2025).

Formally, for two sampled responses $y_1,y_2 \sim ref(y \mid x_T,x_I)$, the winner and loser are obtained by ranking them with the instance-level CHAIR score:

$$CHAIR_i(y)=\frac{|\text{[hallucinated objects]}_y|}{|\text{[all mentioned objects]}_y|}.$$

The paper states the intended selection rule as

$$y_w=\arg\min_{y\in\{y_1,y_2\}} CHAIR_i(y), \qquad y_l=\arg\max_{y\in\{y_1,y_2\}} CHAIR_i(y).$$

The scope is broader than captioning. Preference data are built from LLaVA-Instruct-665k, which contains multi-turn visual dialogues, and candidate completions are conditioned on both the image and a textual dialogue context. This suggests that CHAIR-DPO is meant for image-grounded instruction following and dialogue whenever object mention accuracy matters, not only for caption generation (Compagnoni et al., 27 Aug 2025).

2. CHAIR-based preference construction

The paper’s main novelty is not a new preference loss but a simple, automatic pipeline for producing DPO pairs from hallucination scores. The input source is LLaVA-Instruct-665k, viewed as triplets $x_T$0, where the original target answer $x_T$1 is discarded for preference construction (Compagnoni et al., 27 Aug 2025).

For multi-turn dialogues, the conversation is randomly truncated: the model receives the first $x_T$2 human-assistant turns plus the current question as textual context, where $x_T$3 is randomly chosen. Two candidate answers are then sampled from the reference MLLM with temperature $x_T$4. The paper does not report other decoding parameters such as top-$x_T$5, top-$x_T$6, or maximum generation length (Compagnoni et al., 27 Aug 2025).

Object supervision on the image side is derived automatically with DETR-DC5-R101. Its predicted class labels are treated as the set of objects present in the image, with vocabulary based on MSCOCO class names. On the text side, object mentions are identified using MSCOCO object class names together with the synonym list from the reference CHAIR implementation. A mentioned object is counted as hallucinated if it is not in the detected object set (Compagnoni et al., 27 Aug 2025).

The paper distinguishes two CHAIR variants. $x_T$7 is the instance-level fraction of mentioned object instances that are hallucinated, with object mentions treated as a list, not a set, so repeated hallucinations can be penalized multiple times. $x_T$8 is the percentage of responses containing at least one hallucinated object; in AMBER evaluation, HalRate corresponds to $x_T$9 (Compagnoni et al., 27 Aug 2025).

A key implementation choice is to discard all pairs with zero $CHAIR_i$0 difference. The paper reports that these pairs produce noisy supervision because they do not contain a meaningful preference signal. After filtering, the final preference datasets contain 70k training samples for LLaVA-1.5-7B and 77k training samples for LLaVA-MORE-8B, with 500 held-out samples for validation in both cases (Compagnoni et al., 27 Aug 2025).

This filtering is not incidental. The paper explicitly states that it yields a more reliable optimization signal and significantly speeds up fine-tuning by eliminating nearly 90% of the dataset (Compagnoni et al., 27 Aug 2025).

3. DPO objective and training protocol

Once winner-loser pairs are constructed, CHAIR-DPO uses a standard DPO loss. The model generates continuation text $CHAIR_i$1 conditioned on prompt $CHAIR_i$2 and image $CHAIR_i$3:

$CHAIR_i$4

The DPO objective is

$CHAIR_i$5

Here $CHAIR_i$6 is the trainable policy model, $CHAIR_i$7 is the frozen reference model, and $CHAIR_i$8 controls the strength of the implicit KL regularization toward the reference policy (Compagnoni et al., 27 Aug 2025).

The paper is explicit that there is no change to the DPO loss itself. The novelty lies in two places: first, constructing preference labels from $CHAIR_i$9; second, filtering out zero-difference pairs. It also notes that CHAIR-DPO could be applied iteratively by updating the reference with the current policy and regenerating preference data, but no iterative experiments are reported (Compagnoni et al., 27 Aug 2025).

Two open-source MLLMs are fine-tuned:

• LLaVA-1.5-7B, based on Vicuna-7B
• LLaVA-MORE-8B, based on LLaMA-3.1-8B

Both follow the LLaVA-style pipeline with CLIP-ViT-L/14@336 as visual encoder and a lightweight vision-language adapter (Compagnoni et al., 27 Aug 2025).

Fine-tuning is performed with LoRA, using rank $y_w$0 and $y_w$1. The optimizer is Adam with a cosine learning rate scheduler and peak learning rate $y_w$2. Warmup uses 33 steps for LLaVA-1.5-7B and 145 steps for LLaVA-MORE-8B. Training runs on 2 nodes with 8 NVIDIA A100 64GB GPUs each, for a total of 16 A100 64GB GPUs, with DeepSpeed ZeRO Stage 2 and gradient checkpointing. The total batch size is 64 for LLaVA-1.5-7B and 16 for LLaVA-MORE-8B. The paper evaluates $y_w$3, $y_w$4, and $y_w$5 (Compagnoni et al., 27 Aug 2025).

The paper also states what CHAIR-DPO does not require: no human preference annotations and no proprietary teacher or judge models such as GPT-4 or Gemini (Compagnoni et al., 27 Aug 2025).

4. Evaluation and empirical behavior

CHAIR-DPO is evaluated on three hallucination benchmarks: AMBER, CHAIR-MSCOCO, and Object HalBench. AMBER uses 1,004 manually annotated images and reports macro-averaged $y_w$6, HalRate, Coverage, and Cognition (Cog). CHAIR-MSCOCO uses 500 images from MSCOCO val and reports $y_w$7 and $y_w$8. Object HalBench uses a 300-image MSCOCO-val subset and reports micro-averaged $y_w$9 and $y_l$0 (Compagnoni et al., 27 Aug 2025).

Benchmark	Size	Metrics
AMBER	1,004 images	$y_l$1, HalRate, Coverage, Cog
CHAIR-MSCOCO	500 images	$y_l$2, $y_l$3
Object HalBench	300 images	$y_l$4, $y_l$5

For LLaVA-1.5-7B, the baseline reports AMBER $y_l$6, Cover $y_l$7, HalRate $y_l$8, and Cog $y_l$9. With CHAIR-DPO at $y_w$0, these become $y_w$1, Cover $y_w$2, HalRate $y_w$3, and Cog $y_w$4. On CHAIR-MSCOCO, baseline $y_w$5 and $y_w$6 drop to $y_w$7 and $y_w$8. On Object HalBench, CHAIR-DPO at $y_w$9 reaches $y_l$0 and $y_l$1 (Compagnoni et al., 27 Aug 2025).

For LLaVA-MORE-8B, the baseline reports AMBER $y_l$2, Cover $y_l$3, HalRate $y_l$4, and Cog $y_l$5. With CHAIR-DPO at $y_l$6, these become $y_l$7, Cover $y_l$8, HalRate $y_l$9, and Cog $y_1,y_2 \sim ref(y \mid x_T,x_I)$0. On CHAIR-MSCOCO, baseline $y_1,y_2 \sim ref(y \mid x_T,x_I)$1 and $y_1,y_2 \sim ref(y \mid x_T,x_I)$2 drop to $y_1,y_2 \sim ref(y \mid x_T,x_I)$3 and $y_1,y_2 \sim ref(y \mid x_T,x_I)$4. On Object HalBench, the corresponding CHAIR-DPO numbers are $y_1,y_2 \sim ref(y \mid x_T,x_I)$5 and $y_1,y_2 \sim ref(y \mid x_T,x_I)$6 (Compagnoni et al., 27 Aug 2025).

Across both backbones, the paper states that lower $y_1,y_2 \sim ref(y \mid x_T,x_I)$7 yields lower hallucination scores but somewhat lower Coverage. It interprets this as a controllable trade-off rather than a collapse in capability. For general MLLM evaluation, the paper reports that CHAIR-DPO improves MME, slightly improves MMMU and Science-QA, and is comparable on SEED and AI2D for LLaVA-1.5-7B, while LLaVA-MORE-8B shows moderate regression on several general benchmarks without catastrophic forgetting (Compagnoni et al., 27 Aug 2025).

The paper also compares CHAIR-DPO to methods including DoLa, VCD, OPERA, Woodpecker, POVID, HA-DPO, HALVA, EOS, mDPO, MFPO, and REVERSE. Its claim is that CHAIR-DPO is always best or second-best on direct hallucination metrics, with a stronger hallucination/coverage trade-off than methods whose lower hallucination numbers are accompanied by severe Coverage reduction (Compagnoni et al., 27 Aug 2025).

5. Strengths, trade-offs, and limitations

The method’s principal strength is operational simplicity. It uses an existing hallucination metric, an off-the-shelf detector, open instruction-tuning data, and standard DPO. This avoids the complex synthetic preference pipelines and proprietary judge models used by several recent hallucination-alignment approaches (Compagnoni et al., 27 Aug 2025).

Another strength is that the supervision signal is tightly targeted. Preference labels are induced by whether mentioned objects are visually present, so the optimization directly favors object-grounded generations. The paper argues that this makes the model more conscious of what objects are present and absent in the image (Compagnoni et al., 27 Aug 2025).

The main reported trade-off is between hallucination reduction and Coverage. Lower $y_1,y_2 \sim ref(y \mid x_T,x_I)$8 improves hallucination metrics but tends to reduce object recall or mention coverage. The paper nevertheless reports mostly preserved general capability for LLaVA-1.5-7B and no catastrophic forgetting for LLaVA-MORE-8B (Compagnoni et al., 27 Aug 2025).

Several limitations are explicit or directly implied by the setup. CHAIR-DPO relies on object presence labels inferred by DETR-DC5-R101, so detector errors can inject noise into preference construction. The paper does not deeply analyze detector error, but the dependence is structurally unavoidable. The method is also limited to object hallucinations as defined by the CHAIR vocabulary: it does not directly address attribute hallucinations, relational hallucinations, temporal hallucinations, or broader semantic errors not tied to object vocabulary. Because CHAIR is tied to MSCOCO class names plus synonym mappings, hallucinations outside that vocabulary may be missed (Compagnoni et al., 27 Aug 2025).

The filtering ablation further shows that CHAIR-DPO depends on high-signal preference pairs. For example, on LLaVA-1.5-7B with $y_1,y_2 \sim ref(y \mid x_T,x_I)$9, training without filtering yields AMBER $$CHAIR_i(y)=\frac{|\text{[hallucinated objects]}_y|}{|\text{[all mentioned objects]}_y|}.$$0 and HalRate $$CHAIR_i(y)=\frac{|\text{[hallucinated objects]}_y|}{|\text{[all mentioned objects]}_y|}.$$1, while training with filtering yields AMBER $$CHAIR_i(y)=\frac{|\text{[hallucinated objects]}_y|}{|\text{[all mentioned objects]}_y|}.$$2 and HalRate $$CHAIR_i(y)=\frac{|\text{[hallucinated objects]}_y|}{|\text{[all mentioned objects]}_y|}.$$3. The method’s simplicity therefore depends in part on discarding ambiguous pairs rather than attempting to learn from them (Compagnoni et al., 27 Aug 2025).

6. Terminological scope and relation to adjacent work

The acronym CHAIR in CHAIR-DPO refers to the hallucination metric, not to physical chairs or to graph-theoretic chair structures. This distinction matters because arXiv includes unrelated “chair” research. The motion-synthesis paper “COUCH: Towards Controllable Human-Chair Interactions” studies contact-conditioned human-chair interaction generation and explicitly does not mention DPO; its subject is controllable 3D motion synthesis rather than multimodal preference optimization (Zhang et al., 2022). Likewise, “Vertex-critical $$CHAIR_i(y)=\frac{|\text{[hallucinated objects]}_y|}{|\text{[all mentioned objects]}_y|}.$$4-free and $$CHAIR_i(y)=\frac{|\text{[hallucinated objects]}_y|}{|\text{[all mentioned objects]}_y|}.$$5-free graphs” uses “chair” as the name of a forbidden induced graph and is unrelated to MLLM hallucination mitigation (Jooken, 27 May 2026).

Within DPO-based alignment research, a closer conceptual neighbor is Cat-DPO, which reformulates safety alignment as a per-category constrained optimization problem and introduces a separate adaptive safety margin for each harm category (Yang et al., 19 Apr 2026). CHAIR-DPO differs in both target and mechanism: it addresses object hallucination rather than harmfulness categories, and it uses CHAIR-derived winner-loser pairs with unmodified DPO instead of adaptive per-category margins (Compagnoni et al., 27 Aug 2025). A plausible implication is that CHAIR-DPO occupies the “object-grounded preference construction” end of the design space, whereas methods such as Cat-DPO occupy a “category-adaptive safety margin” end.

The practical takeaway is correspondingly narrow and precise. CHAIR-DPO is best understood as DPO with CHAIR-derived preferences: sample two answers to the same image and prompt, prefer the one with lower object hallucination rate, fine-tune with DPO, and accept a measurable trade-off with Coverage in exchange for substantial hallucination reduction (Compagnoni et al., 27 Aug 2025). The paper reports that source code and trained models are publicly available at https://github.com/aimagelab/CHAIR-DPO (Compagnoni et al., 27 Aug 2025).