---
title: 'CHAIR-DPO: Fine-Tuning to Reduce Object Hallucinations'
url: https://www.emergentmind.com/topics/chair-dpo
type: topic
---

# CHAIR-DPO: Fine-Tuning to Reduce Object Hallucinations

CHAIR-DPO is a preference-construction and fine-tuning method for reducing **object hallucinations** in multimodal large language models (MLLMs). It combines the **CHAIR** hallucination metric with **Direct Preference Optimization (DPO)**: for a fixed image \(x_I\) and textual prompt \(x_T\), it samples two answers, scores both with \(CHAIR_i\), labels the less hallucinatory answer as the winner \(y_w\) and the more hallucinatory one as the loser \(y_l\), and then fine-tunes the MLLM with a standard DPO objective so that the model assigns higher relative probability to \(y_w\) than to \(y_l\) [2508.20181]. In the paper’s framing, the method addresses hallucination as an **alignment problem** rather than as a decoding-only or post hoc correction problem: the optimization signal is made explicitly **object-aware** by depending on whether generated object mentions are supported by the visual input.

## 1. Problem setting and definition

The target failure mode is **object hallucination**: the model mentions objects that are not actually present in the image. The paper treats this as especially problematic in visually grounded generation because the output can remain fluent and plausible while being factually ungrounded. Examples in the paper include descriptions that mention a chair, bench, person, or vehicle that does not appear in the image [2508.20181].

The method’s central premise is that hallucination mitigation can be cast as pairwise preference learning. Given two responses to the same multimodal input, the preferred response is the one with fewer hallucinated objects. This yields a specific interpretation of “object-aware”: the preference signal depends on **object presence or absence** in the image, not on explicit region grounding, token-level grounding, or a general-purpose learned reward model [2508.20181].

Formally, for two sampled responses \(y_1,y_2 \sim ref(y \mid x_T,x_I)\), the winner and loser are obtained by ranking them with the instance-level CHAIR score:
$$
CHAIR_i(y)=\frac{|\text{[hallucinated objects]}_y|}{|\text{[all mentioned objects]}_y|}.
$$
The paper states the intended selection rule as
$$
y_w=\arg\min_{y\in\{y_1,y_2\}} CHAIR_i(y), \qquad
y_l=\arg\max_{y\in\{y_1,y_2\}} CHAIR_i(y).
$$

The scope is broader than captioning. Preference data are built from **LLaVA-Instruct-665k**, which contains multi-turn visual dialogues, and candidate completions are conditioned on both the image and a textual dialogue context. This suggests that CHAIR-DPO is meant for image-grounded instruction following and dialogue whenever object mention accuracy matters, not only for caption generation [2508.20181].

## 2. CHAIR-based preference construction

The paper’s main novelty is not a new preference loss but a simple, automatic pipeline for producing DPO pairs from hallucination scores. The input source is **LLaVA-Instruct-665k**, viewed as triplets \(\{x_T,x_I,y\}\), where the original target answer \(y\) is discarded for preference construction [2508.20181].

For multi-turn dialogues, the conversation is **randomly truncated**: the model receives the first \(k\) human-assistant turns plus the current question as textual context, where \(k\) is randomly chosen. Two candidate answers are then sampled from the reference MLLM with **temperature \(=0.7\)**. The paper does not report other decoding parameters such as top-\(p\), top-\(k\), or maximum generation length [2508.20181].

Object supervision on the image side is derived automatically with **DETR-DC5-R101**. Its predicted class labels are treated as the set of objects present in the image, with vocabulary based on **MSCOCO class names**. On the text side, object mentions are identified using MSCOCO object class names together with the **synonym list from the reference CHAIR implementation**. A mentioned object is counted as hallucinated if it is not in the detected object set [2508.20181].

The paper distinguishes two CHAIR variants. **\(CHAIR_i\)** is the instance-level fraction of mentioned object instances that are hallucinated, with object mentions treated as a **list**, not a set, so repeated hallucinations can be penalized multiple times. **\(CHAIR_s\)** is the percentage of responses containing at least one hallucinated object; in AMBER evaluation, **HalRate** corresponds to \(CHAIR_s\) [2508.20181].

A key implementation choice is to **discard all pairs with zero \(CHAIR_i\) difference**. The paper reports that these pairs produce noisy supervision because they do not contain a meaningful preference signal. After filtering, the final preference datasets contain **70k** training samples for **LLaVA-1.5-7B** and **77k** training samples for **LLaVA-MORE-8B**, with **500 held-out samples** for validation in both cases [2508.20181].

This filtering is not incidental. The paper explicitly states that it yields a more reliable optimization signal and significantly speeds up fine-tuning by eliminating nearly **90%** of the dataset [2508.20181].

## 3. DPO objective and training protocol

Once winner-loser pairs are constructed, CHAIR-DPO uses a standard DPO loss. The model generates continuation text \(y\) conditioned on prompt \(x_T\) and image \(x_I\):
$$
P(y\mid x_T,x_I).
$$
The DPO objective is
$$
\mathcal{L}_{\mathrm{DPO}}(\pi_\theta;\pi_{\mathrm{ref}})
=
-\mathbb{E}_{(x_T,x_I,y_w,y_l)\sim\mathcal D}
\left[
\log \sigma\left(
\beta \log \frac{\pi_\theta(y_w\mid x_T,x_I)}{\pi_{\mathrm{ref}}(y_w\mid x_T,x_I)}
-
\beta \log \frac{\pi_\theta(y_l\mid x_T,x_I)}{\pi_{\mathrm{ref}}(y_l\mid x_T,x_I)}
\right)
\right].
$$
Here \(\pi_\theta\) is the trainable policy model, \(\pi_{\mathrm{ref}}\) is the frozen reference model, and \(\beta\) controls the strength of the implicit KL regularization toward the reference policy [2508.20181].

The paper is explicit that **there is no change to the DPO loss itself**. The novelty lies in two places: first, constructing preference labels from \(CHAIR_i\); second, filtering out zero-difference pairs. It also notes that CHAIR-DPO could be applied iteratively by updating the reference with the current policy and regenerating preference data, but no iterative experiments are reported [2508.20181].

Two open-source MLLMs are fine-tuned:

- **LLaVA-1.5-7B**, based on **Vicuna-7B**
- **LLaVA-MORE-8B**, based on **LLaMA-3.1-8B**

Both follow the LLaVA-style pipeline with **CLIP-ViT-L/14@336** as visual encoder and a lightweight vision-language adapter [2508.20181].

Fine-tuning is performed with **LoRA**, using **rank \(r=128\)** and **\(\alpha=256\)**. The optimizer is **Adam** with a **cosine learning rate scheduler** and peak learning rate \(2\times 10^{-6}\). Warmup uses **33** steps for LLaVA-1.5-7B and **145** steps for LLaVA-MORE-8B. Training runs on **2 nodes** with **8 NVIDIA A100 64GB GPUs each**, for a total of **16 A100 64GB GPUs**, with **DeepSpeed ZeRO Stage 2** and gradient checkpointing. The total batch size is **64** for LLaVA-1.5-7B and **16** for LLaVA-MORE-8B. The paper evaluates \(\beta=0.5\), \(\beta=0.3\), and \(\beta=0.2\) [2508.20181].

The paper also states what CHAIR-DPO does **not** require: no human preference annotations and no proprietary teacher or judge models such as GPT-4 or Gemini [2508.20181].

## 4. Evaluation and empirical behavior

CHAIR-DPO is evaluated on three hallucination benchmarks: **AMBER**, **CHAIR-MSCOCO**, and **Object HalBench**. AMBER uses **1,004 manually annotated images** and reports macro-averaged \(CHAIR_i\), **HalRate**, **Coverage**, and **Cognition (Cog)**. CHAIR-MSCOCO uses **500 images from MSCOCO val** and reports \(CHAIR_s\) and \(CHAIR_i\). Object HalBench uses a **300-image MSCOCO-val subset** and reports micro-averaged \(CHAIR_s\) and \(CHAIR_i\) [2508.20181].

| Benchmark | Size | Metrics |
|---|---:|---|
| AMBER | 1,004 images | \(CHAIR_i\), HalRate, Coverage, Cog |
| CHAIR-MSCOCO | 500 images | \(CHAIR_s\), \(CHAIR_i\) |
| Object HalBench | 300 images | \(CHAIR_s\), \(CHAIR_i\) |

For **LLaVA-1.5-7B**, the baseline reports **AMBER \(CHAIR_i=7.6\)**, **Cover \(=51.7\)**, **HalRate \(=35.0\)**, and **Cog \(=4.2\)**. With CHAIR-DPO at **\(\beta=0.2\)**, these become **\(CHAIR_i=3.0\)**, **Cover \(=46.6\)**, **HalRate \(=14.7\)**, and **Cog \(=1.3\)**. On CHAIR-MSCOCO, baseline **\(CHAIR_s=50.6\)** and **\(CHAIR_i=13.9\)** drop to **\(CHAIR_s=14.4\)** and **\(CHAIR_i=3.6\)**. On Object HalBench, CHAIR-DPO at \(\beta=0.2\) reaches **\(CHAIR_s=8.6\)** and **\(CHAIR_i=4.6\)** [2508.20181].

For **LLaVA-MORE-8B**, the baseline reports **AMBER \(CHAIR_i=8.1\)**, **Cover \(=53.2\)**, **HalRate \(=38.4\)**, and **Cog \(=4.0\)**. With CHAIR-DPO at **\(\beta=0.2\)**, these become **\(CHAIR_i=2.6\)**, **Cover \(=49.7\)**, **HalRate \(=14.2\)**, and **Cog \(=1.0\)**. On CHAIR-MSCOCO, baseline **\(CHAIR_s=51.2\)** and **\(CHAIR_i=14.4\)** drop to **\(CHAIR_s=11.8\)** and **\(CHAIR_i=3.1\)**. On Object HalBench, the corresponding CHAIR-DPO numbers are **\(CHAIR_s=9.2\)** and **\(CHAIR_i=4.5\)** [2508.20181].

Across both backbones, the paper states that lower \(\beta\) yields lower hallucination scores but somewhat lower **Coverage**. It interprets this as a controllable trade-off rather than a collapse in capability. For general MLLM evaluation, the paper reports that CHAIR-DPO improves **MME**, slightly improves **MMMU** and **Science-QA**, and is comparable on **SEED** and **AI2D** for LLaVA-1.5-7B, while LLaVA-MORE-8B shows moderate regression on several general benchmarks without catastrophic forgetting [2508.20181].

The paper also compares CHAIR-DPO to methods including **DoLa**, **VCD**, **OPERA**, **Woodpecker**, **POVID**, **HA-DPO**, **HALVA**, **EOS**, **mDPO**, **MFPO**, and **REVERSE**. Its claim is that CHAIR-DPO is always best or second-best on direct hallucination metrics, with a stronger hallucination/coverage trade-off than methods whose lower hallucination numbers are accompanied by severe Coverage reduction [2508.20181].

## 5. Strengths, trade-offs, and limitations

The method’s principal strength is operational simplicity. It uses an existing hallucination metric, an off-the-shelf detector, open instruction-tuning data, and standard DPO. This avoids the complex synthetic preference pipelines and proprietary judge models used by several recent hallucination-alignment approaches [2508.20181].

Another strength is that the supervision signal is tightly targeted. Preference labels are induced by whether mentioned objects are visually present, so the optimization directly favors object-grounded generations. The paper argues that this makes the model more conscious of what objects are present and absent in the image [2508.20181].

The main reported trade-off is between hallucination reduction and **Coverage**. Lower \(\beta\) improves hallucination metrics but tends to reduce object recall or mention coverage. The paper nevertheless reports mostly preserved general capability for LLaVA-1.5-7B and no catastrophic forgetting for LLaVA-MORE-8B [2508.20181].

Several limitations are explicit or directly implied by the setup. CHAIR-DPO relies on object presence labels inferred by **DETR-DC5-R101**, so detector errors can inject noise into preference construction. The paper does not deeply analyze detector error, but the dependence is structurally unavoidable. The method is also limited to **object hallucinations** as defined by the CHAIR vocabulary: it does not directly address attribute hallucinations, relational hallucinations, temporal hallucinations, or broader semantic errors not tied to object vocabulary. Because CHAIR is tied to **MSCOCO class names** plus synonym mappings, hallucinations outside that vocabulary may be missed [2508.20181].

The filtering ablation further shows that CHAIR-DPO depends on high-signal preference pairs. For example, on **LLaVA-1.5-7B** with **\(\beta=0.3\)**, training **without filtering** yields **AMBER \(CHAIR_i=4.0\)** and **HalRate \(=20.2\)**, while training **with filtering** yields **AMBER \(CHAIR_i=3.2\)** and **HalRate \(=16.2\)**. The method’s simplicity therefore depends in part on discarding ambiguous pairs rather than attempting to learn from them [2508.20181].

## 6. Terminological scope and relation to adjacent work

The acronym **CHAIR** in CHAIR-DPO refers to the hallucination metric, not to physical chairs or to graph-theoretic chair structures. This distinction matters because arXiv includes unrelated “chair” research. The motion-synthesis paper **“COUCH: Towards Controllable Human-Chair Interactions”** studies contact-conditioned human-chair interaction generation and explicitly does **not** mention DPO; its subject is controllable 3D motion synthesis rather than multimodal preference optimization [2205.00541]. Likewise, **“Vertex-critical \((P_5,\text{chair})\)-free and \((P_5,\text{cricket})\)-free graphs”** uses “chair” as the name of a forbidden induced graph and is unrelated to MLLM hallucination mitigation [2605.28537].

Within DPO-based alignment research, a closer conceptual neighbor is **Cat-DPO**, which reformulates safety alignment as a per-category constrained optimization problem and introduces a separate adaptive safety margin for each harm category [2604.17299]. CHAIR-DPO differs in both target and mechanism: it addresses **object hallucination** rather than harmfulness categories, and it uses **CHAIR-derived winner-loser pairs** with unmodified DPO instead of adaptive per-category margins [2508.20181]. A plausible implication is that CHAIR-DPO occupies the “object-grounded preference construction” end of the design space, whereas methods such as Cat-DPO occupy a “category-adaptive safety margin” end.

The practical takeaway is correspondingly narrow and precise. CHAIR-DPO is best understood as **DPO with CHAIR-derived preferences**: sample two answers to the same image and prompt, prefer the one with lower object hallucination rate, fine-tune with DPO, and accept a measurable trade-off with Coverage in exchange for substantial hallucination reduction [2508.20181]. The paper reports that source code and trained models are publicly available at `https://github.com/aimagelab/CHAIR-DPO` [2508.20181].

Source: https://www.emergentmind.com/topics/chair-dpo