---
title: 'TI-DPO: Token-Importance Guided DPO'
url: https://www.emergentmind.com/topics/token-importance-guided-dpo-ti-dpo
type: topic
---

# TI-DPO: Token-Importance Guided DPO

Token-Importance Guided Direct Preference Optimization (TI-DPO) is a class of methods designed to enhance preference alignment for large language models (LLMs) by integrating token-level discriminative weighting into the standard Direct Preference Optimization (DPO) paradigm. Unlike conventional DPO, which uniformly aggregates the log-likelihood ratio over all tokens in a sequence, TI-DPO leverages fine-grained token-level signals to emphasize critical tokens and adaptively modulate gradient updates. These approaches target the shortcomings of sequence-level preference modeling, such as reward dilution, length bias, and weak discrimination at salient tokens, by focusing learning on the tokens most impactful for preference outcomes.

## 1. Theoretical Foundations and Motivation

Traditional DPO optimizes a policy $\pi_\theta$ to maximize the likelihood of preferred complet

Source: https://www.emergentmind.com/topics/token-importance-guided-dpo-ti-dpo