---
title: In-Distribution Finetuning (IDFT)
url: https://www.emergentmind.com/topics/in-distribution-finetuning-idft
type: topic
---

# In-Distribution Finetuning (IDFT)

Searching arXiv for the cited papers to ground the article in current records.
In-Distribution Finetuning (IDFT) denotes a family of finetuning strategies that attempt to align optimization with an explicitly defined “in-distribution” reference rather than treating all training tokens, samples, or domains as equally reliable. In the most explicit formulation, introduced for large language model supervised fine-tuning, IDFT is a loss-level method derived from Distribution Discriminant Theory (DDT): it down-weights tokens that are unlikely under the model’s own next-token distribution and thereby seeks to improve generalization while preserving pre-trained structure [2602.12222]. In adjacent literatures, closely related ideas appear as target-distribution matching, feature-preserving finetuning, data reweighting, and quality-aligned adaptation, all motivated by the same empirical observation: naive full fine-tuning often improves in-distribution accuracy while degrading robustness or out-of-distribution behavior [2202.10054].

## 1. Terminological scope and motivating problem

The literature uses the label “in-distribution” in more than one sense. In standard supervised fine-tuning (SFT) for LLMs, the relevant distribution is the model-induced next-token distribution \(p_\theta(\cdot \mid c_t)\); IDFT then means suppressing gradients from tokens that behave as out-of-distribution relative to that distribution [2602.12222]. In transfer learning for vision, the relevant distribution is typically the target-domain data distribution \(P_{in}\), and the problem is to adapt a pre-trained feature extractor \(f_{\theta_0}\) and head \(g_w\) while preserving the generalization of the pre-trained representation [2308.07728]. In data-centric settings, the distributional question is whether the finetuning corpus matches the intended deployment distribution, whether defined by a small representative query set, by image-quality ratios, or by an inverse-density reweighting scheme [2410.11303].

The common motivation is that full fine-tuning can distort pre-trained features. One influential empirical study compared linear probing (LP), full fine-tuning (FT), and linear probing followed by full fine-tuning (LP-FT) across 10 distribution shift datasets and reported that fine-tuning obtained on average \(2\%\) higher accuracy ID but \(7\%\) lower accuracy OOD than linear probing; LP-FT was reported as \(1\%\) better ID and \(10\%\) better OOD than full fine-tuning [2202.10054]. This establishes the central tension that IDFT addresses across domains: optimization should adapt the model to the target task without overwriting the structures that support transfer and robustness.

## 2. Distribution Discriminant Theory and the canonical IDFT objective

The DDT formulation begins from the standard SFT loss
\[
\mathcal{L}_{\rm SFT}(\theta)
\;=\;
-\frac{1}{L}\sum_{t=1}^L \log p_\theta(x_t\mid c_t),
\]
with \(c_t=(Q,x_{<t})\), vocabulary \(\mathcal V\), and next-token distribution
\[
p_t(v)=p_\theta(v\mid c_t),\quad v\in\mathcal V.
\]
The paper contrasts this with an external data distribution \(q_t(v)\ne p_t(v)\) and frames “is this token in-distribution or out-of-distribution?” as the hypothesis test
\[
\mathcal H_0\colon x_t\sim p_t,\qquad
\mathcal H_1\colon x_t\sim q_t.
\]
The statistic is optimized through a signal-to-noise ratio
\[
\mathrm{SNR}[S]
=
\frac{\bigl(\mathbb{E}[S\mid\mathcal H_1]-\mathbb{E}[S\mid\mathcal H_0]\bigr)^2}
{\mathrm{Var}(S\mid\mathcal H_0)}.
\]
Within statistics of the form \(\log p_t(x_t)+\mathcal C[p_t]\), Theorem 3.1 identifies the Centralized Log-Likelihood
\[
\varphi_t(x_t)=\log p_t(x_t)+H[p_t]
\]
as the SNR-maximizing choice, where
\[
H[p_t]=-\sum_{v\in\mathcal V}p_t(v)\log p_t(v)
\]
is the Shannon entropy [2602.12222].

The sequential properties of \(\varphi_t\) are central to the method. With \(S_k=\sum_{t=1}^k\varphi_t\), the cumulative statistic is a martingale under \(\mathcal H_0\),
\[
\mathbb{E}[S_k\mid S_{k-1},\mathcal H_0]=S_{k-1},
\]
while under \(\mathcal H_1\) it has negative drift,
\[
\mathbb{E}[\varphi_t\mid\mathcal H_1]=-\mathrm{KL}(q_t\|p_t)<0.
\]
A Freedman-inequality concentration bound is also given:
\[
\Pr\bigl(S_L\le -\gamma\mid\mathcal H_0\bigr)
\le
\exp\!\Bigl(-\tfrac{\gamma^2}{2(V_L + c\gamma/3)}\Bigr),
\]
where \(V_L=\sum_{t=1}^L\mathrm{Var}(\varphi_t\mid\mathcal H_0)\) and \(|\varphi_t|\le c\). The practical implication is that strongly negative \(\varphi_t\) values signal tokens that are misaligned with the model-induced distribution.

IDFT turns this statistic into an adaptive multiplier
\[
\gamma_t=\exp(-\varphi_t)
\]
and defines the loss
\[
\mathcal{L}_{\rm IDFT}(\theta)
=
-\frac{1}{L}\sum_{t=1}^L
p_t(x_t)^{\,\gamma_t}\log p_t(x_t)
=
-\frac1L\sum_{t=1}^L w_t\log p_t(x_t),
\quad
w_t=p_t(x_t)^{\gamma_t}.
\]
The paper states the role of the weighting explicitly: strongly OOD tokens with \(\varphi_t\ll 0\) yield \(\gamma_t\gg 1\) and \(w_t\approx 0\), while high-confidence tokens with \(\varphi_t>0\) are up-scaled. In implementation, \(\varphi_t\) is clipped, for example with \(B=10\), before applying \(\exp(-\varphi_t)\), and the resulting procedure adds only a small constant-factor overhead above SFT [2602.12222].

## 3. Empirical profile in on-policy-style supervised finetuning

The LLM experiments position IDFT as a computationally efficient alternative to offline RL-style training. The base models are Phi-4-mini-instruct (3.8B) and Qwen2.5-7B-instruct; training uses 333K problems from Numina-Math; benchmarks are Math-C \(=\{\)AIME24, AMC23, OlympiadBench\(\}\), Math-G \(=\{\)College-math, Math-OAI, Minerva-math\(\}\), and General-R \(=\{\)MMLU-stem, ARC-challenge\(\}\). Baselines include SFT, DFT, EAFT, and offline RL methods such as Rej@2, Rej@4, DPO, SimPO, RPO, and CPO [2602.12222].

The reported aggregate effect is that IDFT yields \(5\text{–}10\) pp relative gain over SFT and outperforms DFT and EAFT by \(3\text{–}7\) pp, thereby closing most of the gap to RL methods. A table excerpt for Phi4-mini gives the following values under the stated FLOPs budgets: Origin \(16.5\%\) / \(38.1\%\) / \(59.2\%\) on Math-C / Math-G / General-R; SFT \(14.4\%\) / \(2.8\%\) / \(50.1\%\); DPO \(17.4\%\) / \(35.7\%\) / \(59.3\%\); SimPO \(24.6\%\) / \(41.6\%\) / \(59.3\%\); HD+SFT \(17.8\%\) / \(37.5\%\) / \(58.7\%\); and HD+IDFT \(19.8\%\) / \(41.9\%\) / \(59.4\%\). Here HD denotes Hinted Decoding, the companion data-level technique that re-aligns the training corpus to the model’s distribution [2602.12222].

The ablation evidence is also structurally important. Replacing the soft modulation \(\gamma_t=\exp(-\varphi_t)\) with a hard mask \(\mathbb{I}[\varphi_t>\tau]\) produces an inverted-U curve as \(\tau\) varies, which the paper uses to argue that a continuous, self-tuning gate is superior to fixed truncation. This places IDFT in a broader class of adaptive-risk objectives: rather than rejecting entire examples, it modulates token contributions continuously according to model-relative in-distribution evidence.

## 4. Feature distortion, head alignment, and representation-preserving finetuning

A closely related line of work studies in-distribution adaptation through the lens of representation drift. In an overparameterized two-layer linear network
\[
f_{v,B}(x)=v^\top(Bx),\quad v\in\mathbb R^k,\;B\in\mathbb R^{k\times d},
\]
full fine-tuning updates the head \(v\) and feature extractor \(B\) simultaneously. The analysis shows that when the head is initialized poorly, feature updates occur along directions spanned by the in-distribution data, leaving orthogonal directions almost unchanged; the resulting distorted features fit the training distribution but can perform poorly OOD. The same study reports that on 10 distribution shift benchmarks FT achieved \(85.1\%\) ID and \(59.3\%\) OOD accuracy on average, LP achieved \(82.9\%\) ID and \(66.2\%\) OOD, and LP-FT achieved \(85.7\%\) ID and \(68.9\%\) OOD [2202.10054].

Domain-Aware Fine-Tuning (DAFT) operationalizes this concern in a target-domain setting with pre-trained feature extractor \(f_\theta:X\to\mathbb R^d\), head \(g_w:\mathbb R^d\to Y\), and objective
\[
L(\theta,w)=\mathbb E_{(x,y)\sim D_{in}}[\ell(g_w(f_\theta(x)),y)].
\]
Its contribution is twofold: a batch-normalization conversion that replaces source-domain running statistics \((M_s,\Sigma_s,\gamma_s,\beta_s)\) with target-domain-adapted parameters \((M_t,\Sigma_t,\gamma_t,\beta_t)\), and an integrated LP+FT procedure with zero-initialized head and separate learning rates \(\eta_w\gg \eta_\theta\). The converted BatchNorm parameters are
\[
\gamma_t=\gamma_s\sqrt{\frac{\Sigma_t+\epsilon}{\Sigma_s+\epsilon}},
\qquad
\beta_t=\beta_s+\gamma_s\frac{M_t-M_s}{\sqrt{\Sigma_s+\epsilon}},
\]
and feature distortion is quantified by
\[
\|\Delta W\|_2/\|W\|_2=\|\hat W-W\|_2/\|W\|_2.
\]
On the MoCo-v2 CIFAR-10 series, DAFT reports \(97.84\) ID, \(94.18\) on CIFAR-10.1, and \(92.36\) on STL, compared with LP \(91.78/82.53/86.01\), FT \(97.66/92.95/81.18\), and LP-FT \(97.66/93.60/90.79\); on CIFAR-10-C mean Corruption Error, lower is better, DAFT gives \(19.4\) versus LP \(30.2\), FT \(21.7\), and LP-FT \(20.0\) [2308.07728].

FD-Align addresses the same problem from a feature-alignment perspective. It is introduced as “Feature Discrimination Alignment,” a fine-tuning approach that aims to bolster generalizability by preserving the consistency of spurious features across the fine-tuning process. The available summary states that it decouples features into category-related and category-independent information, imposes an alignment constraint that penalizes changes in label-independent feature channels or subspaces, and demonstrates efficacy for both ID and OOD tasks; once fine-tuned, the model can seamlessly integrate with existing methods, leading to performance improvements [2310.15105]. Taken together, these methods suggest that a substantial portion of what is called IDFT in transfer learning is better understood as controlled adaptation: align the head, constrain the backbone, and preserve useful pre-trained structure.

## 5. Data-centric forms of IDFT: selection, quality matching, and deployment alignment

Another line of work treats IDFT as a data selection or data composition problem. TSDS (Task-Specific Data Selection) begins from a small representative set \(Q=\{q_1,\ldots,q_M\}\) from the target task distribution and a large candidate repository \(D=\{x_1,\ldots,x_N\}\). It formulates data selection for task-specific model finetuning as an optimal-transport problem with a distribution alignment loss
\[
\min_{\gamma\ge 0}\;
\frac{\alpha}{C}\sum_{ij}\gamma_{ij}d_{ij}
+
(1-\alpha)G(\gamma)
\quad
\text{s.t.}\quad
\sum_j\gamma_{ij}=1/M\;\forall i,
\]
followed by marginalization \(p_j=\sum_i\gamma_{ij}\). To discourage near-duplicates, TSDS adds a kernel-density-based regularizer with
\[
\rho_j=\sum_{x'\in D}\max(1-f(x_j,x')^2/h^2,0),
\]
yielding a KDE-aware transport objective. The framework is evaluated for continued pretraining and instruction tuning and is reported to show that instruction tuning using data selected with a \(1\%\) selection ratio often outperforms using the full dataset and beats the baseline selection methods by \(1.5\) points in F1 score on average [2410.11303].

In medical imaging, the same distribution-matching principle appears in the relation between finetuning and deployment quality distributions. For prostate multiparametric MRI, the study defines high-quality and low-quality exams through PI-QUAL, with \(D_{ft}(r_{ft})\) denoting the finetuning distribution and \(D_{test}(r_{test})\) the test distribution, where
\[
r_{ft}=|X_{HQ}|/|X_{LQ}|,
\qquad
r_{test}=|X_{HQ}^{test}|/|X_{LQ}^{test}|,
\]
and
\[
\mathcal L_D(\theta)=\mathbb E_{(x,y)\sim D}[\ell(f_\theta(x),y)].
\]
The ideal in-distribution regime is \(r_{ft}=r_{test}\). The experiments vary \(r_{test}\in\{14{:}1,11{:}1,8{:}1,5{:}1,2{:}1\}\) and show systematic degradation under mismatch. For PI-RADS 5-class, the HQ-only model’s T1-ACC falls from approximately \(0.75\) at \(r_{test}=14{:}1\) to approximately \(0.60\) at \(2{:}1\), while the Mixed model declines more gracefully. Against training from scratch under identical conditions, HQ-only finetuning yields average ACC gains of \(+12.55\%\) and AUC gains \(+9.38\%\), Mixed finetuning yields \(+8.34\%\) ACC gain, and LQ-only yields \(-0.31\%\) ACC change. The study therefore recommends aligning quality distributions and notes that a minimum HQ proportion of approximately \(90\%\) in finetuning is advised for tasks where scratch accuracy gains are critical [2508.11864].

## 6. Theoretical extensions, misconceptions, and open questions

A broader theoretical perspective treats IDFT as gentle finetuning or rebalancing toward a more uniform domain coverage. One formulation defines
\[
Q_\gamma=\{q:\operatorname{support}(q)\subseteq\mathcal X,\;H(q)\ge H(u)-\gamma\},
\]
where \(u\) is the uniform density on \(\mathcal X\), and studies the worst-case risk over all sufficiently diverse test distributions. The main result is that training on the uniform distribution is minimax-optimal for this worst-case objective. When uniform samples are unavailable, the proposed operationalization is an IDFT objective
\[
L_{IDFT}(\theta)
=
\mathbb E_{(x,y)\sim \text{reweighted } p_{train}}
[\ell(f_\theta(x),y)],
\]
with reweighting \(w(x)\propto 1/p_{train}(x)\) and a small, decaying learning rate so that the fine-tuned model remains close to the pre-trained prior. A PAC-Bayes bound then controls the worst-case diverse-distribution risk through the divergence between posterior and prior over weights [2410.05980].

Several misconceptions are corrected by this literature. First, IDFT is not a single algorithmic recipe. Depending on context, it may denote model-distribution-aware token weighting, inverse-density reweighting, head-aligned feature-preserving adaptation, or finetune/test distribution matching. Second, “better ID accuracy” is not equivalent to “better generalization”: full fine-tuning can outperform LP in-distribution while underperforming OOD, and medical-image quality mismatch can negate the gains from pretraining altogether [2202.10054]. Third, the relevant distribution need not be the empirical training distribution; in DDT-based IDFT, the reference is the model-induced distribution itself, and in quality-aware finetuning the critical variable is the mismatch between the finetuning and deployment distributions [2602.12222].

The open questions are correspondingly heterogeneous. The LLM formulation highlights calibration sensitivity and the need to clip \(\varphi_t\). The quality-aware medical setting explicitly points to adapter-based or quality-aware sampling strategies and richer low-quality datasets. The rebalancing framework raises the problem of density estimation in high dimensions and outside-support shift. Across these strands, however, the underlying principle remains stable: finetuning is most effective when its gradients, data weights, and sample composition are made commensurate with the distribution on which the adapted model is expected to operate [2508.11864].

Source: https://www.emergentmind.com/topics/in-distribution-finetuning-idft