---
title: 'PeSFT: Perception-Aligned Supervised Fine-Tuning'
url: https://www.emergentmind.com/topics/perception-aligned-supervised-fine-tuning-pesft
type: topic
---

# PeSFT: Perception-Aligned Supervised Fine-Tuning

Perception-Aligned Supervised Fine-Tuning (PeSFT) refers to a class of supervised fine-tuning methodologies designed to explicitly address and optimize the perceptual components of multimodal and vision-language models. These approaches aim to balance the learning signal between perception (i.e., accurate extraction or alignment of input modality information such as image-to-text grounding, visual layout, or aesthetics) and downstream tasks such as reasoning, planning, or generation. PeSFT seeks to overcome systematic imbalances in standard supervised fine-tuning regimes that cause perception modules to be under-optimized, resulting in bottlenecks for overall system performance, especially in end-to-end visual reasoning, text-to-image alignment, and complex multimodal workflows [2605.29496][2605.20640][2511.05553].

## 1. Motivation and Problem Formulation

A recurring empirical finding across recent large-scale vision-language and multimodal generation models is an asymmetry in post-training improvements: while supervised fine-tuning (SFT) or post-training significantly enhances reasoning capabilities, perception components (e.g., extracting or aligning low-level scene information or attributes) show limited gains. This discrepancy constrains overall task performance, notably for tasks requiring tightly coupled perception and reasoning or photorealistic, semantically aligned generation. 

In the canonical chain-of-thought (CoT) framework for VLMs, the output sequence can be decomposed into two contiguous segments: a perception segment $p$ (grounded in visual/textual input) and a reasoning segment $r$ (goal-directed or logical deduction). Let $\pi_\theta$ be the trained auto-regressive model, $x$ the multimodal input, and $y=(p, r)$ the CoT sequence. Measured metrics commonly include:
- **Perception Accuracy**: $a_p = \mathbb{1}[\hat{p} = p^*]$
- **Reasoning Accuracy**: $a_r = \mathrm{Acc}(r' | p^*)$ (evaluated by sampling reasoning conditioned on oracle perception)
- **End-to-end Accuracy**: $a = \mathrm{Acc}(\hat{r} | p^*)$

Observed during conventional SFT: $a_r$ increases rapidly with training, while $a_p$ remains near baseline, choking end-to-end accuracy [2605.29496]. This is not confined to VLMs, but also affects text-to-image generators and embodied planning models [2605.20640][2511.05553].

## 2. Root Causes: Token Imbalance and Feature-Level Misalignment

### Token Imbalance in Chain-of-Thought Supervision

Standard SFT computes cross-entropy over the full output sequence $y$:
$$
L_{\mathrm{SFT}}(\theta) = - \mathbb{E}_{(x, y)} \left[\frac{1}{|y|} \sum_{t=1}^{|y|} \log \pi_\theta(y_t | x, y_{<t})\right]
$$
When parsing $y=(p,r)$, the perception segment $p$ typically constitutes only 2–3% of tokens, so its contribution to the total loss and gradient is disproportionately small. This starves perception of optimization signal, especially as $|p| \ll |r|$. Chain-of-thought SFT thus inherently favors reasoning at the expense of perception [2605.29496].

### Inadequate Feature-Level Alignment in Generative Models

For text-to-image diffusion models (MM-DiT, etc.), naïve SFT on curated datasets may increase photorealism but at the expense of text-image alignment or aesthetic controllability. This reflects a lack of dense, semantically grounded guidance at the feature level—pixel/latent-level targets risk overfitting and prior collapse, failing to resolve the trilemma between alignment, realism, and aesthetics [2605.20640].

*This suggests that both gradient flow (token imbalance) and supervision granularity (missing vision-aligned feature guidance) are central to perception collapse in most current SFT pipelines.*

## 3. Core Methodologies in PeSFT

### Dynamic Loss Reweighting for Perception and Reasoning

A central innovation is the explicit rebalancing of loss contributions from perception and reasoning sub-tasks. Several variants are documented:

- **Fixed Reweighting**: Introducing a static hyperparameter $\lambda$ to upweight the normalized perception loss:
  $$
  L_{\mathrm{SFT},\lambda} = \lambda\cdot (L_p/|p|) + (1-\lambda)\cdot (L_r/|r|)
  $$
  where $L_p$, $L_r$ are total negative log-likelihoods over each segment.

- **Dynamic Gradient-Norm Balancing (NGDiff)**: At each mini-batch, compute the $L_2$ norm of the perception- and reasoning-segment gradients, and set
  $$
  \lambda = \frac{1 / \|g_p\|_2}{1 / \|g_p\|_2 + 1 / \|g_r\|_2}
  $$
  yielding an adaptive, tuning-free weighting that equalizes gradient impact and restores perception learning [2605.29496].

### Feature-Space Alignment in Diffusion Models

For text-to-image diffusion transformers, PeSFT integrates a lightweight supervision path using frozen vision foundation models (e.g., SigLIP 2):
- The text prompt is encoded into multi-granularity features (global semantics, patch-level correspondences, structural/aesthetics).
- A single shallow image-feature layer is projected to the text feature space via an MLP, and a cosine loss is computed:
  $$
  L_{\mathrm{align}} = \mathbb{E}_{t, z_1, c}[1 - \cos(G_{\mathrm{proj}}(H_{\mathrm{img}}^n(z_t, t, c)), F_{\mathrm{tip}}(c))]
  $$
  The total training loss is:
  $$
  L_{\mathrm{total}} = L_{\mathrm{rec}} + \lambda_{\mathrm{align}} L_{\mathrm{align}}
  $$
  By restricting supervision to a shallow layer and using low $\lambda_{\mathrm{align}}$, the generator preserves prior diversity and avoids overfitting [2605.20640].

### Reinforced Supervised Fine-Tuning in Embodied Agents

Perception-aligned objectives in sequential multimodal planning add a reinforcement signal that rewards spatial alignment between generated images and predicted actions:
- $L(\theta) = \alpha L_{\mathrm{sup}} + \beta L_{\mathrm{reinforce}}$
- $L_{\mathrm{reinforce}}$ is a policy-gradient loss, with the reward function $r(\tau)$ quantifying the match in dynamic regions (IoU and MSE) between ground-truth and sampled images [2511.05553].

## 4. Algorithmic Implementations

Table: Overview of Representative PeSFT Algorithms

| Context                | Loss Balancing Method       | Implementation Feature   |
|------------------------|----------------------------|-------------------------|
| Chain-of-thought SFT   | Dynamic gradient-norm (NGDiff) | Adaptive $\lambda$ schedules based on instantaneous gradient norms [2605.29496] |
| Diffusion transformers | Feature-space alignment     | Cross-modal cosine loss at shallow layer using SigLIP 2 features [2605.20640] |
| Multimodal planning    | Reinforced SFT             | Hybrid supervised + RL loss, reward for pixel-space perceptual alignment [2511.05553] |

In all cases, inference-time costs remain unchanged compared to standard SFT; PeSFT modules are only active during training.

## 5. Experimental Evidence and Quantitative Results

### Vision-Language Reasoning (Synthetic Tasks)

On synthetic tasks (Graph Coloring, Sudoku), PeSFT (gradient-norm dynamic reweighting) yields major end-to-end accuracy improvements for both Qwen3-VL-2B-Instruct and InternVL3.5-2B, with gains up to 18.2 percentage points compared to standard SFT. Under PeSFT, perception accuracy ($a_p$) rises from <5% to 25–30%, with minimal trade-off to reasoning accuracy. The tradeoff curve of perception vs. reasoning (by varying $\lambda$) is Pareto-like; moderate perception upweighting optimizes overall performance [2605.29496].

| Task    | Model    | SFT | +Loss RW | +NGDiff (PeSFT) |
|---------|----------|-----|----------|-----------------|
| GC      | Qwen     | 9.8 | 21.0     | 25.0            |
| GC      | InternVL |10.6 | 20.6     | 20.6            |
| Sudoku  | Qwen     | 6.4 | 18.4     | 22.0            |
| Sudoku  | InternVL | 0.8 | 14.6     |19.0             |

### Text-to-Image Generation

PeSFT, when applied to MM-DiT diffusion backbones on portrait generation, outperforms both the baseline and conventional SFT across photorealism (FID), text-image alignment (CLIPScore/HPS), and aesthetics (ImageReward), with zero inference overhead and preservation of generative diversity. The PeSFT method unifies improvement: FID = 27.40 vs. SFT FID = 27.94, ImageReward = 21.57% vs. SFT 18.60%. The resulting Pareto frontier is strictly improved across all axes [2605.20640].

### Embodied Multimodal Planning

Reinforced SFT (a PeSFT instantiation for planners) enhances spatio-aligned planning: Success Rate on Meeting Preparation increases from 62.2% (SFT) to 67.6% (RSFT). Notably, RL-only training collapses, whereas PeSFT hybridization consistently enhances perceptual alignment in image-based subgoals [2511.05553].

## 6. Practical Recommendations and Limitations

- Chains-of-thought that minimally allocate tokens to perception require explicit reweighting—either fixed or, preferably, dynamic (NGDiff)—to mitigate perception collapse [2605.29496].
- Feature-level alignment for generative models should be injected at shallow layers and with low weights to avoid overfitting and preserve open-domain generalization [2605.20640].
- Reinforcement-based perception alignment terms must be paired with maximum-likelihood SFT for sample efficiency and robust convergence [2511.05553].
- Inference performance and latency are unaffected by PeSFT, as all supervision heads and branches can be ablated post-training.
- PeSFT applicability is general, requiring only the ability to segment out perception vs. reasoning or to couple vision-aligned features at train time.

Limitations include the need for explicit Perception/Reasoning decomposition or access to high-quality feature extractors, challenges in highly complex or compositional generation scenarios, and, for RL-based PeSFT, the possible need for credible reward proxies. Future work is suggested in expanding feature-guided PeSFT to U-Net and latent score backbones, and in developing lightweight explicit heads for more granular aesthetic or spatial supervision [2605.20640].

## 7. Connections and Implications for Future Research

PeSFT provides a principled, pragmatic framework for directly addressing long-standing perception-reasoning tradeoffs in multimodal supervised fine-tuning. The formal diagnosis of the token imbalance and feature-level insufficient supervision mechanisms enables broad application across reasoning-centric VLMs, text-conditioned image synthesis, and embodied agents. 

A plausible implication is that future vision-language architectures may co-design data annotation, output format, and training objectives to maximize token efficiency for perception, or further leverage frozen, richly trained foundation models as cross-modal signal sources. The PeSFT paradigm is likely to serve as a baseline for post-training interventions in any system where perception and higher-order cognitive tasks must be co-optimized with minimal training complexity and maximum practical compatibility [2605.29496][2605.20640][2511.05553].

Source: https://www.emergentmind.com/topics/perception-aligned-supervised-fine-tuning-pesft