Token Cleaning Framework

• Token Cleaning Framework is a methodology that scores and selects individual tokens to filter out uninformative or biased data during supervised fine-tuning and inference.
• It employs fixed-model and self-evolving cleaning strategies, typically retaining 50%-70% of tokens to boost signal-to-noise ratios and improve model accuracy.
• In multimodal LLMs, techniques like SVC and CRC recalibrate vision tokens to reduce hallucinations and strengthen the interaction between text and visual context.

The Token Cleaning Framework encompasses a family of methods for fine-grained selection and manipulation of tokens during both supervised fine-tuning (SFT) of LLMs and inference in multimodal LLMs (MLLMs). The central principle is to quantify the informativeness or relevance of individual tokens, selectively filtering or recalibrating those that contribute little or introduce undesirable biases. This methodology stands in contrast to conventional coarse-grained filtering at the sample or sequence level, instead optimizing at token-level granularity to maximize signal-to-noise ratio and overall model performance (Pang et al., 4 Feb 2025, Fa et al., 11 Mar 2026).

1. Core Principles and Motivation

Token Cleaning originated from observations in LLM SFT that not all tokens—despite occurring in otherwise high-quality samples—carry meaningful supervision signals. Tokens including formatting artifacts, common phrases, and model-already-known facts may be uninformative or even detrimental, as they can dilute the task-specific signal or reinforce spurious patterns. This motivates two key objectives: (1) reliably scoring each token for its utility in gradient-driven learning, and (2) dynamically selecting which tokens contribute to parameter updates or output representations (Pang et al., 4 Feb 2025).

In the multimodal domain, vision token manipulation leverages a related philosophy. Here, the goal is not merely to filter tokens but to rebalance the latent vision–language interplay, particularly mitigating hallucinations arising from dominant text priors or weakening visual context as generation proceeds (Fa et al., 11 Mar 2026).

2. Token Influence Scoring and Selection in LLM SFT

The foundational algorithmic innovation is the token-influence score:

$$\mathrm{Score}(x_{i,j}|x_{i,:j}; \theta, \theta') = - [ \ell(x_{i,j}|x_{i,:j}; \theta') - \ell(x_{i,j}|x_{i,:j}; \theta) ]$$

where $\ell(x_{i,j}|x_{i,:j}; \theta)$ is the negative log-likelihood assigned by model $\theta$, and $\theta'$ is a reference model. This score estimates how much fine-tuning on token $x_{i,j}$ increases model confidence—a larger loss drop indicates higher informativeness. Two regimes are defined:

• Fixed-Model Cleaning: Compute influence in a single pass using a fixed, stronger reference model $\theta'$ (e.g., a larger or better fine-tuned LLM) for all tokens, filter via a global threshold (top $k\%$) and fine-tune on the retained set.
• Self-Evolving Cleaning: Iteratively partition the dataset, alternately fine-tune and update the reference model as the “teacher,” and dynamically rescore/clean each split. This enables exploitation of evolving knowledge but can accelerate both improvements and over-cleaning (the “Matthew effect”) depending on data characteristics (Pang et al., 4 Feb 2025).

The cleaned token selection is operationalized by ranking all token scores and retaining the top $k\%$, where optimal performance is observed for $k$ in $[50\%, 70\%]$. Hyperparameters include choice of reference, number of splits, batch size, learning rate (typically $\ell(x_{i,j}|x_{i,:j}; \theta)$0), and LoRA rank if using parameter-efficient fine-tuning adapters.

3. Theoretical Error Analysis and Practical Benefits

A rigorous noisy-label analysis demonstrates that the generalization error of a model trained on all tokens is bounded by the sum of a data-quality term (token label noise rate $\ell(x_{i,j}|x_{i,:j}; \theta)$1) and a data-quantity term $\ell(x_{i,j}|x_{i,:j}; \theta)$2:

$\ell(x_{i,j}|x_{i,:j}; \theta)$3

Token cleaning is provably beneficial if the noise reduction outpaces the loss in effective data size:

$\ell(x_{i,j}|x_{i,:j}; \theta)$4

where $\ell(x_{i,j}|x_{i,:j}; \theta)$5 is the post-cleaning noise rate and $\ell(x_{i,j}|x_{i,:j}; \theta)$6 is the keep-ratio. In Fixed-Model Cleaning, noise falls rapidly but cannot vanish; Self-Evolving Cleaning allows further reductions on rich splits but risks negative feedback on poor-quality splits (Pang et al., 4 Feb 2025).

Empirically, across seven benchmarks (including MMLU, TruthfulQA, TydiQA, HellaSwag), both Fixed-Model and Self-Evolving Cleaning consistently boost downstream task accuracy. For LLaMA-3.2-3B, absolute gains reach +2.1% (Fixed) and +3.1% (Self-Evolving), with similar improvements found in Mistral-7B and LLaMA-3.1-8B (Pang et al., 4 Feb 2025).

4. Token Manipulation in Multimodal LLMs: Vision Token Cleaning

In MLLMs, token-level manipulation extends beyond filtering to strategic calibration of latent representations via vision tokens:

• Synergistic Visual Calibration (SVC): Augmented views (e.g., via image flip, Gaussian blur, noise) generate complementary visual token sets. These are concatenated into a synergistic memory and injected at a specific transformer layer by interpolating between the original hidden state and the attended memory:

$\ell(x_{i,j}|x_{i,:j}; \theta)$7

where $\ell(x_{i,j}|x_{i,:j}; \theta)$8 is an interpolation hyperparameter. This re-strengthens the pure visual signal where it would otherwise diminish (Fa et al., 11 Mar 2026).

• Causal Representation Calibration (CRC): Generating in-distribution negative samples by pruning a random subset of vision tokens in latent space, the CRC mechanism computes a hallucination direction (average difference between full and pruned-token states), then subtracts this from subsequent representations to counteract text-only bias:

$\ell(x_{i,j}|x_{i,:j}; \theta)$9

with

$\theta$0

where $\theta$1 is a control parameter. This operation isolates and mitigates spurious language-confidence directions, directly addressing multimodal hallucinations (Fa et al., 11 Mar 2026).

• Unified Calibration Objective: The interventions above minimize a composite loss combining representation distortion (SVC) and bias projection (CRC), aligning with a dual formulation involving negative inner products between state and corrective directions:

$\theta$2

5. Experimental Validation Across Language and Multimodal Contexts

The Token Cleaning methodology exhibits robust improvements in both textual and multimodal domains.

LLM SFT (Token Cleaning in LLMs)

• Benchmarks: MMLU, TruthfulQA, TydiQA, HellaSwag, ARC-C, BoolQ, LogiQA
• Baseline Comparisons: Base (un-tuned), DS², Full Tokens, Uniform Random, Rho (per-sample influence ranking)
• Key Findings:
  • Fixed-Model Cleaning: Up to +2.2% absolute accuracy (e.g., 56.70→58.90 for LLaMA-3.1-8B)
  • Self-Evolving Cleaning: Up to +3.1% (e.g., 49.87→53.00 for LLaMA-3.2-3B)
  • Ablations confirm optimal $\theta$3. Random token selection and local per-sample strategies underperform (Pang et al., 4 Feb 2025).

Multimodal LLMs (Vision Token Cleaning)

• Models: LLaVA-1.5, MiniGPT-4, Shikra, InstructBLIP
• Benchmarks: POPE (object presence on COCO, AOKVQA, GQA), CHAIR hallucination rates, MMHal-Bench, MME
• Key Findings:
  • POPE accuracy increases by ∼2% (e.g., LLaVA-1.5: 84.79→86.79)
  • CHAIR$\theta$4 hallucination rates decrease sharply (e.g., 25.4%→18.1% at 64 tokens)
  • Combined SVC+CRC outperforms individual modules, with negligible compute overhead (1.06× slower than vanilla decoding) (Fa et al., 11 Mar 2026).

A summary of ablation results on LLaVA-1.5 is provided:

Method	POPE Accuracy (%)
Vanilla	84.79
SVC (original only)	85.04
SVC (synergistic)	85.55
CRC (masked-image neg)	84.77
CRC (pruned tokens)	86.11
SVC + CRC	86.79

6. Integration, Hyperparameters, and Best Practices

Token Cleaning is readily integrated as a preprocessing step in SFT workflows:

• Data Preparation: Use standard prompt-response pools.
• Warm-up: (Optional) Fine-tune the base model on a small split to obtain a competent reference.
• Strategy Selection: Fixed-Model Cleaning for simplicity; Self-Evolving Cleaning for maximal gains with larger datasets.
• Threshold Tuning: Scan $\theta$5 in $\theta$6, but $\theta$7 is typically optimal.
• Parameter Choices: Employ LoRA for efficiency, batch size 48, 1–2 epochs, sequence length up to 2048.
• Monitoring: Track validation to prevent over-cleaning, especially in noisy or imbalanced splits.
• Vision Token Cleaning: For SVC, inject tokens at layer $\theta$8; for CRC, use $\theta$9 negative samples, $\theta'$0 tokens per subset, and hyperparameters $\theta'$1, $\theta'$2 (Pang et al., 4 Feb 2025, Fa et al., 11 Mar 2026).

Practical integration into MLLMs operates entirely at inference, involving no retraining or gradient updates.

7. Significance, Implications, and Emerging Directions

Token Cleaning marks a methodological advance for fine-grained data curation and representation calibration in both supervised language and multimodal modeling. It rationalizes the treatment of training and inference signals by prioritizing those tokens that impart genuine task-specific or context-relevant information. Theoretically grounded improvements in generalization error have been substantiated by consistent empirical gains.

A plausible implication is that as model and data scale continue to increase, the marginal benefits from token-level cleaning will become even more substantial, especially in SFT settings where token-level supervision noise limits progress. In multimodal architectures, the paradigm of vision token manipulation via augmentation- and pruning-based calibration is poised to see further adoption for systematically counteracting text-dominated confounding phenomena, including hallucinations and loss of grounding (Pang et al., 4 Feb 2025, Fa et al., 11 Mar 2026).

Future extensions may involve more adaptive, causal, or learned token selection mechanisms, leveraging dynamic interaction between unimodal and multimodal representations and guided by downstream validation signals.