SoftREPA: Soft Representation Alignment

• The paper demonstrates that SoftREPA improves text–image alignment by optimizing a small set of soft tokens with an InfoNCE-like loss.
• It maintains the pretrained diffusion model by only fine-tuning fewer than 1M parameters, yielding improved metrics on models like SD1.5, SDXL, and SD3.
• SoftREPA leverages internal denoising signals to maximize mutual information between image and text representations without external reward models.

Searching arXiv for the cited SoftREPA and AGSM papers to ground the article in the latest available preprints. Searching arXiv for "SoftREPA" and "Aligning Text to Image in Diffusion Models is Easier Than You Think". SoftREPA, short for “soft REPresentation Alignment,” is a lightweight, contrastive fine-tuning method for text-to-image diffusion and flow models that improves text–image alignment by learning a small set of soft text tokens while keeping the base model frozen. Its central idea is to derive an InfoNCE-like alignment objective from the model’s own denoising or velocity prediction behavior, rather than from an external reward model or human preference signal. In the formulation introduced for Stable Diffusion 1.5, SDXL, and SD3, SoftREPA adds fewer than 1M trainable parameters, leaves the backbone unchanged, and can be interpreted as maximizing a lower bound on the mutual information between image and text representations under the conditional generative model (Lee et al., 11 Mar 2025).

1. Problem setting and conceptual motivation

SoftREPA was proposed against the background of persistent text–image misalignment in modern text-to-image systems. Even strong diffusion models such as Stable Diffusion 1.5, SDXL, and SD3 may exhibit incorrect attribute binding, wrong counts or missing objects, and poor handling of complex relational descriptions. In standard conditional diffusion or flow training, the model is optimized only on positive image–caption pairs by minimizing a denoising, score-matching, or flow-matching loss conditioned on the true caption. This teaches the model to reconstruct or denoise correctly under the correct condition, but does not explicitly teach it to deviate under an incorrect caption (Lee et al., 11 Mar 2025).

That asymmetry motivates a representation-alignment view of text conditioning. Contrastive learning methods such as CLIP explicitly pull matched pairs together and push mismatched pairs apart; SoftREPA imports that intuition into text-to-image generation, but does so using the generative model’s own internal score or velocity prediction errors rather than an external encoder. In this sense, it is related to REPA in its emphasis on representation alignment, yet differs in target and mechanism: REPA aligns internal generative representations with an external visual encoder, whereas SoftREPA aligns text and image representations directly inside a pretrained text-to-image model without introducing external encoders or updating the backbone (Lee et al., 11 Mar 2025).

A common misconception is to place SoftREPA in the same category as RLHF-style post-training. It is instead a reward-free alignment method. It uses ordinary paired image–caption data, constructs positives and negatives within each batch, and modifies only a small conditioning adapter in the form of soft tokens rather than the full denoiser or transformer.

2. Contrastive objective and score-based similarity

The core SoftREPA objective is an InfoNCE-like contrastive loss over image–text pairs. For a batch $\{(x^{(i)}, y^{(i)})\}_{i=1}^n$, the loss is

$$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$

For each image $x^{(i)}$, the matched caption $y^{(i)}$ is the positive example and the other captions in the batch act as negatives. The distinctive feature of SoftREPA is that the similarity score $l(x,y)$ is not a dot product in an embedding space, but a function of the conditional denoising or flow error of the generative model itself (Lee et al., 11 Mar 2025).

For diffusion-style denoisers, the similarity is defined as

$$l(x,y) = \exp\Big( -\mathbb{E}_{t,\epsilon}\big[ \|\epsilon_\theta(x_t, t, y) - \epsilon\|^2 / \tau(t) \big] \Big),$$

where $x_t$ is the noised latent at time $t$, $\epsilon \sim \mathcal{N}(0,I)$, and $\tau(t)$ is a time-dependent scale. For conditional flow models, the same construction is applied to the velocity field. In practice, the expectation is replaced by a single Monte Carlo sample for efficiency, and the score becomes a function of the soft-token parameters $$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$0 through the conditioned predictor. The exponential parameterization is described as making the similarity measure “bounded and well-behaved” via the exponential, avoiding unbounded negative losses that caused instability in preliminary experiments (Lee et al., 11 Mar 2025).

The optimization target is therefore to make the model’s denoising behavior strongly discriminate between correct and incorrect text conditions. For correct captions, the denoising or velocity prediction should remain close to the true target; for mismatched captions, the prediction should deviate. Because the base model parameters $$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$1 are frozen and only $$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$2 is updated, this discrimination is implemented entirely through a small learned perturbation of the conditioning pathway.

3. Soft text tokens and architectural insertion points

SoftREPA’s trainable object is a set of soft text tokens: learnable continuous embeddings that are not tied to any discrete vocabulary token and are prepended as extra tokens to the text sequence at selected layers and timesteps. In a transformer-like formulation, for layer $$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$3 and time index $$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$4, SoftREPA defines

$$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$5

where $$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$6 is the number of soft tokens and $$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$7 is the hidden dimension. Given text hidden states $$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$8, the augmented text sequence becomes

$$\mathcal{L} = -\frac{1}{n}\sum_{i=1}^n \log \frac{\exp\big(l(x^{(i)}, y^{(i)})\big)}{\sum_j \exp\big(l(x^{(i)}, y^{(j)})\big)}.$$9

After the layer update, the soft-token positions are discarded and only the original text-token positions are retained for downstream propagation (Lee et al., 11 Mar 2025).

This design makes the soft tokens layer- and time-specific control signals. They affect the attention computation and the interaction between image and text at the relevant layer, but do not persist as explicit tokens throughout the network. The effect is similar in spirit to prompt-tuning or prefix-tuning, but the parameterization is indexed by both depth and diffusion timestep and is trained under a contrastive score-matching objective rather than for concept insertion or domain adaptation (Lee et al., 11 Mar 2025).

The insertion point depends on the backbone. In SD3 and related MM-DiT architectures, image and text tokens are processed jointly in transformer layers, and the soft tokens are prepended to the text stream in the early layers. In UNet-based SD1.5 and SDXL, soft tokens are injected into the Down and Middle blocks at the level of the original embedding used for cross-attention, while the Up blocks remain unchanged. In all cases, the base text encoder is unchanged, the backbone remains frozen, and the additional parameter count remains below 1M across the reported experiments (Lee et al., 11 Mar 2025).

4. Training regime and information-theoretic interpretation

SoftREPA is a post-hoc fine-tuning method for pretrained text-to-image models. Training begins from a pretrained SD1.5, SDXL, or SD3 model; all original parameters are frozen; and only the soft-token parameters are optimized. For SD3 and SDXL, the soft tokens are initialized from a random Gaussian $x^{(i)}$0, whereas for SD1.5 random initialization was found to hurt performance, and the soft tokens were instead initialized using unconditional text embeddings. Training uses COCO captions paired with images, with AdamW, learning rate $x^{(i)}$1, weight decay $x^{(i)}$2, batch size 16, cosine annealing, and roughly 26–30k iterations; the method can be trained in fewer than 30k steps on 2×A100 GPUs (Lee et al., 11 Mar 2025).

The training loss differs slightly across backbones. In SD3, only the SoftREPA contrastive score-matching loss is used. In SD1.5 and SDXL, the objective is augmented with a small denoising loss,

$x^{(i)}$3

to keep the behavior close to the original denoiser while still improving contrastive alignment (Lee et al., 11 Mar 2025).

At inference time, the generative process is otherwise unchanged. The same sampler, classifier-free guidance, and number of steps are used, and the only modification is the layer-wise and time-wise injection of the learned soft tokens into the conditioning stream. No external reward model is introduced at sampling time, and no additional optimization is performed during generation (Lee et al., 11 Mar 2025).

A defining part of the SoftREPA argument is its mutual-information interpretation. Using results that connect conditional diffusion likelihood to the denoising MSE along the diffusion trajectory, the conditional likelihood $x^{(i)}$4 can be written as an exponential of a negative integrated denoising loss. Replacing the marginal denominator in pointwise mutual information with a Monte Carlo approximation over negative captions yields an objective of the same form as the SoftREPA contrastive loss. The paper therefore states that minimizing the SoftREPA objective is equivalent to maximizing a lower bound on the mutual information between images and texts under the conditional diffusion model, and interprets the resulting gain in semantic consistency as an effect of increasing that dependence (Lee et al., 11 Mar 2025).

5. Empirical performance in generation and editing

SoftREPA was evaluated on text-to-image generation for Stable Diffusion 1.5, SDXL, and SD3 using COCO prompts, with ImageReward, PickScore, CLIP score, HPS-v2, LPIPS, and FID as reported metrics. On SD1.5, ImageReward improved from 0.177 to 0.274, CLIP from 0.265 to 0.271, HPS from 0.251 to 0.252, and LPIPS from 0.438 to 0.437. On SDXL, ImageReward improved from 0.750 to 0.852, CLIP from 0.267 to 0.268, HPS from 0.273 to 0.283, and LPIPS changed from 0.420 to 0.423. On SD3, ImageReward improved from 0.942 to 1.085, CLIP from 0.263 to 0.269, HPS from 0.280 to 0.289, and LPIPS from 0.424 to 0.428. The qualitative characterization accompanying these results is that prompts with detailed compositions show more accurate object placements and attributes, more faithful handling of complex descriptions, and comparable global style and realism relative to the base models (Lee et al., 11 Mar 2025).

SoftREPA was also applied to text-guided image editing by replacing the conditioning in FlowEdit while leaving the inversion or editing procedure unchanged. On DIV2K real images, vanilla FlowEdit at target CFG 13.5 and 33 edit steps yielded ImageReward 0.380, CLIP 0.260, HPS 0.255, and LPIPS 0.154, whereas FlowEdit plus SoftREPA soft tokens at CFG 11 and 30 edit steps yielded ImageReward 0.466, CLIP 0.263, HPS 0.260, and LPIPS 0.149. On Cat2Dog, the FlowEdit baseline yielded ImageReward 0.937, CLIP 0.225, HPS 0.266, and LPIPS 0.199, while FlowEdit plus SoftREPA yielded ImageReward 1.111, CLIP 0.226, HPS 0.269, and LPIPS 0.173. The reported interpretation is that SoftREPA improves alignment and human preference proxies, slightly improves perceptual similarity, and requires lower CFG and fewer editing steps, suggesting more efficient and stable conditioning (Lee et al., 11 Mar 2025).

Ablation studies clarify where SoftREPA is most effective. Soft tokens in very early layers, specifically layers 2–5, yield the best ImageReward while maintaining image quality. Extending soft tokens beyond layer 7 significantly degrades generation quality even though CLIP score continues to increase, which was interpreted as evidence that early layers are more associated with global semantic alignment while later layers are more involved in fidelity refinement. Token length also exhibits a trade-off: 1–4 tokens preserve perceptual similarity, 4–8 tokens provide the best trade-off between ImageReward and CLIP with stable LPIPS, and 16–32 tokens lead to degradation, plausibly through overfitting and excessive steering (Lee et al., 11 Mar 2025).

6. Relation to adjacent methods, limitations, and later reassessment

SoftREPA occupies a distinct position among alignment methods. Relative to REPA, it aligns text–image representations directly via a contrastive score-based objective, uses only soft-token parameters on top of a frozen model, and does not depend on external encoders. Relative to RLHF, Diffusion-DPO, DDPO, or other preference-optimization approaches, it uses no reward model, no human preference data, and no policy-optimization loop; it relies only on paired image–caption data and batchwise negatives. Relative to training-free controls such as CFG++, Attend-and-Excite, or cross-attention editing heuristics, it changes the learned conditional behavior itself rather than only the sampling trajectory. Relative to textual inversion or LoRA, it is closest to prompt-tuning, but its soft tokens are layer- and time-indexed and are trained to improve global alignment rather than to encode new concepts (Lee et al., 11 Mar 2025).

The original limitations noted or implied include dependence on the strength of the pretrained base model, a capacity–overfitting trade-off when too many tokens or layers are used, possible domain shift outside COCO-like distributions, and continuing difficulty with very long or logically complex prompts. These limitations indicate that SoftREPA sharpens conditioning within the representational capacity of the pretrained system rather than replacing that capacity (Lee et al., 11 Mar 2025).

A later analysis in “Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models” re-situated SoftREPA as a strong reward-free representation-alignment baseline, but argued that its contrastive formulation can excessively penalize negative pairs (Lee et al., 28 May 2026). In that analysis, the InfoNCE objective is rewritten so that minimizing the negative log-sum-exp term encourages larger denoising error for negative captions, and the absence of an explicit bound on those negative denoising losses is linked to off-manifold behavior. The paper associates this with characteristic failure cases of SoftREPA: over-counting, repetition, and semantic incoherence. On GenEval for SD3, the reported Counting score for SoftREPA is 0.29, versus 0.64 for AGSM, while SoftREPA remains strong on other categories and on some preference-oriented metrics. The same work also reports training dynamics in which SoftREPA’s validation ImageReward peaks early and then degrades while training loss continues to decrease, and interprets this as over-optimization of the contrastive objective (Lee et al., 28 May 2026).

This later reassessment does not erase SoftREPA’s original contribution. Rather, it clarifies the trade-off inherent in its design. SoftREPA established that reward-free alignment can be achieved by optimizing a small set of soft tokens against the model’s own denoising signal, with minimal computational overhead and broad architectural portability. Subsequent work suggests that the same contrastive mechanism that increases mutual information and sharpens text sensitivity may also, if negative pairs are pushed too aggressively, distort the conditional manifold. In that sense, SoftREPA has become both a method and a reference point: a demonstration that internal score-matching signals suffice for alignment, and a case study in how contrastive objectives interact with generative score fields (Lee et al., 11 Mar 2025).