Conformative Decoding in LLMs

• Conformative decoding is a strategy that combines base and instruction-tuned models to mitigate the diversity gap in narrative generation tasks.
• The method employs post-truncation weighted mixing (γ = 0.5) of log probabilities to balance diversity recovery and maintain output quality.
• Empirical evaluations show increased lexical and semantic diversity, with the technique most effective when the original diversity gap is pronounced.

Searching arXiv for the cited papers to ground the article in current literature. Conformative decoding is a decoding-time strategy for instruction-tuned LLMs that addresses the “diversity gap”: the observed reduction in output diversity that accompanies instruction-tuning, especially on open-ended generation tasks such as narrative writing. The method guides an instruct model with its more diverse base model during next-token selection, using a weighted mixture of their token log-probabilities after truncation, with the goal of reintroducing diversity while maintaining or improving quality (Peeperkorn et al., 28 Jul 2025). In adjacent literature, “conformative” language also appears in broader decoding settings that force outputs to conform to auxiliary evidence or structure, including coarse-to-fine generative feedback in large vision-LLMs (LVLMs) and circuit-structured real-time decoding in fault-tolerant quantum computation (Cao et al., 29 Dec 2025, Bombín et al., 2023).

1. Diversity gap as the motivating phenomenon

The central empirical premise of conformative decoding is that instruction-tuning reduces output diversity. On a narrative generation task derived from the Writing Prompts dataset, base models are more diverse, whereas instruction-tuned variants are typically higher in quality or alignment but less diverse. The paper characterizes this as a systematic trade-off: instruction-tuned models show significantly lower diversity than their base counterparts under lexical and semantic diversity metrics, while Precision increases, indicating improved realism or quality but narrower coverage of the human reference manifold (Peeperkorn et al., 28 Jul 2025).

The experimental design is tailored to isolate this effect. The study uses prompts from training data with at least 50 human responses, manually filtered for inappropriate or offensive prompts and for responses that begin with commentary rather than story text. The first 50 human responses per prompt are kept, yielding 53 prompts and 2650 human responses. Generations are compared under three prompting regimes: Base (comp), Instruct (comp), and Instruct (chat). To make base models produce story continuations rather than unrelated completions, each generation is prefixed with a 20-token incipit from a human response (Peeperkorn et al., 28 Jul 2025).

The evaluated model families are Gemma 2 9B, Meta Llama 3.1 8B, Mistral 7B v0.3, OLMo 7B 0724, and OLMo 2 7B 1124. The OLMo families are particularly important because they expose intermediate checkpoints across base, supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning with verifiable rewards (RLVR). The reported stage-wise analysis finds that SFT reduces diversity, DPO reduces diversity the most, and RLVR has little additional effect relative to DPO. The paper therefore identifies preference optimization, especially DPO, as the main driver of diversity loss (Peeperkorn et al., 28 Jul 2025).

A further observation is that chat templating can intensify the problem. Using a chat template with instruct models typically further reduces diversity and quality compared with completion prompting. This matters because the diversity gap is not only a property of model weights; it is also mediated by the inference protocol used to elicit outputs (Peeperkorn et al., 28 Jul 2025).

2. Formal decoding rule

Conformative decoding combines the instruct model and the base model directly at decoding time. Let $p_{\theta}$ denote the instruction-tuned model distribution and $p_{\phi}$ the base model distribution. The rule is defined as

$$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$

where $\gamma \in [0,1]$ controls the mixture and $\mathcal{V}_{\mathrm{valid}}$ is the set of tokens allowed by truncation. Tokens outside $\mathcal{V}_{\mathrm{valid}}$ are assigned log-probability $-\infty$, so they are never sampled (Peeperkorn et al., 28 Jul 2025).

The operational sequence is explicit. First, next-token distributions are computed for the instruct model and the base model. Second, the nucleus set $\mathcal{V}_{\mathrm{valid}}$ is taken from the instruct model. Third, the two models’ log probabilities are combined with weight $\gamma$. Fourth, the resulting distribution is renormalized and sampled. The reported truncation strategy is nucleus sampling with $p = 0.95$, and the experiments use temperature $p_{\phi}$0 and conformative decoding weight $p_{\phi}$1 (Peeperkorn et al., 28 Jul 2025).

A crucial design choice is that the mixture is applied after truncation rather than before. The paper states that pre-truncation conformative decoding can raise diversity more strongly, but quality often drops. Post-truncation mixing is therefore described as safer and more stable, and also as conceptually elegant because the base model perturbs only the space that the instruct model has already deemed plausible. For efficiency, the implementation mixes logits directly, which the paper notes is equivalent after softmax normalization (Peeperkorn et al., 28 Jul 2025).

The limiting cases are straightforward. When $p_{\phi}$2, the method reduces to standard instruction-tuned decoding. When $p_{\phi}$3, decoding follows the base model. Intermediate values provide a controlled interpolation between the more aligned but narrower instruct model and the more varied base model. The paper uses $p_{\phi}$4 as a neutral choice rather than a heavily tuned optimum (Peeperkorn et al., 28 Jul 2025).

3. Evaluation protocol and diversity metrics

The conformative decoding experiments reuse the same narrative generation setting used to establish the diversity gap. The default sampling configuration is nucleus sampling with $p_{\phi}$5, temperature $p_{\phi}$6, 50 samples per prompt, and maximum story length 500 tokens. Instruct models are evaluated with the conversation prompt, and incipits are retained for alignment to the story task (Peeperkorn et al., 28 Jul 2025).

The evaluation framework mixes per-prompt diversity measures with across-prompt distributional comparisons. Per-prompt lexical and semantic diversity are measured with Vendi Score (VS) and Truncated Entropy (TE). VS is computed with $p_{\phi}$7-gram features for $p_{\phi}$8 and also on text embeddings. TE is computed on text embeddings. The formulas reported in the paper are

$p_{\phi}$9

where $$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$0 are the eigenvalues of the similarity matrix $$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$1, and

$$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$2

For semantic embeddings, the paper uses jinaai/jina-embeddings-v3 (Peeperkorn et al., 28 Jul 2025).

Across prompts, the study uses Mauve and Improved Precision and Recall with $$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$3. The interpretation given in the paper is that Precision approximates quality or realism, while Recall approximates coverage of the human reference manifold and thus diversity. This metric decomposition is important because conformative decoding is not intended to maximize diversity in isolation; it is intended to recover diversity without sacrificing the aspects of realism that instruction-tuning improved (Peeperkorn et al., 28 Jul 2025).

Statistical testing uses a one-tailed paired $$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$4-test with the hypothesis that conformative decoding improves diversity over the nucleus-sampling baseline. The baseline comparison is configuration A, nucleus sampling only with $$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$5, against configuration B, nucleus sampling plus conformative decoding with $$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$6 and $$\mathrm{ConformativeDecoding}(x_t \mid x_{<t}) = \gamma \log p_{\theta}(x_t \mid x_{<t}) + (1 - \gamma)\log p_{\phi}(x_t \mid x_{<t}) \quad \mathrm{for}\ x_t \in \mathcal{V}_{\mathrm{valid}},$$7 (Peeperkorn et al., 28 Jul 2025).

4. Empirical behavior and observed gains

The principal empirical claim is that conformative decoding usually increases diversity and maintains or slightly improves quality. On per-prompt metrics, most models show significant improvements in lexical diversity and semantic diversity. On across-prompt metrics, results are more consistently positive: Mauve improves, Recall increases, and Precision is maintained and sometimes slightly improves. The paper interprets this as evidence that conformative decoding recovers coverage of the human reference manifold without sacrificing realism (Peeperkorn et al., 28 Jul 2025).

The gains are characterized as significant but modest. The reason given is structural: the instruct model’s truncated distribution still constrains the output space, so the base model can diversify only within a plausibility region already selected by the instruct model. This means the method is deliberately conservative. A plausible implication is that its effectiveness depends on preserving the instruct model’s quality advantages while reopening parts of the output manifold that instruction-tuning narrowed (Peeperkorn et al., 28 Jul 2025).

The least favorable model-specific result is for Meta Llama 3.1 8B. In that case, one per-prompt diversity measure does not improve, and VS even declines significantly in one case. The paper speculates that this may reflect a weaker diversity gap to begin with, or a stronger model whose outputs are less in need of correction. This yields an important empirical caveat: conformative decoding appears most useful when the diversity gap is pronounced rather than marginal (Peeperkorn et al., 28 Jul 2025).

The qualitative interpretation follows directly from the measured trade-off. Base models are broader but less aligned; instruct models follow the requested task better but are more repetitive. Conformative decoding seeks an intermediate regime in which the instruct model remains instruction-following while the base model reintroduces broader next-token possibilities. The method is therefore positioned not as a replacement for alignment, but as a decoding-time means of partially recovering the creativity or exploratory capacity lost during alignment-oriented fine-tuning (Peeperkorn et al., 28 Jul 2025).

5. Limitations, caveats, and common misconceptions

Conformative decoding is not described as ensemble decoding in the usual sense. The paper emphasizes that existing methods generally operate within one model’s distribution, whereas conformative decoding explicitly leverages the relation between a base model and its instruction-tuned descendant. Its mechanism is therefore tied to a training lineage: a more diverse base model and a narrower instruct model must coexist, and the latter must still define the valid truncated token set (Peeperkorn et al., 28 Jul 2025).

The method is also not a license to remove truncation. The paper states that conformative decoding is only safe when paired with a truncation strategy such as nucleus sampling. Too little truncation risks degenerate text, whereas too much truncation limits diversity recovery. This dependence is central rather than incidental, because the instruct model’s nucleus is the mechanism that bounds quality degradation (Peeperkorn et al., 28 Jul 2025).

Several limitations are explicit. Narrative diversity is hard to evaluate, especially for long text, and human evaluation would be expensive and fatiguing. Only smaller open models are studied, so larger models may behave differently. Incipits themselves introduce some diversity, although they are needed to make base models produce valid stories. The method is more expensive at inference because it requires two models during decoding. The reported experiments used up to 3 Nvidia A100 80GB GPUs, with diversity-gap runs taking about 7–18 hours per run depending on model and conformative decoding runs taking about 18–24 hours per run (Peeperkorn et al., 28 Jul 2025).

These caveats delimit the method’s scope. The paper suggests that conformative decoding is most useful when an instruction-tuned model is clearly better aligned than its base model but has also clearly lost diversity, and when task utility depends on output variety as much as correctness, as in creative writing, ideation, or exploratory generation. If the instruct model is already diverse, or if the base model is weak or misaligned, the benefit may be small (Peeperkorn et al., 28 Jul 2025).

6. Broader uses of “conformative” and adjacent decoding paradigms

The term “conformative” has also appeared in adjacent decoding literature, but with different technical meanings. In LVLMs, "CoFi-Dec: Hallucination-Resistant Decoding via Coarse-to-Fine Generative Feedback in Large Vision-LLMs" introduces a training-free framework in which the model first generates intermediate textual responses from coarse- and fine-grained views of the original image, converts those responses into synthetic images using a text-to-image model, and then fuses the resulting next-token distributions using a Wasserstein barycenter. The paper describes this as a multi-path, self-corrective, conformative decoding framework whose goal is to reduce hallucination by making token choices conform to a consensus across original, coarse, and fine visual conditions (Cao et al., 29 Dec 2025).

A distinct structural use appears in quantum fault tolerance. "Modular decoding: parallelizable real-time decoding for quantum computers" proposes a modular decomposition of the global decoding problem into parallel sub-tasks aligned with the logical-block-network structure of a fault-tolerant circuit. In the paper’s discussion, this is presented as especially relevant to conformative decoding ideas because the decoder is designed to conform to the circuit’s logical-block structure and to emit intermediate logical outcomes in time for feed-forward control. The key technical device is the buffering condition, which preserves soundness and fault distance when sufficiently large buffers separate committed corrections from unavailable future data (Bombín et al., 2023).

By contrast, Conformer-based neural decoding in intracortical speech neuroprostheses is terminologically different. "End-to-End Intracortical Speech Decoding from Neural Activity" uses a Conformer encoder with a CTC decoder to map intracortical recordings directly to character sequences without an external LLM. In that work, “Conformer” names the neural backbone rather than a conformative decoding objective, and the main issues are session-specific alignment, data augmentation, inter-session degradation, and word-boundary segmentation errors rather than diversity restoration or auxiliary-distribution conformation (Khanday et al., 23 May 2026).

Taken together, these usages suggest that “conformative decoding” is best understood as a family resemblance rather than a single cross-domain formalism. In LLM narrative generation, it denotes weighted guidance from a more diverse base model to an instruct model. In LVLM hallucination mitigation, it denotes consensus-seeking across coarse-to-fine generative feedback. In quantum fault tolerance, the relevant idea is that decoding conforms to the logical architecture and causal structure of the computation. The shared theme is not a common equation, but a decoding process shaped by auxiliary structure that the base autoregressive or monolithic decoder would otherwise ignore.