Anchored Decoding for Controlled Generation

• Anchored decoding is a technique that uses external reference anchors to guide neural generation by enforcing constraints and mitigating risk.
• It is applied across domains—such as language modeling, code generation, image compression, and autonomous driving—to enhance structure, diversity, and safety.
• This method improves output quality and control by dynamically integrating model predictions with trusted priors, achieving reductions in copying and bias.

Anchored decoding is a class of inference-time algorithms that control, guide, or constrain the generative process of neural models by leveraging privileged reference structures—called “anchors”—at decoding time. The anchor may be a trusted model, a learned prior, a trajectory dictionary, or a dynamically maintained set of verified predictions. This paradigm appears across diverse domains including language modeling, code generation, image compression, autonomous driving, and multi-modal tasks. Anchored decoding methods exploit anchors to enable risk control, increase mode diversity, enforce structure, accelerate generation, and mitigate rule-violation or bias, while often operating in a training-free and plug-and-play fashion.

1. Theoretical Foundations and General Principles

Anchored decoding interprets sequence generation as a balancing act between model utility and external constraints. It often formalizes the generator as producing an output sequence $y$ according to a distribution $p^*(y)$ that must remain close, in a probabilistically quantifiable sense, to a reference “anchor” distribution. In the case of copyright control (He et al., 6 Feb 2026), the risky model's distribution $p_r$ is fused with a safe anchor model’s distribution $p_s$, subject to a global Kullback–Leibler (KL) divergence budget

$KL(p^* \| p_s) \leq K,$

guaranteeing that generation stays within a controllable neighborhood of the permissively trained anchor. The per-step fusion

$$p_t^*(y) \propto p_r(y)^{1/(1+\lambda)} \cdot p_s(y)^{\lambda/(1+\lambda)}$$

acts as a constrained geometric mean, with the mixture weight adaptively selected to satisfy the local KL constraint.

Anchored decoding generalizes via the idea that decoding is not purely confidence-driven, but instead leverages external or dynamically inferred constraints—be they anchor tokens, anchor trajectories, anchor dictionaries, textual anchor banks, or other contextually derived priors—to modulate which outputs can be trusted, unmasked, or used to guide the denoising trajectory.

2. Anchoring in Diffusion and Non-Autoregressive Language Generation

In diffusion LMs, anchored decoding often involves fixing or prioritizing a subset of the sequence to act as a scaffold for the remainder. Soft and hard anchoring schemes have been proposed:

• Static Anchors: Suffix-anchored decoding (Park et al., 27 May 2026) appends a “weak structural hint” (e.g., “The answer is”) as a fixed suffix anchor to the output region, counteracting end-of-text overconfidence and cascading early termination in non-AR decoding. Anchor-proximity confidence modulation further downweights the unmasking bias around anchors according to spatial distance and denoising progress, preserving both the completion signal and local readiness.
• Dynamic Anchors: Anchor-based History-stable Decoding (AHD) (Zou et al., 10 Apr 2026) defines anchors dynamically by analyzing distributional stability of each token’s prediction across successive denoising steps. Tokens whose KL-divergence from their own recent prediction history falls below a small threshold are early-unlocked—even across semi-AR block boundaries—providing history-stable anchors that accelerate and stabilize generation.
• Supportive Revealing: AXON (Ayoub et al., 2 Jun 2026) selects masked tokens (“anchors”) not just on marginal confidence, but by maximizing their ability to support the denoising of other positions as measured by attention-weighted coverage of residual uncertainty. Submodular optimization yields a set of “supportive” anchors that, when revealed, most efficiently unblock downstream denoising.
• Anchor-Guided and Anchor-Perturbed Decoding: ASRD (Yao et al., 15 Jun 2026) tracks a cache of anchor tokens based on temporal consistency windows. These anchors are used both to guide mask filling (by steering masked embeddings toward the anchor centroid) and to verify candidate positions (by perturbing their embeddings orthogonally to the anchor centroid to test robustness), suppressing error propagation and local reinforcement.
• Hierarchical Structure Anchoring: AnchorTree (Xue et al., 5 Feb 2026) uses AST-derived soft anchoring in code diffusion, revealing high-level syntax tokens early and using graded auxiliary losses to explicitly incentivize validity and functional correctness.

3. Anchored Decoding in Constrained or Risk-Controlled Generation

Anchored decoding enables provable control of model risk in high-stakes applications such as copyright-sensitive language generation. The approach introduced by (He et al., 6 Feb 2026) and audited in (Vijayavallabh, 27 May 2026) uses a user-selected KL-divergence budget between a “risky” and a “safe” model. The information budget can be allocated per token or per byte (“Anchored${}_\mathrm{Byte}$”), accommodating different tokenizations. Real-time budget tracking guarantees, via theorem, that generation cannot drift beyond a pre-specified divergence level from the anchor model for any full or partial sequence.

The mechanism prevents verbatim copying: utility-risk trade-offs can be finely tuned by adjusting the per-step budget, tracing out a Pareto frontier between originality and anchor compliance. The framework also incorporates practical constraints like prefix debt (seeded by prompt likelihood spikes), adaptive budget banking, and empirical Bernstein-style bounds for auditability (Vijayavallabh, 27 May 2026).

4. Anchoring with Prior Templates or Meta-Report Banks

Anchored decoding generalizes beyond sequence-to-sequence LLMs. In multi-organ pathology report generation, (Yang et al., 1 Jul 2026) proposes the PriOrGen framework, which constructs a Meta-Report Anchored Bank (MRAB) of clinical templates for each organ, distilled and verified from the training data. At test time, visual features are encoded, aggregated, and used to retrieve the most relevant anchor templates via max-pooled cosine similarity. These anchors are embedded and injected into the decoder as prefixed summary tokens, directly shaping generation away from head-class (frequent organ) narrative bias and toward organ-specific diagnostics.

This approach instantiates an “anchor prior” over template banks, steering decoding at a structural level while preserving end-to-end differentiability (except for the non-differentiable top-v selection). No explicit anchor-fidelity loss is required; the negative log-likelihood on the anchored decoder context suffices.

5. Anchored Decoding for Structured Planning and Trajectory Proposal

In trajectory planning for autonomous driving, anchored decoding is instantiated at the level of multi-timestep proposals. The HMAD framework (Wang et al., 29 May 2025) constructs a trajectory dictionary via k-means clustering over demonstrated human maneuvers, from which representative “anchor” trajectories initialize the proposal set. Iterative offset decoding then refines each anchor under BEV (bird’s-eye-view) features via a stack of cross-attention and residual update layers, with strict clipping of residuals for stability. The approach ensures:

• Mode diversity: Each anchor seeds a distinct trajectory mode.
• Stability: Bounded residuals prevent unrealistic jumps.
• Rule-compliance: BEV-feature attention focuses refinement on drivable paths, respecting traffic constraints.
• Prevention of mode collapse: Fixed, learnable anchors with independent iterative refinement resist converging to the global mean.

Candidate anchors are subsequently scored under simulation-based criteria, such as no at-fault collisions, drivable area compliance, and comfort, before final selection.

6. Dual-Anchoring in Multi-Expert Decoding: Image Compression

Anchored decoding extends to deep image compression. MoDE (Mao et al., 14 May 2026) introduces a dual-latent anchored decoding framework, simultaneously leveraging a scalar-quantized (SQ) “fidelity expert” and a vector-quantized (VQ) “perception expert.” Both pathways are decoded in parallel from a shared bitstream, each enhanced via cross-expert modulation:

• Fidelity-anchored (MoDE-F): SQ branch augmented by selective VQ cues, optimized for distortion metrics.
• Perception-anchored (MoDE-P): VQ branch complemented by SQ structural features, optimized for perceptual metrics.

Decoder-side modules (ESE and CEM) ensure that each expert’s output remains anchored to its specialization, with modulated cross-expert transfer enabling smooth traversal along the rate-distortion-perception plane. The architecture admits no additional transmission cost or retraining at inference.

7. Comparative Perspective and Empirical Impact

Anchored decoding methods offer domain-general benefits:

Method	Anchor Type	Benefit	Domain
Anchored Decoding	Safe LM	Copyright control	Language
Suffix Anchoring	Fixed phrase	Output completion	Diffusion LM
AHD/ASRD	Dynamic tokens	Acceleration/Robust	Diffusion LM
PriOrGen	Template bank	Debiasing	Vision–Language (clinical)
HMAD	Trajectory dict.	Stability/Diversity	Autonomous driving
MoDE	Dual latent	Quality balancing	Image compression

Empirically, anchored decoding achieves up to 75% reduction in measured copying gap while preserving fluency (He et al., 6 Feb 2026), yields +30–50% absolute accuracy gains in diffusion LM benchmarks via anchor modulation (Park et al., 27 May 2026), accelerates decoding 2–16× across language, vision–language, and audio–language domains without retraining (Zou et al., 10 Apr 2026, Yao et al., 15 Jun 2026), and consistently improves robustness to head-tail bias and structural constraint failures (Yang et al., 1 Jul 2026, Wang et al., 29 May 2025, Mao et al., 14 May 2026).

Anchoring variants are often training-free, add minimal computational overhead, and integrate seamlessly with parallel, non-AR, and hybrid decoding schemes. The anchored paradigm’s theoretical grounds—KL constraints, submodular selection, dynamic stability tracking, and structured priors—yield rigorous performance and risk guarantees in safety-critical or structure-sensitive applications.