AR-informed Predictor-Corrector Sampling

• The paper introduces ARPC as a hybrid blockwise sampling method that leverages the model’s own autoregressive predictions to selectively re-generate the least plausible tokens without an auxiliary verifier.
• ARPC operates within the BlockGen framework by interleaving standard diffusion prediction steps with targeted AR-mode corrections, improving performance especially in the high-NFE regime.
• Empirical results show that ARPC narrows and even reverses the traditional performance gap between masked and uniform diffusion models, challenging the sampler-invariant assumption.

AR-informed Predictor-Corrector Sampling (ARPC) is the hybrid blockwise sampler introduced with BlockGen, a blockwise sequence model instantiated with both masked diffusion and uniform-state diffusion. In BlockGen, ARPC uses the model’s own autoregressive (AR) predictions as a verifier-like signal inside a predictor-corrector loop, with the explicit aim of re-generating the least plausible tokens according to an AR pass, without training any auxiliary verifier. The method is introduced against a background in which prior comparisons between masked diffusion models (MDMs) and uniform-state diffusion models (USDMs) typically used ancestral sampling or uninformed predictor-corrector samplers that re-inject noise at random positions, rather than a sampler that explicitly targets the tokens most likely to be mistaken (Deschenaux et al., 1 Jun 2026).

1. Definition and motivation

ARPC stands for AR-Informed Predictor-Corrector. The stated motivation is that standard predictor-corrector methods for discrete diffusion often re-noise tokens at random positions or use only an uninformed uncertainty score, which means they may spend correction budget on tokens that are not actually wrong. The paper argues that this is especially limiting in the block-by-block setting, where parallel decoding can produce locally inconsistent token combinations. ARPC is therefore designed to re-generate the least plausible tokens according to an AR pass, rather than applying non-targeted correction (Deschenaux et al., 1 Jun 2026).

Within the paper’s experimental framing, the importance of this change is not merely algorithmic but comparative. The central claim is that once both masked and uniform paradigms are given a stronger, informed correction mechanism, the apparent advantage of USDMs can change: under ancestral sampling USDMs are better, but under ARPC the gap closes and can reverse at high NFE. This directly bears on a common misconception in the area, namely that the superiority of uniform-state diffusion is sampler-invariant. The reported results do not support that interpretation.

ARPC is also defined by what it does not require. Its informed correction signal is not supplied by a separate verifier model. Instead, the AR pass is produced by the same BlockGen denoiser in $L'=1$ mode. This makes ARPC a single-model hybrid rather than a multi-model verifier pipeline.

2. BlockGen as the enabling model class

BlockGen trains on a mixture over block sizes,

$$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$

This mixture is the mechanism that lets the same denoiser operate at a larger block size $L' > 1$, where it behaves like a block diffusion model, and at block size $L'=1$, where it behaves like an AR model (Deschenaux et al., 1 Jun 2026).

The paper gives two lower bounds for the likelihood:

$$\log p_\theta^{\text{BlockGen}} \geq \log \sum_{i=1}^{M} e^{\mathcal{E}^{(s_i)}(\theta, ) + \log \gamma_i}$$

and, by Jensen,

$$\log p_\theta^{\text{BlockGen}} \geq \sum_{i=1}^{M} \gamma_i \, \mathcal{E}^{(s_i)}(\theta, ).$$

The paper explicitly states that the likelihood interpolates between AR and pure diffusion more finely than fixed-block models. In the paper’s presentation, this interpolation is the key enabler for ARPC, because the same shared denoiser can be used both for the diffusion prediction and for the AR scoring step.

The paper also emphasizes that BlockGen is “agnostic” to whether the within-block diffusion is masked or uniform. Accordingly, ARPC can be applied to masked BlockGen, which is MDM-style within each block, and to uniform BlockGen, which is USDM-style within each block. This point is central to the paper’s comparative design: the informed-correction protocol is held fixed while the within-block diffusion family varies.

3. Sampling mechanics and token selection

For a single block $b$, ARPC proceeds through a standard predictor step followed, at selected iterations, by an informed corrector step. At diffusion time $t$, with block state $\mathbf{z}_t^b$, prefix $^{<b}$, and block size $$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$0, the denoiser produces diffusion predictions

$$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$1

A standard ancestral step would directly sample the next block state using the diffusion posterior. ARPC diverges from this baseline in how it allocates correction effort (Deschenaux et al., 1 Jun 2026).

If the current step is an informed correction step, ARPC first samples a clean proposal from the diffusion model,

$$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$2

then runs a second forward pass in AR mode with block size $$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$3,

$$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$4

Each token position $$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$5 is scored by the AR log-likelihood of the proposed token,

$$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$6

after which the sampler chooses the lowest-scoring $$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$7 positions,

$$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$8

$$p_\theta^{\text{BlockGen}} = \sum_{i=1}^{M} \gamma_i \, p_\theta^{(s_i)}.$$9

The block state is set to the clean proposal,

$L' > 1$0

and only the selected positions are re-noised according to the forward corruption process at time $L' > 1$1,

$L' > 1$2

The core difference from standard predictor-corrector sampling is therefore the selection mechanism for which positions to re-noise. Standard predictor-corrector re-noises random or entropy-selected positions, often without AR scoring; ARPC uses the model’s own AR likelihood to identify the positions most likely to be wrong, then re-noises only those positions.

Sampler	Token score $L' > 1$3	Positions re-noised
Standard predictor-corrector	No targeted AR score	Random or without targeting specific likely errors
EIPC	$L' > 1$4	The least confident positions
ARPC	$L' > 1$5	The bottom-$L' > 1$6 positions under AR log-likelihood

This comparison clarifies the novelty claimed in the paper. EIPC scores positions by diffusion entropy, whereas ARPC scores positions by AR plausibility. The AR signal is internal to BlockGen rather than supplied by an auxiliary verifier, which is why the paper describes the method as using the model both as a diffusion generator and as its own AR verifier.

4. Scheduling, NFE, and block-size dependence

ARPC does not apply informed correction at every step. The sampler uses $L' > 1$7 initial ancestral-only steps, then an informed step every $L' > 1$8 updates thereafter. This interleaving is part of the reported sampling procedure rather than a secondary implementation detail (Deschenaux et al., 1 Jun 2026).

The paper defines NFE as number of function evaluations. For ARPC, an informed step requires two denoiser passes: one diffusion/blockwise pass and one AR-mode pass for scoring. The reported accounting is

$L' > 1$9

In the paper’s compute model, this formula counts the predictor passes plus the extra AR scoring passes for informed steps.

Block size matters in two ways. First, for modeling and likelihood, the paper states that the ELBO becomes looser as block size grows; smaller blocks interpolate more closely toward AR, which improves likelihood and often accuracy. Second, for empirical performance, the paper reports that for GSM8K, block size 16 generally outperforms block size 32 at matched NFE per block under ARPC. The paper says this is expected because the ELBO loosens with larger block size. A plausible implication is that ARPC benefits not only from targeted correction but also from operating in a regime where the blockwise approximation remains close to the $L'=1$0 AR limit.

The same section of the paper also indicates that larger blocks still benefit from the hybrid correction idea. The point is therefore not that ARPC is restricted to small blocks, but that the mixture-over-block-sizes formulation gives especially strong leverage when the chosen block size remains relatively close to the autoregressive endpoint.

5. Empirical behavior, metrics, and comparative findings

The paper evaluates ARPC using validation perplexity, Generative Perplexity, unigram entropy, and GSM8K exact-match accuracy. Validation perplexity is defined as

$L'=1$1

For generated text quality, the paper reports Generative Perplexity (Gen. PPL) under a reference LM, GPT-2 Large:

$L'=1$2

On GSM8K, accuracy is exact-match accuracy after generating code/programs and executing them against the ground-truth answers. The paper treats generation quality mainly through Generative Perplexity and unigram entropy as a diversity proxy; it also notes that low Gen. PPL alone can be misleading if text is degenerate, so unigram entropy is reported as a complementary measure (Deschenaux et al., 1 Jun 2026).

The main empirical story is organized around the contrast between ancestral sampling and ARPC. Under ancestral sampling, uniform diffusion outperforms masked diffusion in the block-by-block setting, especially in the few-step / low-NFE regime. The gap narrows as NFE increases, but uniform remains better in the ancestral baseline across the tested ranges. Under ARPC, the gap closes, and at higher NFE the ranking can reverse, with masked diffusion becoming better than uniform.

The paper makes this reversal concrete in two domains. On GSM8K with block size 16, masked + ARPC slightly exceeds uniform + ARPC at high NFE. On OpenWebText, the same qualitative pattern appears: at lower per-block NFE, uniform-ARPC has the better frontier, whereas at higher per-block NFE, masked-ARPC becomes better. This is the paper’s principal comparative conclusion about ARPC: the masked-versus-uniform story depends strongly on whether the corrector is uninformed or AR-informed.

At the same time, the reported results delimit ARPC’s gains. The paper notes that ARPC narrows the gap to AR sampling, but does not surpass AR on OpenWebText in the practical low-temperature regime; greedy AR remains strongest on GSM8K. ARPC is therefore presented as a hybrid blockwise sampling strategy that improves on uninformed predictor-corrector methods, not as a general replacement for strong AR decoding.

6. Relation to earlier informed-correction and AR-acceleration work

ARPC sits within a broader predictor-corrector lineage but differs in the source of its correction signal and in its blockwise integration. In "Informed Correctors for Discrete Diffusion Models" (Zhao et al., 2024), the corrector is informed by the diffusion model to counter accumulating approximation errors in continuous-time discrete diffusion, especially for absorbing-state forward processes. That paper replaces the standard forward-backward corrector with informed correctors based on locally balanced proposals, including MPF and Barker operators, and introduces hollow transformers so that the model can learn the score information required for absorbing diffusion. By contrast, ARPC changes the selection mechanism for which positions to re-noise by using AR plausibility from the same BlockGen denoiser in $L'=1$3 mode; it does not rely on an auxiliary verifier and is described as designed specifically for block generation with KV caching and block-causal attention.

There are also close conceptual precursors on the autoregressive side. "Predictive Sampling with Forecasting Autoregressive Models" (Wiggers et al., 2020) proposes predictive sampling for ARMs in which a predictor forecasts future values and the ARM itself acts as a corrector/validator. The fixed-point iteration variant is especially close in logic to a predictor-corrector loop, although its stated objective is exact sampling acceleration for the original ARM distribution rather than approximate correction dynamics over a diffusion trajectory. The paper has been characterized as a direct and technically important precursor to AR-informed predictor-corrector sampling.

A second related line appears in "Neural Approximation of an Auto-Regressive Process through Confidence Guided Sampling" (Yoo et al., 2019), where a lightweight predictor proposes future priors in parallel, the mother AR model post-processes those priors, and a confidence predictor decides whether a sample should be accepted or re-sampled by the true AR model. That work presents the idea as NARA rather than ARPC, and its mechanism depends on an i.i.d. prior-sample assumption together with a learned confidence gate. The shared structural theme is that a cheap proposal stage is followed by selective correction using AR information, but ARPC differs in that the verifier-like signal is the AR-mode pass of the same BlockGen denoiser and the correction operation is token-selective re-noising inside a diffusion sampler.

Taken together, these comparisons suggest that ARPC is best understood not as an isolated sampler but as a specific synthesis: blockwise discrete diffusion, a mixture over block sizes that interpolates between AR and diffusion, and informed correction driven by the model’s own AR likelihood. In that synthesis, the paper’s bottom line is precise: ARPC improves on standard predictor-corrector methods by targeting the least AR-plausible tokens for re-noising instead of choosing positions randomly or only via diffusion uncertainty, and empirically it changes the masked-versus-uniform comparison, especially at high NFE and for smaller block sizes such as 16 (Deschenaux et al., 1 Jun 2026).