Block-R1: Block-Size-Aware dLLM RL

• Block-R1 is a framework that treats block size as a dynamic, domain-dependent variable during RL post-training of diffusion large language models.
• It introduces the Block-R1-41K dataset and a teacher-student filtering approach to determine sample-level optimal block sizes, mitigating domain conflicts.
• The method complements dynamic inference (b1) by adapting block boundaries using entropy descent objectives to enhance reasoning coherence and overall performance.

Block-R1 denotes a block-size-aware framework for reinforcement-learning post-training of diffusion LLMs (dLLMs) in multi-domain reasoning settings. In its primary 2026 formulation, Block-R1 studies block size as a domain-dependent variable rather than a fixed decoding hyperparameter, introduces the Block-R1-41K dataset with sample-level best-improved training block sizes, and defines a benchmark for single-domain and cross-domain RL on dLLMs (Jiang et al., 12 May 2026). The same research line is closely connected to the dynamic-block framework b1, which replaces fixed-size reasoning blocks at inference time with adaptive block termination learned through a Monotonic Entropy Descent objective under GRPO (Jiang et al., 4 May 2026). Taken together, these works treat block granularity as a central control variable for both rollout optimization and reasoning coherence in diffusion-based language modeling.

1. Block size in diffusion LLMs

In block-wise semi-autoregressive decoding, a dLLM with block size $c$ generates tokens in parallel within each block, but sequentially across blocks. During RL post-training, including GRPO-style optimization, the rollout distribution, reward estimates, and policy updates therefore depend on $c$ (Jiang et al., 12 May 2026). Block-R1 formalizes the consequence of this dependence in multi-domain settings: different reasoning domains empirically prefer different decoding granularities, so fixing one block size across all domains induces a domain block size conflict.

The benchmark paper states this point as a strict sub-optimality result under two mild assumptions, sample-level reward alignment and domain-level preference divergence. With

$$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$

forcing a single block size $c_0$ across all domains satisfies

$$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$

This formulation makes block size an optimization variable at the level of post-training policy geometry rather than merely a decoding-speed knob.

A related but distinct observation appears in the dynamic-block work on b1. There, fixed-size blocks are identified as an inference-time bottleneck for coherent reasoning because different tasks, and even different steps within a single task, have different “natural” block lengths, while rigid boundaries can cut logical steps midstream and produce high-entropy spikes correlated with wrong answers (Jiang et al., 4 May 2026). A plausible implication is that Block-R1 addresses block-size mismatch at the domain and sample level during RL post-training, whereas b1 addresses block partitioning within individual reasoning trajectories.

2. Formalization of domain block size conflict

Block-R1 operationalizes domain conflict through teacher-student improvement under a candidate block-size set $B=\{c_1,\dots,c_S\}$. For a sample $x$, the improvement of a teacher model $\theta_T$ over a student model $\theta_S$ at block size $c$ is

$c$0

The sample-level best-improved block size is then defined as

$c$1

This assigns each training sample a preferred rollout granularity derived from measured reward improvement rather than heuristic selection (Jiang et al., 12 May 2026).

At the domain level, Block-R1 defines the empirical distribution of preferred sizes,

$c$2

and quantifies pairwise conflict by the Block Size Conflict Score (BCS), given by the $c$3-Wasserstein distance

$c$4

where $c$5. A larger BCS indicates that two domains prefer substantially different decoding granularities.

The benchmark further reports that block-size distributions differ strongly by domain. Countdown concentrates on $c$6, whereas Sudoku concentrates on $c$7 (Jiang et al., 12 May 2026). This is not merely descriptive; it is used as evidence that a single fixed $c$8 can create systematic optimization conflict in mixed-domain rollouts.

3. Block-R1-41K dataset and benchmark design

Block-R1-41K assembles approximately $c$9 high-quality, multi-domain reasoning samples, each tagged with its sample-level best-improved block size $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$0 (Jiang et al., 12 May 2026). Its construction has five stages.

First, source selection covers six source domains: code generation (KodCode), mathematical reasoning (GSM8K, MATH500, Countdown), logical puzzles (Knights-and-Knaves, Sudoku), general capabilities (HellaSwag), and advanced reasoning (MMLU-Pro, ARC-C). For large datasets, up to $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$1 training examples are randomly sampled; smaller datasets are either excluded or used only for evaluation.

Second, Block-R1 uses unified reward design. Each domain’s reward combines format-based signals, accuracy checks such as code execution or symbolic math equivalence, and constraint-based partial credits to reduce sparsity.

Third, teacher-student filtering evaluates each sample across the candidate block-size set $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$2 using $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$3 trajectories from teacher and student models. Samples are discarded if they are too easy, too hard, or anomalous in the sense that the teacher never outperforms the student.

Fourth, best-improved block size selection computes $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$4 and stores $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$5, yielding a domain-specific pool $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$6.

Fifth, balanced multi-domain assembly samples $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$7 examples from each pool and forms

$$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$8

The benchmark is designed for both single-domain RL and cross-domain or multi-domain RL. In the single-domain setting, a model is post-trained on one source domain and evaluated on the same domain. Reported baselines include supervised fine-tuning and seven RL algorithms: Diffu-GRPO, d1, wd1, GDPO, MDPO, ESPO, and StableDRL. The default fixed block size is $$J_k(\theta,c)=\mathbb{E}_{x\sim\mathcal D_k,\;\tau\sim\pi_\theta^{(c)}}[R(\tau)],$$9, the rollout group size is $c_0$0, and the number of diffusion steps is $c_0$1 (Jiang et al., 12 May 2026).

Supported backbones total ten, with GSAI-ML/LLaDA-8B-Instruct as the default; examples listed include LLaDA-1.5, LLaDA2.0-mini, Dream-v0-7B, SDAR-8B-Chat, and TraDo-8B-Instruct. The benchmark and dataset are open-sourced through the Block-R1 repository and the Block-R1-41K Hugging Face release (Jiang et al., 12 May 2026).

4. Sample-conditioned cross-domain RL

The central training innovation in Block-R1 is to replace the fixed-$c_0$2 policy $c_0$3 in GRPO with a sample-specific policy $c_0$4. In the benchmark’s cross-domain setting, the vanilla baseline mixes all source domains under a single fixed block size $c_0$5, whereas Block-R1 mixes all domains but conditions each rollout on its sample’s annotated $c_0$6 (Jiang et al., 12 May 2026).

The resulting sequence-level surrogate objective is

$c_0$7

Here $c_0$8 are importance ratios, $c_0$9 are group advantages, and all trajectories are sampled from $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$0.

The benchmark also provides condensed pseudocode: sample a batch of $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$1, generate $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$2 trajectories under $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$3 for each pair, compute rewards and advantages, form the surrogate loss, and update $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$4 via a policy-gradient step (Jiang et al., 12 May 2026). In practical terms, the method changes the rollout distribution at the sample level without altering the general GRPO training template.

This design targets a specific pathology of multi-domain dLLM RL: rollout-based optimization can degrade when heterogeneous domains are forced through a single block-size schedule. The benchmark reports that this degradation correlates with high BCS values, linking an explicit dataset statistic to downstream training failure modes (Jiang et al., 12 May 2026).

5. Dynamic-size reasoning blocks and Monotonic Entropy Descent

The companion framework b1 extends block-size adaptivity from training-time domain selection to inference-time block segmentation. In dLLMs such as LLaDA, generation typically proceeds in fixed-size blocks, for example $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$5, with parallel denoising inside each block (Jiang et al., 4 May 2026). b1 replaces that rigid scheme by learning to end each block dynamically through a special end-of-step token $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$6.

At the diffusion step $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$7 when $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$8 first appears, block $$\sum_{k=1}^{M}\lambda_k\,J_k(\theta,c_0) \;<\; \sum_{k=1}^{M}\lambda_k \max_{c\in B}J_k(\theta,c).$$9 of length $B=\{c_1,\dots,c_S\}$0 spans positions $B=\{c_1,\dots,c_S\}$1. Token-wise Shannon entropy at position $B=\{c_1,\dots,c_S\}$2 is

$B=\{c_1,\dots,c_S\}$3

and block entropy is defined as

$B=\{c_1,\dots,c_S\}$4

The empirical observation motivating b1 is that correct reasoning traces show steadily decreasing block entropy, whereas incorrect traces fluctuate or increase.

To exploit this pattern, b1 introduces a Monotonic Entropy Descent objective. Its local surrogate reward is

$B=\{c_1,\dots,c_S\}$5

which enforces entropy descent across adjacent blocks. A separate block-ending indicator reward,

$B=\{c_1,\dots,c_S\}$6

encourages multi-step reasoning. The full RL reward is

$B=\{c_1,\dots,c_S\}$7

with default weights $B=\{c_1,\dots,c_S\}$8 (Jiang et al., 4 May 2026).

Training uses GRPO with approximate policy ratios and a KL penalty. During each update, prompts are sampled, $B=\{c_1,\dots,c_S\}$9 completions are generated, dynamic blocks are reconstructed through $x$0, $x$1, $x$2, and $x$3 are computed, and the GRPO update is applied using group-normalized advantages. At inference, the model begins from a fully masked sequence of length $x$4, iterates through diffusion steps on positions $x$5, monitors for $x$6, commits the current block once it appears, and repeats until EOS or the length limit (Jiang et al., 4 May 2026).

b1 is described as plug-and-play with existing diffusion-GRPO pipelines such as d1 and wd1. This suggests a complementary layering: Block-R1 assigns effective training block sizes across domains and samples, while b1 relaxes fixed boundaries inside individual reasoning trajectories.

6. Empirical results, ablations, and limitations

Block-R1 reports that all RL methods improve in-domain accuracy over the base model in single-domain training, but gains are domain-specific; for example, StableDRL excels on arithmetic, whereas wd1 excels on Knights-and-Knaves (Jiang et al., 12 May 2026). In cross-domain training, the vanilla fixed-block mix often hurts performance relative to single-domain RL and can even fall below the base model. By contrast, sample-conditioned Block-R1 yields consistent gains across all 13 tasks. With LLaDA-8B, Countdown improves from $x$7 to $x$8 $x$9, GSM8K from $\theta_T$0 to $\theta_T$1 $\theta_T$2, and HumanEval from $\theta_T$3 to $\theta_T$4 $\theta_T$5; similar improvements are reported across all ten backbones, often in the $\theta_T$6–$\theta_T$7 percentage-point range (Jiang et al., 12 May 2026).

The dynamic-block results for b1 are reported on GSM8K, MATH500, Sudoku $\theta_T$8, and Countdown under zero-shot pass@1 with maximum lengths $\theta_T$9. Across Diffu-GRPO, GDPO, d1, and wd1, adding b1 consistently improves fixed-size block baselines. Examples include wd1 to wd1+b1 on GSM8K, from $\theta_S$0 to $\theta_S$1 $\theta_S$2; on Countdown, from $\theta_S$3 to $\theta_S$4; and on MATH500, from $\theta_S$5 to $\theta_S$6. Even on d1, b1 adds $\theta_S$7–$\theta_S$8 absolute (Jiang et al., 4 May 2026).

Coherence metrics in b1 are $\theta_S$9 and $c$0, the proportion of descents. The reported average $c$1 rises, for example from $c$2 to $c$3 on GSM8K, while $c$4 increases from approximately $c$5 to $c$6. Correlation analysis further shows that higher $c$7 bins correlate with higher accuracy, and b1 shifts hard examples out of negative-$c$8 zones into positive ones (Jiang et al., 4 May 2026).

Ablations support the role of both entropy descent and block-ending rewards. Removing $c$9 or $c$00 hurts performance; in particular, the “w/o MED” setting lowers Countdown accuracy from $c$01 to $c$02. Efficiency overhead is reported as negligible, with training approximately $c$03 slower and inference throughput remaining comparable or better (Jiang et al., 4 May 2026).

The stated limitations of b1 are that it still relies on approximate GRPO and block independence assumptions, and that it introduces an indicator token together with a slight $c$04 complexity term, although this is described as dominated by self-attention. Future directions include alternative monotonicity metrics such as Kendall’s $c$05, application to multimodal or code-reasoning diffusion models, and joint learning of dynamic block sizes during supervised or mixed RL/SFT training (Jiang et al., 4 May 2026). Within the broader Block-R1 program, a plausible implication is that future work will continue to unify training-time, sample-level block-size conditioning with inference-time dynamic block segmentation rather than treating either problem in isolation.