Block-R1-41K: Adaptive RL for Diffusion LLMs

• Block-R1-41K is a multi-domain corpus that annotates each sample with its optimal block size to guide adaptive reinforcement learning post-training for diffusion LLMs.
• It comprises over 41K balanced examples from diverse domains, including code generation, mathematical reasoning, and puzzle solving, to support heterogeneous training.
• The dataset demonstrates that sample-specific block size adaptation improves rollout trajectories and RL accuracy compared to fixed block size approaches.

Searching arXiv for the primary paper and closely related work on Block-R1-41K and diffusion LLM RL post-training. Block-R1-41K is a purpose-built, block-annotated multi-domain training corpus for reinforcement-learning post-training of diffusion LLMs (dLLMs) in heterogeneous reasoning settings. Its central design choice is to annotate each individual sample with a “best-improved training block size,” rather than to treat block size as a single fixed hyperparameter for all tasks. In the formulation used by the Block-R1 benchmark, block size is the number of tokens denoised in parallel during block-wise semi-autoregressive generation, and the dataset is intended to address the domain block size conflict that emerges when one block size cannot simultaneously support strong rollout trajectories across code generation, mathematical reasoning, puzzle solving, and advanced multi-step reasoning (Jiang et al., 12 May 2026).

1. Conceptual basis and problem setting

Block-R1-41K is defined around a specific claim about dLLM post-training: in multi-domain reinforcement learning, block size is not merely an inference-time decoding parameter, but a factor that affects rollout trajectories and therefore the effectiveness of rollout-based RL methods such as GRPO (Jiang et al., 12 May 2026). Conventional dLLM post-training uses a single fixed block size, which the benchmark describes as a rigid hyperparameter. Block-R1-41K replaces that assumption with sample-level annotation.

The key annotation is the “best-improved training block size” for each sample. Operationally, this is the decoding granularity that yields the largest reachable improvement for a student model relative to a teacher-student filtering procedure. The resulting corpus is therefore not just a collection of prompts and targets; it is a training set of pairs $(x, c_x^*)$, where each sample $x$ is associated with a preferred block size $c_x^*$ (Jiang et al., 12 May 2026).

A common misconception in this area is that block size should be optimized only per domain or only at inference. The Block-R1 formulation explicitly studies block size from a domain conflict perspective in multi-domain RL post-training, and its dataset construction is designed to expose disagreement in preferred training block sizes across heterogeneous tasks. This suggests that block size functions as a domain-sensitive control variable rather than as a universally transferable default.

2. Corpus composition and benchmark coverage

Block-R1-41K comprises over 41 000 high-quality training examples drawn from seven source domains, with 13 total evaluation benchmarks (Jiang et al., 12 May 2026). Its training pool is organized across code generation, mathematical reasoning, and puzzle solving, and the final assembled training set samples roughly 7 000 examples per domain, subject to dataset size, for a total of approximately 41 000 records.

Domain family	Source benchmark(s)	Training samples in source pool
Code generation	MBPP, KodCode	374; 9 285
Mathematical reasoning	GSM8K, MATH500, Countdown	7 473; 7 500; 240 632
Puzzle solving	Knights-and-Knaves, Sudoku	6 200; 1 000 000

The balancing strategy is important. Because the raw source pools vary substantially in size—from 374 MBPP samples to 1 000 000 Sudoku samples—the final dataset is not a proportional reflection of raw corpus frequency. Instead, it is an assembled training set intended to support balanced multi-domain RL experiments (Jiang et al., 12 May 2026).

Evaluation spans 13 benchmarks under a unified zero-shot pass@1 protocol with generation length 256 and diffusion steps 128. The paper explicitly names HumanEval, ARC-E/ARC-C, MMLU, MMLU-Pro, and HellaSwag among the evaluation tasks. This uniform evaluation protocol makes cross-domain comparisons more interpretable, because changes in reported best-checkpoint pass@1 are not confounded by benchmark-specific generation settings.

3. Construction pipeline and per-sample block-size annotation

The dataset is built through a five-stage pipeline (Jiang et al., 12 May 2026).

Stage 1, source selection: official training splits are collected from each reasoning benchmark, and overlapping examples are deduplicated.

Stage 2, reward design: a shared multi-component reward is defined per domain. The reward combines format-based tags such as <reasoning> and <answer>, accuracy checks including symbolic equivalence, test-case execution, or puzzle correctness, and constraint-based partial credit for well-formed but incomplete outputs.

Stage 3, teacher-student filtering: a strong teacher dLLM, exemplified by LLaDA2.0-mini, and a base student, exemplified by LLaDA-8B-Base, are used to evaluate each candidate sample over the candidate block-size set

$B=\{4,8,16,32,64,128\}.$

Samples are discarded if the student already achieves perfect reward at every $c \in B$, if the teacher never succeeds, or if the teacher’s maximum reward does not exceed the student’s.

Stage 4, best-improved block-size selection: for each surviving sample $x$, the benchmark estimates

$$A_{\theta_T}(x,c)=\mathbb{E}_{\tau\sim\pi_{\theta_T}^{(c)}}[R(\tau)] \quad\text{and}\quad A_{\theta_S}(x,c)=\mathbb{E}_{\tau\sim\pi_{\theta_S}^{(c)}}[R(\tau)],$$

then computes the improvement gap

$$\Delta(x,c)=A_{\theta_T}(x,c)-A_{\theta_S}(x,c), \qquad c_x^*=\arg\max_{c\in B}\Delta(x,c).$$

The selected $c_x^*$ is the decoding granularity where the student stands to gain most from RL.

Stage 5, dataset assembly: up to $n\approx 7\,000$ annotated examples are uniformly sampled per domain, yielding the final dataset

$x$0

A representative record given in the benchmark has the following fields: an input prompt such as “<reasoning>⁠… Solve 17+29 step by step⁠…</reasoning>”, domain “GSM8K”, best block size $x$1, and improvement gap $x$2 (Jiang et al., 12 May 2026). The example is illustrative rather than exhaustive, but it shows that the unit of annotation is the individual training sample, not the domain.

4. Block Size Conflict Score and cross-domain disagreement

To quantify disagreement in preferred block sizes across domains, Block-R1 introduces the Domain Block Size Conflict Score (BCS) (Jiang et al., 12 May 2026). For each domain $x$3, the empirical training preference distribution is defined as

$x$4

with cumulative distribution

$x$5

For two domains $x$6 and $x$7, the BCS is the one-dimensional Wasserstein distance

$x$8

Within the reported analyses, the per-domain histograms are markedly different. Countdown exhibits a heavy concentration at $x$9, whereas Sudoku and Knights-and-Knaves skew strongly toward $c_x^*$0 (Jiang et al., 12 May 2026). The benchmark also provides an example query in which the domain pair (Countdown, KodCode) has $c_x^*$1.

The pairwise BCS heatmap in Figure 1(b) is described as showing that Countdown has among the highest conflict scores with every other domain, and the paper states that this correlates with severe multi-domain performance degradation when a single fixed block size is enforced. In Figure 1(a), a scatter plot of BCS versus the mean drop in in-domain RL accuracy under vanilla mix-domain RL shows a clear positive trend. A plausible implication is that BCS serves not only as a descriptive statistic, but also as a diagnostic variable for predicting when fixed-block multi-domain training is likely to fail.

5. Use in the Block-R1 benchmark and training protocol

Block-R1-41K is the training core of the broader Block-R1 benchmark (Jiang et al., 12 May 2026). In this benchmark, the usual GRPO objective is modified so that each sample $c_x^*$2 generates trajectories under its own $c_x^*$3 rather than under a globally shared block size. This changes the role of block size from a single training hyperparameter into a sample-conditioned component of rollout generation.

Training proceeds in balanced mini-batches drawn across domains, with StableDRL as the default algorithm, LoRA-parameterisation with $c_x^*$4, and four AMD MI300X GPUs (Jiang et al., 12 May 2026). Evaluation follows the same zero-shot protocol across all 13 benchmarks and reports best-checkpoint pass@1.

The benchmark is open-sourced at https://github.com/YanJiangJerry/Block-R1, and the dataset is released at https://huggingface.co/datasets/dLLM-R1/Block-R1-41K (Jiang et al., 12 May 2026). In encyclopedic terms, this matters because the dataset is embedded in a full experimental framework rather than distributed as a standalone corpus without a reference training recipe.

6. Reported findings, interpretation, and research significance

The paper’s reported findings are both theoretical and empirical. It states that a one-for-all block size is structurally sub-optimal in multi-domain RL, citing Theorem 1, and that sample-level block adaptation largely eliminates this conflict (Jiang et al., 12 May 2026). In single-domain settings, each domain’s peak RL performance occurs at a distinct $c_x^*$5. Under vanilla multi-domain RL with fixed $c_x^*$6, performance not only falls short of in-domain RL but can even drop below the base model.

Within the reported benchmark results, Block-R1 yields consistent gains across every target domain, with examples including Countdown from 30.08 to 62.11 and KK from 30.14 to 50.14 (Jiang et al., 12 May 2026). The paper further states that Block-R1 closes most of the gap to the oracle sample-wise block-selection bound. Combining Block-R1 post-training with dynamic inference (b1) further raises accuracy, indicating that sample-level block-conditioning has effects in both training and test-time generation.

The broader significance of Block-R1-41K lies in the way it reframes block size. Rather than treating decoding granularity as a single scalar to be tuned once per model, the dataset operationalizes it as a per-sample supervisory signal derived from reachable teacher-student improvement. This suggests a more general research direction in which rollout structure, reward realizability, and domain heterogeneity are co-optimized. The benchmark itself concludes that Block-R1-41K enables principled, reproducible experimentation and guides future work toward adaptive decoding granularity in both training and inference (Jiang et al., 12 May 2026).