Block Size Conflict Score in RL

• Block Size Conflict Score (BCS) is defined as the 1-Wasserstein distance between domain-level distributions of best-improved training block sizes.
• It reveals how differing block size preferences, influenced by rollout trajectories and reward structures, affect multi-domain reinforcement learning performance.
• BCS serves as a diagnostic tool within the Block-R1 framework, prompting block-conditioned training to mitigate domain conflicts and enhance overall results.

Block Size Conflict Score (BCS) is an empirical metric for measuring how much two domains disagree about which training block sizes are most beneficial for reinforcement learning (RL) post-training of diffusion LLMs (dLLMs). It was introduced in the Block-R1 framework for multi-domain RL, where block size is not treated as a purely implementation-level hyperparameter but as a variable that changes rollout trajectories, reward structure, and optimization dynamics. In that setting, BCS is defined from domain-level distributions of sample-level best-improved training block sizes and computed as a $1$-Wasserstein distance, so it quantifies both probability mismatch and the ordered distance between preferred block sizes (Jiang et al., 12 May 2026).

1. Background: block size as a source of domain conflict

In the setting where BCS was introduced, dLLMs use block-wise semi-autoregressive generation. A sequence is partitioned into blocks

$$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$

where $c$ is the block size. Tokens within a block are denoised in parallel, while blocks are generated sequentially. The paper’s central claim is that block size determines the parallel decoding granularity and affects rollout trajectories during RL optimization, including methods such as GRPO. The GRPO-style objective is explicitly block-conditioned,

$$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$

Because rollouts are sampled from a block-conditioned policy $\pi^{(c)}$, changing $c$ changes the rollout trajectory distribution, the sequence evolution path, the reward distribution, and the group-relative advantage estimates (Jiang et al., 12 May 2026).

This dependence creates the possibility of domain block size conflict. For domain $\mathcal{D}_k$, the paper defines

$$J_k(\theta, c) = \mathbb{E}_{x \sim \mathcal{D}_k,\; \tau \sim \pi_\theta^{(c)}(\cdot \mid x)}[R(\tau)].$$

Two domains $\mathcal{D}_i$ and $\mathcal{D}_j$ are in conflict when their reward-optimal block sizes do not overlap: $$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$0 The paper further proves that if such preference divergence exists, then any single fixed block size $$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$1 yields a strictly worse aggregated objective than allowing each domain to use its own best block size: $$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$2 BCS is introduced as an empirical diagnostic proxy for this structural conflict (Jiang et al., 12 May 2026).

2. Formal definition of the Block Size Conflict Score

BCS is defined in three stages. The first stage is a sample-level notion of which training block size yields the largest teacher–student reward improvement. For a sample $$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$3 and block size $$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$4,

$$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$5

where

$$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$6

The sample-level best-improved training block size is then

$$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$7

The second stage aggregates these sample-level labels into a domain-level distribution. For each domain $$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$8,

$$\mathbf{b}_k = \{i \mid (k-1)c + 1 \le i \le \min(kc,L)\},$$9

This is called the Domain Block Size Preference Distribution. It is the probability that a random sample from domain $c$0 selects block size $c$1 as its best-improved training block size.

The third stage defines BCS itself as the $c$2-Wasserstein distance between two such domain-level preference distributions: $c$3 where $c$4 is the ordered candidate block-size set and $c$5 are the corresponding cumulative distribution functions. The rationale for Wasserstein distance is explicit: block sizes are ordered decoding granularities, not unordered categories, so the metric should account not only for probability mismatch but also for how far apart the preferred block sizes are along the block-size axis (Jiang et al., 12 May 2026).

A larger BCS means the two domains place mass on more different preferred training block sizes, especially on block sizes that are farther apart. In the paper’s interpretation, this indicates stronger domain block size conflict under fixed-block multi-domain RL.

3. Computation pipeline and relation to Block-R1-41K

BCS is tightly coupled to the construction of the Block-R1-41K dataset. The candidate block-size set used in the implementation is

$c$6

The teacher model is inclusionAI/LLaDA2.0-mini (16B), and the student model is GSAI-ML/LLaDA-8B-Base. For each sample $c$7 and each candidate block size $c$8, the method estimates $c$9 and $$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$0, computes $$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$1, and assigns $$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$2 by maximizing that improvement. In practice, Algorithm 1 states that expected rewards are estimated using $$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$3 rollouts, and the implementation uses $$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$4 unless otherwise specified (Jiang et al., 12 May 2026).

The paper applies several filters before assigning $$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$5. It removes samples where the student gets full reward under all $$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$6, removes samples where the teacher gets zero reward under all $$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$7, and removes anomaly samples satisfying

$$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$8

BCS is therefore computed on a filtered sample set intended to contain meaningful, reachable improvement.

The dataset construction process has five stages: cross-domain source selection, cross-domain reward design, teacher-student filtering, best-improved training block size selection, and balanced multi-domain assembly. The resulting dataset is

$$\mathcal{J}_{\mathrm{GRPO}}(\theta,c) = \mathbb{E} \Bigg[ \frac{1}{GL} \sum_{g=1}^{G} \sum_{i=1}^{L} \min\Big( r_g^i(\theta)\hat{A}_g, \mathrm{clip}(r_g^i(\theta),1-\epsilon,1+\epsilon)\hat{A}_g \Big) - \beta D_{\mathrm{KL}}(\pi_\theta^{(c)} \| \pi_{\mathrm{ref}}^{(c)}) \Bigg].$$9

BCS is induced directly from those same sample-level block-size labels. At the implementation level, the computation is hierarchical: first per sample through $\pi^{(c)}$0, then per domain through $\pi^{(c)}$1, and finally across domains through pairwise Wasserstein distance. The paper also notes a practical ambiguity about ties: while the formal definition uses $\pi^{(c)}$2, the visualization section states that “tied block sizes are all counted” (Jiang et al., 12 May 2026).

4. Empirical behavior and what BCS measures in practice

The empirical role of BCS is to explain why some domain mixtures degrade badly under fixed-block multi-domain RL while others remain compatible. The main observation is that larger BCS values generally correspond to more severe performance degradation under vanilla fixed-block joint training. Figure 1(a) plots BCS on the horizontal axis and the mean performance change from single-domain RL to fixed-block mix-domain RL on the vertical axis; the reported trend is that larger BCS generally leads to more severe degradation. Figure 1(b) visualizes pairwise BCS as a heatmap and highlights Countdown as having clear conflicts with many other domains. Figure 1(c) and the appendix show the underlying block-size preference distributions: Countdown favors smaller block sizes, while Sudoku and KK allocate more mass to larger block sizes; the appendix states more specifically that Countdown gives higher probability to $\pi^{(c)}$3 and $\pi^{(c)}$4, whereas Sudoku strongly favors $\pi^{(c)}$5 and $\pi^{(c)}$6 (Jiang et al., 12 May 2026).

Appendix Table $\pi^{(c)}$7 compares high-BCS and low-BCS domain pairs.

Domain pair	BCS	Category
Countdown + KK	0.1969	Large BCS
Countdown + MATH500	0.1418	Large BCS
Countdown + KodCode	0.1380	Large BCS
MATH500 + KK	0.0551	Small BCS
MATH500 + GSM8K	0.0219	Small BCS
MATH500 + KodCode	0.0103	Small BCS

For large-BCS pairs, fixed-block mix-domain RL shows large drops. The clearest example reported is Countdown: its in-domain result is $\pi^{(c)}$8, but when mixed with KK it becomes $\pi^{(c)}$9, a change of

$c$0

By contrast, small-BCS pairs remain stable or even improve slightly. The paper uses these pairwise results as direct evidence that BCS is informative about joint-training difficulty (Jiang et al., 12 May 2026).

The empirical meaning of the score is therefore not simply “different block-size histograms.” It is a predictor of whether two domains can tolerate one shared rollout granularity during RL post-training. High BCS indicates that the domains tend to concentrate probability mass on different best-improved training block sizes, especially far-apart ones; low BCS indicates that their training-block-size preferences are more compatible.

5. Uses, interpretation, and limitations

BCS is used primarily as a diagnostic and benchmark-analysis metric, not as a direct optimization target. The paper does not state that BCS is optimized during training, used as a domain-mixing scheduler, used as a loss regularizer, or used to choose batches or weights online. Its role is analytical and motivational: it diagnoses domain compatibility, explains why vanilla domain mixing fails, motivates sample-level block-conditioned training, and supports the Block-R1 benchmark design (Jiang et al., 12 May 2026).

This diagnostic role is closely tied to the Block-R1 method itself. Because Block-R1-41K stores per-sample best-improved training block sizes, the method can avoid forcing all domains to share one fixed rollout granularity. The paper presents this as mitigation of the conflict diagnosed by BCS. In the main cross-domain results, Block-R1 improves Countdown from $c$1 to $c$2, GSM8K from $c$3 to $c$4, and KK from $c$5 to $c$6. The benchmark covers 13 distinct datasets, 7 latest RL algorithms, and various different dLLM backbones (Jiang et al., 12 May 2026).

Several limitations are explicit. First, BCS is an empirical diagnostic rather than the theorem-level structural definition of domain conflict. Second, it depends on the teacher–student setup through

$c$7

so changing the teacher model, the student model, the reward functions, the candidate block-size set $c$8, or the rollout estimation procedure could change the resulting score. Third, it is restricted to the finite candidate set $c$9, so it measures preference divergence only at those discrete block sizes. Fourth, rollout estimation is noisy because the expected rewards are estimated from finite rollouts and often with $\mathcal{D}_k$0. Fifth, the implementation detail that “tied block sizes are all counted” introduces some ambiguity relative to the formal $\mathcal{D}_k$1 definition. Finally, the scope is specific to block-based dLLMs; the paper’s limitations section states that the framework may not directly generalize to non-block-based non-autoregressive generation paradigms (Jiang et al., 12 May 2026).

6. Related notions in adjacent literature

Subsequent literature often studies closely related phenomena without defining an explicit Block Size Conflict Score. DreamReasoner-8B studies the mismatch between training block size and inference block size in block diffusion reasoning models and describes a “granularity gap” and “granularity dilemma.” Its pilot results show a strong asymmetry: small-block training transfers reasonably to large-block inference, whereas large-block training can collapse under small-block inference. The paper does not define BCS, but it treats block-size mismatch as a central failure mode and proposes block-size curriculum learning to mitigate it (Wu et al., 17 Jun 2026).

Inference-time diffusion-language-model work has also introduced conflict-like runtime signals rather than domain-level distributional scores. DepCap separates block-boundary decisions from within-block parallel decoding decisions. Its block-sizing mechanism uses the dependency score

$\mathcal{D}_k$2

where $\mathcal{D}_k$3 is the last-block influence measured by KL divergence and $\mathcal{D}_k$4 is predictive entropy, while its CAP-Decoding stage defines a pairwise token conflict score

$\mathcal{D}_k$5

AdaBlock-dLLM likewise does not define BCS, but it formulates fixed-block failure in terms of “late decoding overhead” and “premature decoding error,” and uses confidence-based semantic delimiter selection as an adaptive boundary heuristic (Xia et al., 17 Apr 2026, Lu et al., 30 Sep 2025).

Outside diffusion-language-model training, blockchain systems literature uses other block-level conflict metrics that are analogous in form but different in meaning. A study of transaction concurrency in public blockchains defines the single-transaction conflict rate per block and the group conflict rate per block, both computed from per-block transaction dependency graphs. Those metrics quantify how much of a blockchain block is entangled by transaction dependency, rather than how much two data domains disagree about rollout granularity (Reijsbergen et al., 2020). Static analysis for Ethereum smart contracts provides another adjacent notion: pairwise read-write, write-write, and function-call conflicts between transaction-entry functions can induce a transaction conflict graph, but the paper does not define a Block Size Conflict Score (Atefeh et al., 6 Jul 2025).

These adjacent uses suggest that BCS belongs to a broader family of conflict quantification methods centered on ordered granularities, dependency structure, and compatibility under parallel or multi-domain processing. In the specific sense established by Block-R1, however, the term refers to a Wasserstein-distance measure over domain-level preference distributions of best-improved training block sizes in multi-domain RL for dLLMs (Jiang et al., 12 May 2026).