Routing Agreement Decoding (RAD) in Sparse MoE

• Routing Agreement Decoding (RAD) is a method that uses internal sparse-MoE routing states as consensus signals rather than relying on answer strings.
• It operates by synchronizing rollouts with fixed anchors and selecting the densest route-basin center using a Weighted-Jaccard K-NN criterion.
• RAD is especially effective for code generation and agentic inference where conventional voting is unreliable, though it does not serve as a verifier.

Searching arXiv for the cited RAD and routing-by-agreement papers to ground the article in current sources. Routing Agreement Decoding (RAD) is an answer-string-free selection method for multi-rollout inference in sparse Mixture-of-Experts (MoE) LLMs. In its canonical formulation, RAD locates a fixed anchor in each completed rollout, represents the rollout by MoE routing states in an anchor-centered window, and returns the densest route-basin center under a Weighted-Jaccard $K$-NN criterion, without parsing, normalizing, executing, or voting over answer strings (Chen et al., 22 Jun 2026). The method is motivated by a specific empirical observation: even when the emitted token id is held fixed at repeated anchors, the router state and the experts producing that token still separate task context, trajectory history, and reasoning-effort mode, so routing can function as a model-internal consensus signal rather than a mere byproduct of token generation (Chen et al., 22 Jun 2026).

1. Core definition and problem setting

RAD addresses a failure mode of conventional multi-sample decoding: majority vote over answer strings is effective when answers are short and canonical, but becomes fraught or impossible when outputs are open-ended, lexically fragmented, executable, or difficult to extract and normalize reliably (Chen et al., 22 Jun 2026). In sparse MoE models, however, each token is accompanied by router decisions over experts. RAD uses those router decisions as the basis for rollout selection.

A defining feature of RAD is that inference-time selection is performed in routing space rather than output-string space. The selector operates on internal MoE routing patterns near a synchronized anchor, and only after the selection step is the corresponding answer string read, if it is read at all (Chen et al., 22 Jun 2026). This makes RAD applicable in settings where exact-string voting is ill-defined, such as code generation or patch selection.

The method is explicitly not presented as a verifier. Like majority voting, it is a consensus mechanism, and a dense wrong basin can still win (Chen et al., 22 Jun 2026). Its contribution is therefore an interface for selection under sparse-MoE routing access, not a guarantee of correctness.

2. Antecedents in routing-by-agreement

RAD belongs to a broader line of work that treats aggregation as a parts-to-whole assignment problem. In “Information Aggregation for Multi-Head Attention with Routing-by-Agreement” (Li et al., 2019), multi-head outputs are treated as input capsules whose contributions to output capsules are iteratively updated according to agreement. In “Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement” (Dou et al., 2019), individual layer representations are dynamically assigned to aggregated representations through routing coefficients rather than static weights. In “Learning Compositional Structures for Deep Learning: Why Routing-by-agreement is Necessary” (Venkatraman et al., 2020), routing coefficients are interpreted as learnable probabilities over derivation rules, and low-entropy routing is linked to tree-like compositional structure.

These precedents differ from RAD in their locus of operation. Earlier routing-by-agreement methods aggregate representations within a single forward pass: heads, layers, or capsules are the “parts,” and an internal representation is the “whole” (Li et al., 2019). RAD instead operates after rollout completion: each rollout is summarized by anchor-window routing states, and selection is performed across rollouts rather than across submodules inside one computation graph (Chen et al., 22 Jun 2026).

This suggests that “agreement” has two related but distinct meanings across the literature. In the capsule and routing-aggregation literature, agreement determines how internal components should be composed; in RAD, agreement determines which completed rollout lies at the center of a routing neighborhood that already aligns with an answer basin (Dou et al., 2019).

3. Operational mechanism

RAD begins by synchronizing rollouts with an anchor token. The anchor types described for the method are trajectory anchors, boundary anchors, and delimiter anchors (Chen et al., 22 Jun 2026).

Anchor type	Role	Examples
Trajectory anchors	Recurring markers in reasoning	“So”, “Now”, paragraph breaks
Boundary anchors	End-of-reasoning transitions	Special end/response tokens
Delimiter anchors	Answer-opening delimiters	\boxed{, code fences

For rollout $i$, let $\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$ denote the anchor window, where $a_i$ is the anchor position and $W$ is the window size. If $R_i(t)$ is the sparse per-layer routing vector at token $t$, concatenated across layers, then the rollout representation is

$$z_i = \frac{1}{|\mathcal{W}_i|}\sum_{t\in\mathcal{W}_i} R_i(t).$$

Similarity between two rollout representations is measured with Weighted Jaccard:

$$\mathrm{WJ}(u, v) = \frac{\sum_e \min(u_e, v_e)}{\sum_e \max(u_e, v_e)}.$$

Given a pool of $N$ rollouts for the same prompt, pairwise similarities $i$0 are computed from these routing vectors. For each rollout, RAD forms a routing-space neighborhood and scores its local density by

$i$1

then returns

$i$2

This is the densest Weighted-Jaccard $i$3-NN route-basin center (Chen et al., 22 Jun 2026). The selector therefore privileges the rollout whose anchor-window routing pattern is most central under local routing-space density.

The paper also describes a fusion variant, RAD+DC, which combines RAD with the DeepConf scalar confidence per rollout by retaining the highest-confidence half and then applying RAD density selection within that subset (Chen et al., 22 Jun 2026). This is presented as especially useful in rare-correct regimes.

4. Routing-state structure and agreement

The empirical basis of RAD is the observation that the same emitted token does not imply the same internal state. Holding the token id fixed at repeated anchors, the experts that produce it still separate task context, trajectory history, and reasoning-effort mode (Chen et al., 22 Jun 2026). This means that routing retains information that is not reducible to the visible token string.

The most important readout locations are near the answer boundary. The reported analysis states that near boundary anchors and delimiter anchors, routing neighborhoods already align with final-answer basins at a marker-only readout, and that the alignment is strongest when the routing is read at the answer opening (Chen et al., 22 Jun 2026). The most effective configuration is described as delimiter@16, the 16-token window starting from the answer-opening delimiter (Chen et al., 22 Jun 2026).

For analysis only, the paper introduces a basin-purity diagnostic that measures how strongly routing-space neighbors share answer labels. If $i$4 is the mean fraction of same-answer neighbors for readout $i$5, and $i$6 is the chance baseline, the adjusted purity is

$i$7

This quantity is not used at inference, but it is used to assess alignment between routing neighborhoods and answer agreement (Chen et al., 22 Jun 2026). High purity indicates that routing-space proximity is informative for selection.

A common misconception is that RAD merely re-encodes answer strings in hidden form. The reported evidence is more specific: routing neighborhoods near anchors align with answer basins, but the same token can still be produced by different expert subsets, so routing is not equivalent to token identity (Chen et al., 22 Jun 2026).

5. Empirical profile and application domains

The reported evaluation covers 10 sparse-MoE configurations and 6 datasets spanning math, GPQA, and code, using models that include gpt-oss (20B/120B; Low/Medium/High reasoning effort prompt levels), Qwen3-30B-A3B, and Qwen3-Next-80B-A3B (Chen et al., 22 Jun 2026). The datasets listed in the detailed description are AIME24/25, BRUMO25, HMMT25, GPQA, LiveCodeBench v5, and SWE-bench Verified (Chen et al., 22 Jun 2026).

Where string voting is well-posed, RAD is reported to be on par with Majority, with small positive paired deltas: RAD $i$8 and RAD+DC $i$9 versus Majority $\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$0 (Chen et al., 22 Jun 2026). The differences are described as not statistically significant, which situates RAD as competitive rather than categorically superior in canonical-answer regimes (Chen et al., 22 Jun 2026).

The principal empirical distinction appears in settings where answer strings are unavailable or ill-posed. On code, where exact-string voting degenerates, RAD still provides meaningful selection and is reported as $\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$1 over random (Chen et al., 22 Jun 2026). On SWE-bench Verified, when the method is re-anchored to the agentic boundary, it improves best-of-16 patch selection over random; the detailed summary reports $\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$2–$\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$3 improvement on the subset of problems where some rollouts pass and some fail, and $\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$4–$\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$5 on the full pool (Chen et al., 22 Jun 2026).

These results delimit RAD’s practical role. In canonical-answer settings it provides parity with answer voting while avoiding answer parsing; in code and agentic settings it provides a direct selection rule where conventional voting has no natural object to vote over (Chen et al., 22 Jun 2026).

6. Limitations, misconceptions, and broader interpretations

The sharpest limitation is stated directly in the original formulation: RAD is not a verifier, and a dense wrong basin can still win (Chen et al., 22 Jun 2026). Agreement in routing space is therefore a proxy for consensus, not for truth. This places RAD in the same general risk class as majority voting, with different observables and a broader domain of applicability.

A second limitation is architectural. RAD requires access to sparse-MoE routing states and is not usable on dense models or on deployments where routing information is unavailable (Chen et al., 22 Jun 2026). It also depends on anchor reliability. The detailed discussion notes that wider windows may partially condition on answer-region tokens, whereas strict marker-only readouts avoid this entirely with a small performance sacrifice (Chen et al., 22 Jun 2026).

A third misconception is to treat RAD as identical to earlier routing-by-agreement algorithms. Earlier work used routing to aggregate heads, layers, or capsules inside a model (Li et al., 2019). RAD, by contrast, uses routing states as selection features across completed rollouts (Chen et al., 22 Jun 2026). The two are related by the notion of agreement, but they solve different inference problems.

A broader, explicitly drawn conceptual extension appears in the description of BrainStack, a functionally guided Neuro-MoE framework for EEG-based language decoding. There, a learnable routing gate assigns softmax-normalized coefficients $\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$6 to regional and global experts and produces a fused representation

$\mathcal{W}_i = \{a_i, a_i+1, \ldots, a_i+W-1\}$7

with the weighted sum interpreted as an implicit consensus among experts (Zhao et al., 29 Jan 2026). The same description states that BrainStack’s routing gate is a functional instantiation of RAD themes such as adaptive routing and decision consensus (Zhao et al., 29 Jan 2026). This suggests that RAD can be understood narrowly as the sparse-MoE rollout selector defined above, or more broadly as a family resemblance among agreement-based routing mechanisms that convert internal specialization into a selection or fusion rule.