Diffusion-Adaptive Routing (DAR)

• Diffusion-Adaptive Routing (DAR) is a family of adaptive algorithms that redistribute computational or physical resources based on dynamic workload, network state, or learning dynamics.
• In wireless networks, DAR uses heat-diffusion models to adjust packet forwarding in real time, balancing delay and cost via tunable parameters.
• In diffusion models and transformers, DAR implements expert-choice and timestep-aware routing to balance load, enhance throughput, and accelerate convergence.

Diffusion-Adaptive Routing (DAR) refers to a family of adaptive routing algorithms and architectural mechanisms developed independently for high-throughput networking, non-autoregressive sequence modeling, and large-scale diffusion-based generative models. Though differing in technical realization and context, DAR schemes share a key property: they enable global, adaptive, and often learnable redistribution of computational or physical resources, aligned with varying workload, information content, or network state, to optimize efficiency, convergence, and/or cost.

1. Theoretical Foundations and Historical Context

The term Diffusion-Adaptive Routing has multiple origins. In wireless networking and control, DAR (also known as Heat-Diffusion routing or HD) was established as a policy that models packet flow in multihop, interference-limited networks as a discrete heat equation, dynamically adapting forwarding rates to both local congestion and time-varying link costs. In deep learning for generative modeling, DAR recently denotes adaptive computation architectures within diffusion models, where time-, token-, or expert-aware routing enables non-uniform and learnable resource deployment across denoising steps or layers (Zhang et al., 2 Apr 2026, Sun et al., 2024, Xu et al., 20 May 2026).

This conceptual parallel derives from the shared need to balance efficiency, adaptability, and optimality in highly parallel systems—whether packets on a network or activations in a neural model—by explicitly linking local routing decisions to global system state via diffusion-like principles.

2. Diffusion-Adaptive Routing in Wireless Networks

In time-varying multihop wireless networks, DAR/HD introduces a control parameterized dynamic routing policy that determines which nodes forward packets along which links at each slot, using only current queue occupancies and link state information. The algorithm operates as follows (Banirazi et al., 2019):

• For every link $(i, j)$ at time $n$, computes the queue differential $q_{ij}(n) = q_i(n) - q_j(n)$ and the cost-adaptive scaling factor $\phi_{ij}(n)$.
• Sets each link's tentative flow to $$\widehat{f}_{ij}(n) = \min\left\{\phi_{ij}(n) \cdot [q_{ij}(n)]^+, \mu_{ij}(n)\right\}$$.
• Assigns each link a quadratic weight $$w_{ij}(n) = 2\phi_{ij}(n)q_{ij}(n)\widehat{f}_{ij}(n) - \widehat{f}_{ij}(n)^2$$.
• Selects a maximal set of conflict-free active links to maximize the total link weight.
• Forwards packets accordingly.

A single tunable parameter $\beta \in [0,1]$ interpolates between delay-optimal and cost-optimal behavior, thereby operating along the Pareto frontier of achievable $(\overline{Q}, \overline{R})$ pairs (expected delay and cost). In the fluid limit, system behavior converges to a nonlinear combinatorial heat equation, connecting network optimization to discrete potential theory.

This routing framework guarantees:

• Throughput-optimality for all feasible input rates.
• Minimal network-wide queuing delay within the class of policies using only instantaneous state.
• Pareto optimal trade-off between delay and quadratic path cost, adjustable via $\beta$ (Banirazi et al., 2019).

3. DAR in Mixture-of-Experts Diffusion Language and Vision Models

In diffusion-based generative models for sequence and image synthesis, DAR denotes computation allocation policies that exploit the global, non-causal nature of the denoising process. The dominant instantiation uses Expert-Choice (EC) routing in sparse Mixture-of-Experts (MoE) architectures, contrasting with the Token-Choice (TC) routing inherited from autoregressive models (Zhang et al., 2 Apr 2026, Sun et al., 2024):

• Token-Choice (TC) Routing: Each token independently selects its top-$k$ experts based on local scores, without coordination. This design suffers from severe expert load imbalance and requires auxiliary losses or hard static limits that degrade throughput and learning.
• Expert-Choice (EC) Routing: Each expert globally selects its top-$n$0 tokens to process, enabling deterministic per-expert load balancing. This approach fits diffusion models, as all tokens are present and available at every denoising step, and capacity $n$1 (number of tokens per expert) can itself be varied as a function of the timestep or mask schedule.

The DAR mechanism makes use of this external control to schedule expert capacity adaptively across the sequence of denoising timesteps, maximizing marginal learning efficiency by redistributing compute according to per-step learning dynamics.

Mathematical Formulation

At denoising step $n$2 with mask ratio $n$3, a scheduling function $n$4 determines capacity $n$5:

$n$6

where $n$7 may be:

• Linear-reverse: $n$8 (more compute at low mask ratios)
• Cosine-reverse or Gaussian variants

These schedules match empirical convergence profiles: early steps (low-mask, more context) learn faster; thus more expert slots yield the best reduction in perplexity per FLOP (Zhang et al., 2 Apr 2026).

4. DAR as Residual Aggregation in Diffusion Transformers

In state-of-the-art Diffusion Transformers used for visual generative modeling, DAR also denotes a cross-layer information aggregation mechanism replacing traditional residual addition (Xu et al., 20 May 2026). Instead of incremental, time-blind addition:

$n$9

DAR parameterizes the accumulation as a timestep-adaptive, learnable softmax over previous sources:

$q_{ij}(n) = q_i(n) - q_j(n)$0

with $q_{ij}(n) = q_i(n) - q_j(n)$1 the output of the $q_{ij}(n) = q_i(n) - q_j(n)$2th sublayer, $q_{ij}(n) = q_i(n) - q_j(n)$3 the query embedding for layer $q_{ij}(n) = q_i(n) - q_j(n)$4, and $q_{ij}(n) = q_i(n) - q_j(n)$5 the RMS-normalized key for source $q_{ij}(n) = q_i(n) - q_j(n)$6. Explicit injection of timestep information or dynamic queries parameterized by previous activations enable routing that is both depth- and time-aware, breaking the limitations of fixed skip connections and yielding faster convergence, reduced forward-activation growth, and greater representational diversity (Xu et al., 20 May 2026).

Algorithmically, DAR aggregates across a chunked cache of sublayer outputs, balancing representation richness and memory cost; efficient fused kernels accelerate this computation at scale.

5. DAR in Large-Scale Vision-Language Diffusion (EC-DIT)

In vision applications, EC-DIT applies DAR via adaptive expert-choice MoE routing within transformer-based diffusion models for text-to-image generation (Sun et al., 2024). Each even-numbered FFN is replaced by an MoE layer routing tokens to experts based on a softmax affinity over fused multimodal features. Experts select their own top $q_{ij}(n) = q_i(n) - q_j(n)$7 tokens globally, ensuring perfect computational balancing.

Empirical evidence shows that EC-DIT can scale diffusion transformers to 97 billion parameters (with up to 8.3–13 B active per forward), maintaining high efficiency, interpretability, and marked gains both in text-image alignment (GenEval 71.68% at SOTA) and sample quality compared to equally sized dense models or TC-MoE baselines. The architecture assigns more compute to tokens reflecting higher multimodal complexity (text, fine detail), as revealed by routing heatmaps (Sun et al., 2024).

Key properties:

• No auxiliary balancing or entropy regularization loss required—load balancing is inherent.
• Allocation is dynamic, token-heterogeneous, and interpretable.

6. Empirical Outcomes and Practical Implications

Language and Vision Models (MoE DLMs and EC-DIT):

• EC routing ensures per-expert load balancing, doubling or more the throughput relative to TC, and eliminating compute “straggler” effects (Zhang et al., 2 Apr 2026, Sun et al., 2024).
• Timestep-adaptive scheduling (DAR) yields lower perplexity and higher downstream task accuracy under matched FLOPs, especially when allocating increased expert capacity to low-mask-ratio steps.
• Retrofitting TC-MoEs by simply replacing the router with EC (no expert/embedding retraining) results in faster convergence and improved accuracy in sequence and program synthesis tasks.
• In T2I diffusion, DAR-based MoEs maintain wall-clock efficiency within 20–28% of dense baselines, while substantially improving alignment and sample fidelity across scales up to $q_{ij}(n) = q_i(n) - q_j(n)$8100B parameters.

Transformer Residual Routing:

• Cross-layer DAR residual routes collapse forward activation inflation, mitigate gradient decay, and reduce block redundancy—conditions previously limiting efficient diffusion transformer scaling (Xu et al., 20 May 2026).
• Dynamic and explicit timestep-aware queries improve FID by $q_{ij}(n) = q_i(n) - q_j(n)$9 points, match quality in $\phi_{ij}(n)$0 fewer iterations, and can be stacked with other objectives (e.g., REPA) for further acceleration.

Wireless Networks:

• DAR/HD achieves throughput-optimality for all stabilizable arrivals, minimal delay among all queue-aware policies, and parameterized trade-off along the Pareto frontier of delay and cost.
• The fluid limit connects routing policy to solutions of a nonlinear combinatorial heat equation on the network graph, providing new analytical proof pathways and revealing the deep mathematical structure (Banirazi et al., 2019).

7. Limitations and Open Directions

• Scheduling functions in adaptive computation are currently hand-designed; integrating learned controllers (e.g., neural policies, reinforcement learning) for $\phi_{ij}(n)$1 optimization constitutes a potential enhancement (Zhang et al., 2 Apr 2026).
• Scaling EC-MoE routing and DAR aggregation beyond current model sizes is empirically untested; memory and kernel efficiency become bottlenecks (Xu et al., 20 May 2026).
• Extension of DAR from discrete masked diffusion (language, vision) to continuous or hybrid domains remains an open research avenue.
• In network settings, the complexity of max-weight scheduling may limit immediate deployment without approximation—strategies from back-pressure routing can be ported.

8. Summary Table: DAR Instantiations in Recent Research

Domain	DAR Mechanism	Load/Compute Adaptation
Wireless Nets	Quadratic “heat” routing	Adaptive via queue/cost, $\phi_{ij}(n)$2
DLMs (Text)	EC-MoE, variable $\phi_{ij}(n)$3	Timestep-aware expert capacity
Diffusion T2I	EC-DIT adaptive MoE	Token-heterogeneous per-affinity
DiT Residual	Timestep-adaptive routing	Cross-layer, time-aware mix

Detailed empirical and mathematical analyses confirm DAR as a general principle for resource-efficient, globally adaptive routing in both physical and computational networks, yielding significant gains in scalability, convergence, and performance (Banirazi et al., 2019, Zhang et al., 2 Apr 2026, Sun et al., 2024, Xu et al., 20 May 2026).