RouteDP: Decision Process Routing

• RouteDP is a family of decision processes that formalize routing and task allocation via explicit state representations, actions, and optimization criteria.
• It has been applied across domains—from DRL cognitive routing and selection-region protocols in mobile networks to dynamic programming in overlay networks and DTN routing under uncertainty.
• In LLM-based forecasting, RouteDP dynamically routes tasks to different model sizes based on task difficulty to optimize accuracy under a compute budget.

RouteDP is not a single standardized method. In the cited literature, the label is used for several distinct constructs that share a common formal tendency: routing or allocation is posed as an explicit decision problem with a state representation, an action space, and an optimization target. In networking, RouteDP appears as a Deep Reinforcement Learning-based cognitive routing process that emits link weights for shortest-path routing, as a selection-region protocol that maximizes expected density of progress in mobile ad hoc networks with directional antennas, as a constrained dynamic-programming policy for delay-sensitive overlay traffic, and as a Markov decision process for Delay-Tolerant Networks under uncertain contact plans. In LLM-based forecasting, RouteDP denotes a zero-shot router that sends only the hardest tasks to a larger model under a compute budget (Wu et al., 2020, Li et al., 2010, Singh et al., 2017, Raverta et al., 2021, Ashok et al., 13 Aug 2025).

1. Scope of the term

Across these works, RouteDP denotes different mathematical objects rather than a canonical protocol. The commonality is structural rather than procedural: each formulation specifies what is observed, what can be decided, and what criterion is optimized. The optimized quantity varies substantially across domains: average end-to-end delay in DRL-based cognitive routing, expected density of progress in random mobile ad hoc networks, average queue lengths under delay constraints in overlay control, delivery probability by deadline in DTNs, and forecast accuracy under a budget in context-aided forecasting (Wu et al., 2020, Li et al., 2010, Singh et al., 2017, Raverta et al., 2021, Ashok et al., 13 Aug 2025).

Usage of RouteDP	State/action abstraction	Optimization target
Cognitive routing	Traffic matrix; continuous link weights	Minimize average end-to-end delay
Selection-region routing	Relay geometry via $r_m$ and $\phi$	Maximize expected density of progress
Overlay routing	Queue state; ingress allocations	Meet delay bounds and minimize average delay
DTN routing	Time and copy distribution; wait/forward/replicate	Maximize delivery probability by deadline
Forecast routing	Difficulty score; assign task to small or large LLM	Maximize accuracy under budget

This multiplicity matters because the term can otherwise be misread as referring exclusively to packet routing. In the forecasting paper, for example, the “route” is a task-to-model assignment, not a network path (Ashok et al., 13 Aug 2025).

2. RouteDP as DRL-based cognitive routing

In "Towards Cognitive Routing based on Deep Reinforcement Learning" (Wu et al., 2020), RouteDP is instantiated as a DDPG-based routing decision process that realizes “cognitive routing.” Cognitive routing is defined as “a mechanism learned from historical data for optimal routing decision by considering the inference of network quality state.” The proposed cognitive controller combines inference of network state from monitored data, optimized decision-making that accounts for service quality, and continual learning from experience. This positions the method against conventional link-state routing such as OSPF, which selects shortest paths from static link weights and does not use dynamic quality information such as delay, jitter, or loss.

The routing problem is formalized as an MDP on a network $G=(V,E)$. The state $s^t$ is the $N \times N$ traffic matrix $TM^t=[d^t_{ij}]$, where $d^t_{ij}$ is the total traffic demand from node $i$ to node $j$ in time slot $t$. The action $\phi$0 is the collection of weight vectors $\phi$1 over all nodes, and the environment translates those weights into routing tables through a link-state shortest-path computation. The reward is the negative average packet delay in the slot,

$\phi$2

so the objective is single-objective latency minimization. The discount factor is $\phi$3.

The implementation uses RL4Net, a DRL-oriented simulator built atop ns-3. RL4Net includes a Metric Extractor, Computers that convert raw metrics into state and reward, an Action Operator, an Action Executor, an ns3Env wrapper, and an envInterface. The DDPG actor and critic are fully connected MLPs with ReLU activations. The actor has four layers with $\phi$4, $\phi$5, $\phi$6, $\phi$7, applies a softmax at the output, and scales by $\phi$8 to produce positive bounded link weights. The critic has three layers with $\phi$9, $G=(V,E)$0, $G=(V,E)$1. Training uses a replay buffer of capacity $G=(V,E)$2, starts after threshold $G=(V,E)$3, draws minibatches of size $G=(V,E)$4, and uses actor and critic learning rates of $G=(V,E)$5 and $G=(V,E)$6, respectively, with target update rate $G=(V,E)$7. Exploration is driven by Ornstein–Uhlenbeck noise with $G=(V,E)$8, $G=(V,E)$9, and $s^t$0. The Bellman target and critic loss are

$s^t$1

and the actor is updated with the deterministic policy gradient.

The experimental setting is intentionally small: a 4-node example topology, 5 Mbps link capacities on all links, and a 4.636 Mbps UDP flow with 1024-byte packets from $s^t$2 to $s^t$3, which drives link $s^t$4 into heavy load. Over 43,100 training steps, the critic loss decreases and stabilizes after approximately 15,000 steps; the actor “loss” stabilizes after approximately 2,000 steps; and the average end-to-end packet delay decreases during steps 1–4,000 before stabilizing around $s^t$5 ms. The trained DDPG-based cognitive routing algorithm yields the lowest average delay among DDPG, OSPF, and random-weight routing under the tested conditions.

The study is explicitly preliminary. It reports no generalization tests, no sensitivity analyses on unseen topologies or demand patterns, and no explicit safety mechanism beyond softmax-bounded actions and simulator-enforced feasibility. The paper therefore establishes feasibility rather than a production-ready routing stack. A plausible implication is that scaling the approach beyond small topologies would require larger state/action models and possibly hierarchical or multi-agent designs, as the paper itself notes.

3. RouteDP as route density of progress in directional mobile ad hoc routing

In "A Selection Region Based Routing Protocol for Random Mobile ad hoc Networks with Directional Antennas" (Li et al., 2010), RouteDP is tied to the maximization of expected density of progress, denoted $s^t$6, in a random mobile ad hoc network. The model assumes a homogeneous Poisson Point Process with node density $s^t$7, slotted ALOHA with transmit probability $s^t$8, Rayleigh fading, path-loss exponent $s^t$9, SINR threshold $N \times N$0, and an interference-limited regime with $N \times N$1. The antenna model is Directional Transmission and Omnidirectional Reception, with beamwidth $N \times N$2. Because only interferers whose transmit beams cover the receiver contribute, the effective interfering transmitter density is $N \times N$3.

The routing protocol defines a selection region as a sector of angle $N \times N$4 and inner radius $N \times N$5, oriented toward the final destination. At each hop, the next relay is the nearest receiver within that sector beyond the reference distance $N \times N$6. Formally, the selection region is

$N \times N$7

where $N \times N$8 is the destination direction. This construction eliminates the need for omnidirectional angular filtering and reduces practical relay-selection overhead.

The instantaneous density of progress is

$N \times N$9

where $TM^t=[d^t_{ij}]$0 is the hop success probability, $TM^t=[d^t_{ij}]$1 is hop length, and $TM^t=[d^t_{ij}]$2 is the angular deviation from the destination direction. Under the directional model,

$TM^t=[d^t_{ij}]$3

If receivers form a PPP with intensity $TM^t=[d^t_{ij}]$4, then the nearest-neighbor distance in the selection sector has CDF

$TM^t=[d^t_{ij}]$5

Averaging over distance and angle yields the closed-form expected density of progress

$TM^t=[d^t_{ij}]$6

with

$TM^t=[d^t_{ij}]$7

Optimization proceeds over $TM^t=[d^t_{ij}]$8 and $TM^t=[d^t_{ij}]$9. The paper derives an upper bound for the optimal reference distance from an inequality involving $d^t_{ij}$0, and it gives the jointly optimized reference distance as

$d^t_{ij}$1

A central result is that, when $d^t_{ij}$2 and $d^t_{ij}$3 are jointly optimized, the optimal transmission probability $d^t_{ij}$4 is a constant independent of $d^t_{ij}$5. Under $d^t_{ij}$6 and $d^t_{ij}$7 dB, the numerical optimum is $d^t_{ij}$8. The optimized reference distance scales as $d^t_{ij}$9 and $i$0, and the maximum expected density of progress scales as $i$1, where $i$2 is independent of $i$3.

Relative to omnidirectional selection-region routing, the directional scheme increases expected density of progress significantly and reduces relay-selection complexity because the candidate set size is reduced to approximately $i$4 and per-candidate angular filtering is eliminated. The result is a route-level notion of “progress” rather than an end-to-end delay optimizer. This distinction is important: the protocol optimizes local spatial advance under interference, not a global latency objective.

4. RouteDP as delay-constrained dynamic programming in overlay networks

In "Optimal Routing for Delay-Sensitive Traffic in Overlay Networks" (Singh et al., 2017), RouteDP is a constrained dynamic-programming policy for overlay traffic routed across an unknown underlay. The network evolves in discrete time, contains multiple flows $i$5, and distinguishes overlay nodes, which are controllable, from underlay nodes, which implement legacy scheduling and routing. Overlay control is exercised only at ingress links. The underlay assumptions are specific: static FIFO scheduling with fixed service share $i$6, randomized routing via probabilities $i$7, stochastic capacities $i$8, and stochastic arrivals. Delay-sensitive QoS is represented through mean queue-length constraints, relying on Little’s law.

The problem is posed as an average-cost CMDP. Let $i$9 denote the collection of per-flow link queues $j$0, and let the overlay actions $j$1 specify how many packets of flow $j$2 are injected into each outgoing ingress link. The long-run objective minimizes average delay cost

$j$3

with constraints

$j$4

For per-link delay budgets $j$5, the Lagrangian introduces link prices $j$6,

$j$7

Under the static $j$8 assumption, the problem decomposes by flow: each flow $j$9 solves an independent average-cost MDP with stage cost $t$0. The corresponding Poisson equation is

$t$1

The algorithm uses three timescales. On the fast timescale, each flow learns an average-cost DP policy by relative value iteration Q-learning. On the medium timescale, link prices are updated by stochastic subgradient ascent,

$t$2

On the slow timescale, the delay budget shares $t$3 evolve via replicator dynamics,

$t$4

with stochastic approximation and projection onto the simplex. Step sizes satisfy the separation $t$5 and $t$6, together with standard stochastic approximation summability conditions.

The main structural result is decentralized optimality under the stated underlay assumptions. The optimal policy factorizes as $t$7, so each source needs only its local per-flow queue state and the current link prices. This is a strong claim, but it is conditional: if the underlay were to use dynamic scheduling rather than static FIFO with fixed service shares, decentralized optimality would no longer be guaranteed in the same form.

The reported simulations compare the policy against Overlay BackPressure. In a 2-flow network, Overlay BackPressure has mean delay of approximately $t$8 units, whereas RouteDP achieves a bound as low as $t$9 units by tuning budgets and prices. In a 4-source network, Overlay BackPressure average queues are approximately $\phi$00, while RouteDP with target $\phi$01 converges within a few episodes and keeps queues below target. The paper also reports effective behavior under Markovian arrivals, though convergence becomes slower. The substantive contribution is therefore not merely dynamic routing, but dynamic routing coupled to adaptive dual prices and adaptive budget allocation.

5. RouteDP as MDP-based routing in Delay-Tolerant Networks under uncertain contact plans

In "Routing in Delay-Tolerant Networks under Uncertain Contact Plans" (Raverta et al., 2021), RouteDP is a routing-as-decision-process formulation for DTNs in which contact schedules are known only probabilistically. The network is represented as an Uncertain Time-Varying Graph

$\phi$02

where $\phi$03 is the underlying directed graph, $\phi$04 is the discrete time horizon, $\phi$05 is the per-slot edge failure probability, $\phi$06 is transmission delay, and $\phi$07 is failure-detection delay. This model occupies the intermediate regime between deterministic contact-plan routing and purely opportunistic routing.

The corresponding MDP encodes time and copy placement. A state includes the time index $\phi$08, a copy-distribution vector $\phi$09 with $\phi$10, and, when $\phi$11 or $\phi$12, availability times for copies in flight. Delivered states are those with at least one copy at the destination. Actions allow waiting, forwarding, or replication through sets of rules $\phi$13, where $\phi$14 is the number of copies sent along an admissible path $\phi$15. Transition probabilities enumerate contact-failure subsets: $\phi$16 The objective is finite-horizon reachability: maximize delivery probability by deadline $\phi$17. With terminal reward $\phi$18 on delivered states and $\phi$19 otherwise, the Bellman recursion is

$\phi$20

The paper develops three forms. RUCoP is the centralized optimal solver based on backward dynamic programming. L-RUCoP adapts the policy to local knowledge by precomputing, for each node, time slot, and local copy count, a local routing-table entry derived from “safe states.” CGR-UCoP embeds the same probabilistic reasoning into Contact Graph Routing by replacing the route-scoring criterion with a successful delivery probability metric. For a candidate partial route $\phi$21 in slot $\phi$22, the score combines the probability that all hops on $\phi$23 succeed with the probability that the $\phi$24-th hop is the first failure and rerouting begins after $\phi$25.

The worst-case complexity is exponential, stated as 2-EXPTIME, because the algorithm enumerates admissible paths, rule sets, joint actions, and all failure subsets. Nonetheless, the reported practical runtime is moderate for realistic DTNs such as LEO constellations: policies are computed in minutes to about an hour, including multiple-copy settings.

Empirically, RUCoP and L-RUCoP closely approach the delivery ratio of an oracle, and CGR-UCoP improves state-of-the-art DTN routing schemes’ delivery ratio by up to $\phi$26. The reported gains are $\phi$27 in random networks, $\phi$28 in RRN-A with inter-satellite links, and $\phi$29 in RRN-B without inter-satellite links. The method is especially suitable when contact plans are informative but imperfect, rerouting after failure detection is feasible, and purely opportunistic methods would waste schedule information.

6. RouteDP as task routing for zero-shot context-aided forecasting with LLMs

In "Beyond Naïve Prompting: Strategies for Improved Zero-shot Context-aided Forecasting with LLMs" (Ashok et al., 13 Aug 2025), RouteDP is a model-routing policy rather than a network-routing policy. The setting is context-aided forecasting on CiK, where each task $\phi$30 contains historical observations $\phi$31 and textual context $\phi$32, and the forecasting objective is to estimate $\phi$33. Evaluation uses average RCRPS, a region-of-interest CRPS that penalizes context violations. RouteDP targets the accuracy–cost tradeoff by keeping a small model $\phi$34 as the default forecaster, a large model $\phi$35 as the high-capability forecaster, and a router LLM $\phi$36 that estimates task difficulty.

The optimization target is explicitly budgeted: $\phi$37 Operationally, the budget is implemented by choosing $\phi$38 out of $\phi$39 tasks to send to the large model. Difficulty is estimated by constrained decoding on a binary prompt, and the ranking score is

$\phi$40

Tasks are sorted by $\phi$41, and the top-$\phi$42 are routed to the large model: $\phi$43 The exact router prompt is the direct forecasting prompt followed by: $\phi$61

The main experiments use Qwen2.5 models as both small forecasters and routers, and Llama-3.1-405B-Inst as the large model. Forecasts are produced by Direct Prompting, with 25 samples per task to form probabilistic predictions. Random routing and ideal routing define lower and upper baselines for efficiency, and performance is summarized by average RCRPS versus $\phi$44 together with the “area captured” between random and ideal curves.

The smallest-model case is the clearest demonstration. With Qwen2.5-0.5B-Inst as both main model and router, routing only $\phi$45 of tasks ($\phi$46) to Llama-3.1-405B-Inst reduces average RCRPS from $\phi$47 to $\phi$48, a $\phi$49 reduction, and the router captures $\phi$50 of the area between random and ideal routing curves. At $\phi$51 routed, the average RCRPS drops further to $\phi$52; at $\phi$53 routed, performance matches the large-model ceiling of $\phi$54. Cross-router comparisons show that router choice matters: with the same 0.5B main model and $\phi$55 routing, a Qwen2.5-1.5B router yields $\phi$56, whereas a Qwen2.5-32B router yields $\phi$57, both worse than the 0.5B router. The paper identifies an “own-best-router” tendency, where the best router for a main model is often from the same family and size.

The gains diminish as the main model becomes stronger. For Qwen2.5-14B-Inst, the baseline is $\phi$58, the ceiling remains $\phi$59, and the best routers yield smaller improvements, reflecting reduced headroom. RouteDP therefore functions as an inference-time compute allocator. It does not change the forecasting prompt, does not require fine-tuning, and is fully zero-shot. Its central assumption is that estimated difficulty correlates with the marginal benefit of invoking the larger model.

7. Comparative interpretation

The various RouteDP usages are linked less by domain than by formal design. Each defines an explicit state, a constrained or structured action, and an optimization target. The DRL cognitive-routing formulation uses continuous actions over link weights and simulator-induced dynamics (Wu et al., 2020). The mobile ad hoc formulation uses geometric relay selection and closed-form stochastic-geometry optimization (Li et al., 2010). The overlay-network formulation uses CMDP decomposition, dual prices, and replicator dynamics (Singh et al., 2017). The DTN formulation uses finite-horizon reachability dynamic programming over uncertain contacts and replication states (Raverta et al., 2021). The forecasting formulation uses task ranking by $\phi$60 and budgeted assignment to a large or small LLM (Ashok et al., 13 Aug 2025).

Several misconceptions are therefore avoidable. RouteDP is not always a shortest-path algorithm; in the cognitive-routing work, the learned policy outputs link weights, and shortest-path computation is performed by the environment afterward (Wu et al., 2020). It is not always an end-to-end delay optimizer; in the directional ad hoc work, the optimized quantity is expected density of progress (Li et al., 2010). It is not always packet routing; in the forecasting work, it is task-to-model routing (Ashok et al., 13 Aug 2025). Nor is decentralized optimality universal; in the overlay CMDP setting, it depends on static FIFO scheduling with fixed service shares and randomized routing in the underlay (Singh et al., 2017). Likewise, the DTN gains depend on having uncertain contact plans known a priori rather than purely opportunistic connectivity (Raverta et al., 2021).

This suggests that “RouteDP” is best understood as a family resemblance across decision-theoretic routing and allocation problems rather than a single algorithmic lineage. What unifies the term in practice is not implementation detail but the elevation of routing or assignment to an explicitly optimized decision process with measurable performance criteria and clearly defined uncertainty models.