Calibrated Graph RL Router

• The paper introduces a calibration-aware graph reinforcement-learning router that integrates uncertainty estimates to enhance decision making in dynamic network environments.
• The methodology combines graph neural networks and reinforcement learning to adaptively select routes using topology, telemetry, and risk-sensitive calibration.
• Empirical results show throughput gains up to 7.8% over conventional SDN methods, with improved delay performance and adaptability in optical and quantum routing.

Calibration-Aware Graph Reinforcement-Learning Router denotes a class of routing systems in which the decision problem is represented on a graph, routing actions are optimized by reinforcement learning, and confidence calibration or uncertainty estimates are incorporated to moderate exploration, fallback, and risk-sensitive control. In packet networks, this design is exemplified by a DGCNN-DQN controller for adaptive SDN routing that learns from topology, link attributes, and node telemetry without labeled data; in quantum circuit routing, a graph policy uses same-day hardware calibration to choose SWAPs; and in packet-optical and optical LEO settings, telemetry-derived or physics-derived feasibility information is embedded directly into the action graph or its masks (Abrol et al., 2024, Tomar et al., 11 Jun 2026, Navarro et al., 2024, Dabiri et al., 23 Dec 2025).

1. Conceptual scope and motivation

A calibration-aware graph RL router arises from two converging requirements. The first is structural: routing problems are naturally graph-valued, whether the nodes are routers and switches, physical qubits, or satellites, and whether the edges are physical links, hardware couplers, or feasible inter-satellite links. The second is operational: in dynamic environments, static shortest-path or hand-crafted heuristics can be systematically misaligned with current conditions. In next-gen programmable networks, OSPF always prefers the static shortest path and ignores time-varying link loads, leading to congestion and higher delay under dynamic conditions; hand-crafted heuristics also require unrealistic assumptions such as static traffic matrices and do not generalize across topologies or traffic regimes (Abrol et al., 2024).

The “calibration-aware” qualifier has multiple technical meanings in this literature. In the SDN routing formulation, it refers to explicit confidence calibration for action probabilities or Q-values so that overconfident decisions on congested or failing links can be detected, corrected, or overridden. In quantum routing, it refers to embedding same-day hardware calibration—two-qubit error, one-qubit error, readout error, and coherence—directly into the policy input, so that routes are chosen by expected fidelity rather than by gate count alone. In packet-optical and optical LEO routing, it refers to telemetry- or physics-grounded thresholds, such as BER-based penalties or jitter-aware feasible-range masks, that define which actions are trustworthy or even admissible (Abrol et al., 2024, Tomar et al., 11 Jun 2026, Navarro et al., 2024, Dabiri et al., 23 Dec 2025).

This scope also clarifies a common misconception. Graph reinforcement learning and calibration are related but distinct layers: graph structure supplies topology-aware representation learning, whereas calibration governs the reliability of the router’s action preferences, value estimates, or feasibility masks. A graph policy can therefore be high-performing on average while still being miscalibrated under drift or failure.

2. Markov decision formulations on graph-structured routing problems

In the adaptive SDN formulation, routing is cast as an MDP with graph-centric state

$s = (S^{link}, S^{node}),$

where

$S^{link} = D^{-1/2} W' D^{-1/2},$

$W$ is the adjacency matrix with entries $w_{i,j} \in \{0,1\}$, $W' = W + I$ adds self-loops, and $D$ is the diagonal degree matrix of $W'$. Each node feature vector is

$s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$

where $D_i$ is node degree, $I_i$ is centrality/importance, $S^{link} = D^{-1/2} W' D^{-1/2},$0 are aggregated egress/ingress rates, $S^{link} = D^{-1/2} W' D^{-1/2},$1 are one-hot context of the current request, and $S^{link} = D^{-1/2} W' D^{-1/2},$2 is the requested data rate for the current flow; all features are normalized. The action is per-flow path selection over precomputed $S^{link} = D^{-1/2} W' D^{-1/2},$3 loop-free shortest paths between $S^{link} = D^{-1/2} W' D^{-1/2},$4,

$S^{link} = D^{-1/2} W' D^{-1/2},$5

with the selected path deployed by $S^{link} = D^{-1/2} W' D^{-1/2},$6 or split if supported. The reward is

$S^{link} = D^{-1/2} W' D^{-1/2},$7

where $S^{link} = D^{-1/2} W' D^{-1/2},$8 is achieved throughput on the selected path, $S^{link} = D^{-1/2} W' D^{-1/2},$9 is the requested rate, and $W$0 is end-to-end delay. A time step corresponds to a new flow request $W$1, and the environment records congestion by a flag $W$2 when an action exceeds link capacity (Abrol et al., 2024).

Alternative graph-RL formulations modify the action granularity rather than the graph semantics. In GDDR, the network is a directed graph with capacities $W$3, traffic is represented by demand matrices $W$4, and the objective is to minimize worst-case utilization

$W$5

The non-iterative policy outputs a vector of edge weights, while the iterative policy outputs one edge weight per step and a routing temperature $W$6, after which routing is derived by modified softmin with DAG pruning. The reward is computed relative to LP-optimal routing:

$W$7

This formulation is destination-based and naturally emphasizes topology transfer across graphs (Hope et al., 2021).

Packet-optical routing adopts a tabular next-hop MDP with state $W$8, deterministic transition to the chosen neighbor, and telemetry-driven reward

$W$9

with

$w_{i,j} \in \{0,1\}$0

This preserves a graph routing interpretation, but calibration enters through hybrid telemetry rather than through a graph neural architecture (Navarro et al., 2024).

3. Graph encoders, policies, and value functions

The canonical packet-network architecture combines deep graph convolution with Deep Q-Learning. The DGCNN stack uses graph convolution layers with hidden sizes $w_{i,j} \in \{0,1\}$1 and ReLU activation, followed by SortPooling with pool size equal to the number of nodes, then 1D convolution layers $w_{i,j} \in \{0,1\}$2, MaxPooling, a fully connected layer $w_{i,j} \in \{0,1\}$3, and a softmax output over the $w_{i,j} \in \{0,1\}$4 candidate paths. Propagation follows

$w_{i,j} \in \{0,1\}$5

where $w_{i,j} \in \{0,1\}$6, $w_{i,j} \in \{0,1\}$7 is the degree matrix of $w_{i,j} \in \{0,1\}$8, $w_{i,j} \in \{0,1\}$9, and $W' = W + I$0 is ReLU (Abrol et al., 2024).

The associated Deep Q-Learning procedure uses $W' = W + I$1-greedy exploration with exponential decay, initial $W' = W + I$2, decay $W' = W + I$3 per episode, and minimum $W' = W + I$4. It also implements adaptive re-exploration: if the reward drifts outside $W' = W + I$5, where $W' = W + I$6 is the mean reward of the last $W' = W + I$7 exploitation steps, $W' = W + I$8 is reset to $W' = W + I$9. Replay is prioritized with buffer size $D$0, batch size $D$1, and priority

$D$2

where $D$3. Double-network stabilization copies the target network every $D$4 flows. The Bellman target is

$D$5

with $D$6 if the chosen path exceeds link capacity, and the loss is

$D$7

The paper reports learning rate $D$8, discount $D$9, Xavier initialization, and a softmax final layer (Abrol et al., 2024).

Other graph-RL routers instantiate the same design principle with different message-passing operators. GDDR uses the Battaglia et al. Graph Network encode–process–decode architecture, with permutation-invariant aggregation and modified softmin routing. Quantum circuit routing uses a graph neural policy with two message-passing layers of hidden dimension $W'$0, then an MLP that scores legal edges via $W'$1, and PPO training with AdamW, learning rate $W'$2, $W'$3, GAE $W'$4, clipping $W'$5, and entropy coefficient $W'$6 with floor $W'$7 (Hope et al., 2021, Tomar et al., 11 Jun 2026).

This architectural variation matters for transfer. GDDR reports that GNN policies perform at least as well as MLP baselines while enabling generalization to different network topologies with no extra work, whereas fixed-output architectures tied to a specific node set can be less topology-agnostic (Hope et al., 2021).

4. Calibration, uncertainty, and risk-sensitive routing

In the packet-network DGCNN-DQN design, calibration is introduced because miscalibrated Q-values or action probabilities can produce overconfident decisions on congested or failing links, harming SLA. The standard calibration metric is Expected Calibration Error,

$W'$8

where $W'$9 are bins of predictions by confidence, $s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$0 is empirical accuracy, and $s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$1 is predicted confidence. If Q-values are mapped to action probabilities, NLL and Brier score can also be evaluated (Abrol et al., 2024).

The basic post-hoc mechanism is temperature scaling on logits:

$s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$2

with $s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$3 softening overconfident distributions and $s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$4 chosen by minimizing NLL on held-out data or on a sliding validation window from recent episodes. Uncertainty in $s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$5 can then be estimated by Monte Carlo dropout, deep ensembles, Bootstrapped DQN, or distributional RL such as Quantile/QR-DQN (Abrol et al., 2024).

These uncertainty estimates directly modify routing rules. For reliability-oriented pessimism,

$s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$6

where $s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$7 and $s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$8 is tuned by SLA. For exploration-oriented optimism,

$s_i = [D_i, I_i, tx_i, rx_i, src_i, dest_i, bw_i],$9

with $D_i$0 decreasing over training. A constrained RL variant formulates SLA control as

$D_i$1

using the Lagrangian

$D_i$2

Operationally, uncertainty-gated $D_i$3 reduces exploration when calibrated confidence is high and increases it when ECE or predictive entropy is high, while confidence-aware fallback reverts to a conservative path or to OSPF if $D_i$4 or uncertainty exceeds threshold (Abrol et al., 2024).

Across adjacent domains, calibration is also encoded as structured side information rather than only as post-hoc probability correction. Quantum routing embeds a calibration snapshot $D_i$5 consisting of $D_i$6, $D_i$7, $D_i$8, and a coherence proxy $D_i$9, and evaluates routes by exact simulated fidelity

$I_i$0

Packet-optical routing calibrates BER thresholds from historical data and smooths telemetry with EWMA. Optical LEO routing computes class-dependent feasible ranges $I_i$1 and $I_i$2 from a jitter-aware Gaussian-beam model and enforces them through feasible-action masking (Tomar et al., 11 Jun 2026, Navarro et al., 2024, Dabiri et al., 23 Dec 2025).

A plausible implication is that calibration-aware routing is not a single algorithmic add-on but a family of mechanisms that make graph-RL decisions commensurate with measured reliability, telemetry uncertainty, or hardware quality.

5. Router operation, deployment, and safety mechanisms

In the SDN deployment model, the controller receives northbound route requests with $I_i$3, ingests streaming link utilization, queue lengths, per-link delay, per-node $I_i$4, and topology changes, and assembles $I_i$5 and $I_i$6. The action set is defined by $I_i$7-shortest simple paths precomputed offline per topology snapshot, and the controller installs flow rules through OpenFlow, P4Runtime, or gNMI. On each flow request, the router gathers telemetry, computes logits $I_i$8, applies $I_i$9-greedy or stochastic routing, installs the path, observes reward and next state, and writes the transition into prioritized replay (Abrol et al., 2024).

The resulting inference complexity is approximately the sum of graph-convolution costs $S^{link} = D^{-1/2} W' D^{-1/2},$00, SortPooling $S^{link} = D^{-1/2} W' D^{-1/2},$01, and small 1D-convolution and fully connected costs. For networks of tens to hundreds of nodes and links, the reported inference time fits sub-10–50 ms on a modern CPU/GPU in an SDN controller, and batching multiple flows amortizes cost (Abrol et al., 2024).

Update cadence is equally part of the router design. In the SDN version, online training updates occur every batch via prioritized replay, the target network is synchronized every $S^{link} = D^{-1/2} W' D^{-1/2},$02 flows, temperature $S^{link} = D^{-1/2} W' D^{-1/2},$03 can be recalibrated periodically on a sliding validation window, and ensemble uncertainty statistics can be refreshed continuously. Packet-optical deployments add hysteresis and hold-down timers to prevent route flapping, while LEO deployments rely on per-hop feasible-action masking rather than global recomputation (Abrol et al., 2024, Navarro et al., 2024, Dabiri et al., 23 Dec 2025).

Safety layers are explicit. Loop prevention is enforced by restricting the action space to loop-free candidate paths or by DAG pruning. Capacity adherence is implemented by pre-admission checks against residual link capacities and by penalties or prohibitions when constraints are violated. Low-confidence fallback can route via OSPF or a conservative traffic-engineering policy, and the design explicitly allows rapid re-exploration and human-in-the-loop alerts when calibrated confidence is low (Abrol et al., 2024).

6. Empirical performance, trade-offs, and open directions

The packet-network DGCNN-DQN router was evaluated on a random $S^{link} = D^{-1/2} W' D^{-1/2},$04-node, $S^{link} = D^{-1/2} W' D^{-1/2},$05-link graph and NSFNET with all link capacities $S^{link} = D^{-1/2} W' D^{-1/2},$06 Mbps and link distances in $S^{link} = D^{-1/2} W' D^{-1/2},$07 m. With training flows $S^{link} = D^{-1/2} W' D^{-1/2},$08 Mbps and concurrent flows in $S^{link} = D^{-1/2} W' D^{-1/2},$09, it increased throughput versus OSPF by $S^{link} = D^{-1/2} W' D^{-1/2},$10 on the random topology and $S^{link} = D^{-1/2} W' D^{-1/2},$11 on NSFNET, and reduced delay by $S^{link} = D^{-1/2} W' D^{-1/2},$12 and $S^{link} = D^{-1/2} W' D^{-1/2},$13, respectively. Versus a DRL-MLP baseline, the throughput gains were $S^{link} = D^{-1/2} W' D^{-1/2},$14 and $S^{link} = D^{-1/2} W' D^{-1/2},$15, while delays were reduced by $S^{link} = D^{-1/2} W' D^{-1/2},$16 and $S^{link} = D^{-1/2} W' D^{-1/2},$17. Under traffic regime changes, the router re-entered exploration and required approximately $S^{link} = D^{-1/2} W' D^{-1/2},$18 episodes to adapt (Abrol et al., 2024).

Topology-transfer results in intradomain traffic engineering point in the same direction. GDDR reports that GNN-based policies perform at least as well as MLP architectures and that the iterative GNN policy is consistently better under topology change, which supports the claim that graph-structured policies are not merely parameter-efficient but structurally aligned with routing generalization (Hope et al., 2021).

Calibration-aware routing also changes what counts as success. In quantum circuit routing, pooled mean exact fidelity across nine MQT Bench circuits and three calibration snapshots is $S^{link} = D^{-1/2} W' D^{-1/2},$19 for the graph RL router, compared with $S^{link} = D^{-1/2} W' D^{-1/2},$20 for SABRE-best20 and $S^{link} = D^{-1/2} W' D^{-1/2},$21 for target-aware SABRE. These gains come with higher routed two-qubit count and depth, and all $S^{link} = D^{-1/2} W' D^{-1/2},$22 families favor SABRE-best20 under the fixed tree action graph. The result is therefore not “fewer operations,” but higher calibrated hardware fidelity under the available action graph (Tomar et al., 11 Jun 2026).

In optical LEO routing, the reported routes maintain end-to-end delay of $S^{link} = D^{-1/2} W' D^{-1/2},$23–$S^{link} = D^{-1/2} W' D^{-1/2},$24 ms with $S^{link} = D^{-1/2} W' D^{-1/2},$25–$S^{link} = D^{-1/2} W' D^{-1/2},$26 hops, while feasible-action masking eliminates physically infeasible or congestion-blocked links by construction. In fixed-grid RWA-LR with graph attention and PPO, the best RL agent exceeds the previous state-of-the-art RL approach by $S^{link} = D^{-1/2} W' D^{-1/2},$27 and the best heuristic by about $S^{link} = D^{-1/2} W' D^{-1/2},$28, but the paper emphasizes that these are marginal gains in a long-horizon resource-allocation problem (Dabiri et al., 23 Dec 2025, Doherty et al., 20 Feb 2025).

The limitations are correspondingly concrete. The adaptive SDN router has no explicit confidence calibration in its base form, remains single-agent and single-commodity, does not fully explore fairness or robustness under failures, and exhibits convergence times on the order of hundreds of episodes, dependent on telemetry cadence. Quantum routing can overfit to specific calibration snapshots and is constrained by a fixed tree action graph. This suggests that future work is likely to focus on empirical validation of calibration metrics such as ECE, NLL, and Brier; distributional DGCNN-DQN or bootstrapped heads for return uncertainty; multi-agent coordination for multi-commodity flows and fairness; robustness to topology dynamics with domain randomization and adversarial training; and real testbed integration with P4 INT or similar telemetry stacks (Abrol et al., 2024, Tomar et al., 11 Jun 2026).

A calibration-aware graph reinforcement-learning router is therefore best understood not as a single standardized architecture, but as a research program: graph-native routing policies supply topology-aware control, while calibration mechanisms determine when those policies should be trusted, regularized, or overridden.