PAGU: Policy-Adaptive Graph Update in TAGRL

• The paper introduces a retain/add mechanism that uses policy behavior shifts and reward feedback to dynamically update the graph's adjacency structure.
• PAGU is a graph adaptation method that mitigates the rigidity of static topologies by adjusting edges in response to real-time policy signals in cloud environments.
• Empirical evidence shows that PAGU reduces maximum link utilization and improves throughput by dynamically balancing network routes.

Policy-Adaptive Graph Update (PAGU) is a graph adaptation mechanism in which the graph structure used by a learning system is modified according to policy behavior and task feedback. In the explicit formulation currently associated with the term, PAGU appears as a module within Topology-Aware Graph Reinforcement Learning (TAGRL) for cloud-network routing, where it updates the edge set and adjacency matrix on the basis of policy behavior shifts, reward feedback, edge scheduling frequency, and candidate-edge policy relevance (Wang et al., 5 Sep 2025). More broadly, the term also describes a recognizable methodological family in which graph updates are treated as adaptive, learned, and state-contingent rather than fixed; in that broader sense, earlier work on policy-controlled message passing, adaptive propagation depth, learned recurrent graph dynamics, and policy-driven neighborhood selection can be read as precursors or partial instances of PAGU-like design (Deng et al., 2019).

1. Formal definition and scope

In TAGRL, PAGU is defined as the mechanism that “dynamically adjusts the adjacency structure of the network graph in response to real-time feedback” by “monitoring behavioral deviations and reward signals from the policy network” (Wang et al., 5 Sep 2025). Its stated purpose is to address two weaknesses attributed to static graph modeling in dynamic cloud environments: insufficient structural awareness under evolving topologies and decision instability when topology or traffic changes frequently. PAGU therefore updates the model’s internal graph topology representation rather than assuming a fixed adjacency throughout training and decision-making.

The graph under PAGU is written as

$G_t = (V, E_t),$

where the node set $V$ remains unchanged while the edge set $E_t$ is adapted over time. The mechanism returns an updated adjacency matrix $A_{t+1}$ to the structure encoder and environment module, thereby completing what the paper calls the “periodic reconstruction of the graph state” (Wang et al., 5 Sep 2025). This establishes PAGU as an edge-set update mechanism rather than merely an attention layer or edge-weight reweighting procedure.

A narrower and a broader usage can be distinguished. In the narrow sense, PAGU denotes the explicit threshold-based retain/add graph-update rule introduced in TAGRL (Wang et al., 5 Sep 2025). In the broader sense, the term can be used for graph-learning methods in which the update process itself is learned or policy-conditioned. This broader reading is not a formal definition from a single paper. Rather, it is supported by a sequence of related methods that adapt message passing, propagation depth, or effective neighborhood structure through learned controllers or policies (Deng et al., 2019).

2. Mechanism in the TAGRL formulation

Within TAGRL, PAGU operates after Structure-Aware State Encoding (SASE) and the policy network. The information flow is: $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$ SASE itself computes topology-aware node representations through graph convolution and structural positional embeddings: $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$ followed by concatenation with structural position embedding,

$z_i = [\, h_i^{(l)} \, \| \, p_i \,],$

and neighborhood aggregation

$s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$

Because PAGU changes the underlying graph, it changes $\hat A$, neighborhoods $\mathcal N(i)$, and consequently the state encodings supplied to the policy (Wang et al., 5 Sep 2025).

The first explicit PAGU signal is policy behavior deviation: $V$0 The paper states that this reflects the change between the current policy behavior and the behavior at the previous moment. The text does not specify whether $V$1 is used as a vector difference, scalar norm, or thresholded divergence, and the exact use of the reported action deviation threshold is not formalized in an equation (Wang et al., 5 Sep 2025).

For each existing edge $V$2, PAGU computes a reward-based importance score

$V$3

where $V$4 is the global reward and $V$5 is the scheduling frequency of the edge. The paper states that larger $V$6 implies greater contribution to the strategy and higher retention priority (Wang et al., 5 Sep 2025).

The edge set is then updated by threshold-based pruning and addition: $V$7 where $V$8 is the edge retention threshold, $V$9 is the edge introduction threshold, and $E_t$0 is the policy relevance score of a candidate edge. The resulting adjacency is

$E_t$1

This retain/add structure is the defining technical signature of PAGU in TAGRL (Wang et al., 5 Sep 2025).

The paper also provides concrete hyperparameters: action deviation threshold $E_t$2, edge retention threshold $E_t$3, and edge addition threshold $E_t$4. To preserve graph connectivity and convergence stability, the number of edges is constrained within

$E_t$5

where $E_t$6 is the initial edge set (Wang et al., 5 Sep 2025).

3. Conceptual lineage and related adaptive graph-update methods

Although the term PAGU is explicit in TAGRL, several earlier methods instantiate closely related principles. The most direct precursor is “Policy Message Passing” (Deng et al., 2019), which replaces fixed message aggregation with a learned stochastic policy over message-passing actions. Its core formulation,

$E_t$7

treats reasoning trajectories $E_t$8 as latent variables and allows each edge at each step to select a message type $E_t$9, including a no-message action. This makes graph-state evolution history-dependent and policy-controlled. The paper does not use the term PAGU, but it is explicitly described as a close conceptual match to policy-adaptive graph updating because graph-state transitions are chosen by a stochastic policy rather than a fixed aggregation schedule (Deng et al., 2019).

A second lineage comes from learned recurrent graph dynamics. “Deep Graphs” replaces hand-designed iterative update rules with a learned recurrent vertex-state transition

$A_{t+1}$0

and presents graph computation as adaptive learned update dynamics rather than fixed algorithms such as PageRank or Weisfeiler–Lehman (Platanios et al., 2018). Here the aggregation structure remains largely a sum and the number of update steps $A_{t+1}$1 is fixed, so the method is adaptive in the learned transition function but not in explicit action selection.

A third family adapts propagation depth rather than topology. “Policy-GNN” models the number of aggregation iterations as a node-wise action selected by a DQN-based meta-policy,

$A_{t+1}$2

with state $A_{t+1}$3 and reward based on validation-performance improvement (Lai et al., 2020). “Adaptive Propagation GCN” instead uses a differentiable halting mechanism,

$A_{t+1}$4

to adapt the number of propagation steps independently at each node (Spinelli et al., 2020). Neither method rewires edges, but both embody PAGU-like control over where graph-state updating stops.

A fourth line concerns adaptive effective neighborhoods. “GPS: A Policy-driven Sampling Approach for Graph Representation Learning” computes layer-wise adaptive correlations and applies a row-wise top-$A_{t+1}$5 policy,

$A_{t+1}$6

thereby updating the active local propagation graph without changing the stored global adjacency (Zhang et al., 2021). This suggests a local PAGU interpretation in which the policy updates the working subgraph rather than the canonical graph.

These precursors reveal a stable conceptual pattern: fixed graph computation is progressively replaced by learned control over message types, propagation depth, or active neighborhoods. TAGRL’s PAGU differs in one decisive respect: it explicitly updates the adjacency structure itself through retain/add edge operations (Wang et al., 5 Sep 2025).

4. Signals, control loop, and operational interpretation

PAGU is tightly coupled to the reinforcement-learning loop. Policy behavior shift serves as an update trigger signal, while reward feedback determines which current edges appear useful for future decision-making (Wang et al., 5 Sep 2025). This yields a closed loop in which graph structure influences state encoding, state encoding influences policy decisions, and policy outcomes in turn influence the next graph.

A central operational feature is that PAGU modifies the graph used by representation learning, not the physical network itself. The updated adjacency matrix is returned to SASE and the environment module, which means that graph convolution paths, structural aggregation, and state representations are all recalculated under the new topology (Wang et al., 5 Sep 2025). This distinguishes PAGU from methods that only add an attention layer on top of a fixed graph.

The mechanism can therefore be interpreted as adaptive topology selection under policy feedback. Existing edges are retained when their reward-weighted scheduling frequency is sufficiently high, while non-edges may be added when their policy relevance score exceeds the addition threshold. This suggests that PAGU is designed to preserve edges associated with successful routing behavior and to admit new edges that become relevant under changing policy preferences. The paper explicitly supports the retain/add logic; the stronger interpretation in terms of strategic topology refinement is an inference from that mechanism (Wang et al., 5 Sep 2025).

A common misconception is to treat PAGU as simple edge weighting. The published formulation does not support that reading. The update acts on the edge set $A_{t+1}$7 and produces a new adjacency matrix $A_{t+1}$8, so the connectivity pattern itself changes (Wang et al., 5 Sep 2025). Another misconception is to assume that PAGU is a fully specified graph-rewiring framework. The candidate-edge policy relevance score $A_{t+1}$9, the exact use of the action deviation threshold, and the precise update schedule remain unspecified in the paper. Thus the conceptual mechanism is explicit, but the full operationalization is incomplete (Wang et al., 5 Sep 2025).

5. Empirical evidence

The strongest direct empirical evidence comes from the TAGRL experiments on the GEANT topology dataset. Against representative baselines, the full model reports the following averages: GDDR $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$0 throughput, $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$1 latency, $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$2 max link utilisation, $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$3 reward; DeepCQ+ $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$4, $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$5, $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$6, $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$7; CFR-RL $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$8, $$\text{Current graph } G_t \rightarrow \text{SASE} \rightarrow \text{policy} \rightarrow \text{action/reward} \rightarrow \text{PAGU} \rightarrow G_{t+1}.$$9, $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$0, $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$1; GRL-TE $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$2, $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$3, $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$4, $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$5; and the proposed model $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$6, $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$7, $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$8, $$H^{(l+1)} = \sigma \!\left( \hat{D}^{-1/2}\hat{A}\hat{D}^{-1/2} H^{(l)} W^{(l)} \right),$$9 (Wang et al., 5 Sep 2025). The paper states that the model reduces maximum link utilization by nearly 10 percentage points through PAGU.

The ablation study is especially informative because it isolates PAGU. Relative to the baseline $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$0 throughput, $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$1 latency, $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$2 max link utilisation, $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$3 reward, the $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$4PAGU variant reaches $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$5, $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$6, $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$7, $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$8. The paper summarizes this as showing that PAGU “enhances link balance through dynamic graph adjustments” (Wang et al., 5 Sep 2025). Numerically, the most visible isolated gain is indeed the drop in maximum link utilization from $z_i = [\, h_i^{(l)} \, \| \, p_i \,],$9 to $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$0.

Indirect support for PAGU-like ideas appears in earlier adaptive graph methods. Policy Message Passing reports large gains on reasoning-heavy and noisy-graph tasks, including Cora under added noisy edges: at $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$1 noise, GCN $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$2, GAT $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$3, PMP $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$4; at $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$5 noise, GCN $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$6, GAT $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$7, PMP $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$8; at $s_i = \sum_{j \in \mathcal{N}(i)} a_{ij} z_j.$9 noise, GCN $\hat A$0, GAT $\hat A$1, PMP $\hat A$2 (Deng et al., 2019). Because PMP controls effective message flow rather than adjacency, these results do not validate PAGU in the narrow TAGRL sense; however, they do support the broader claim that policy-conditioned graph updating can improve robustness under ambiguity and corrupted connectivity.

Further adjacent evidence comes from modality-structured graph learning. MANAGER’s ablations show that removing graph structure or replacing semantic sparsity with a full graph degrades both F1 and MSE on monetary-policy conference-call prediction tasks. For example, on Stock Index (Small) movement, MANAGER reports $\hat A$3 versus $\hat A$4 for w/o-Graph; on Currency Exchange Rate volatility, MANAGER reports $\hat A$5 versus $\hat A$6 for w/o-Graph (Ouyang et al., 2024). This does not constitute evidence for policy-conditioned adjacency updating, but it does indicate that selective graph structure is materially important.

6. Limitations, ambiguities, and open directions

The most immediate limitation is underspecification. TAGRL does not provide the formula for the candidate-edge policy relevance score $\hat A$7, does not mathematically define how the action deviation threshold $\hat A$8 is applied, and does not fully specify whether graph updates occur every step, every episode, or only under certain trigger conditions (Wang et al., 5 Sep 2025). As a result, the retain/add mechanism is clear, but faithful reimplementation still requires additional design choices.

A second limitation is the absence of formal complexity and convergence analysis for PAGU itself. The paper attributes improved “overall convergence efficiency” to the mechanism and includes a graph-size stability constraint to avoid excessive graph changes, but it does not provide a formal complexity bound or convergence theorem (Wang et al., 5 Sep 2025). This contrasts with neighboring literatures in which either update cost is analyzed explicitly, as in Deep Graphs with $\hat A$9 per stage (Platanios et al., 2018), or adaptive updates are given stability certificates, as in incremental policy-gradient adaptation for unknown time-varying systems (Zhao et al., 3 Jul 2026).

A third issue concerns the meaning of “policy-adaptive” in the literature. Some methods adapt learned recurrent transitions, some adapt propagation depth, some adapt effective neighborhoods, and some update the adjacency itself. Treating all of them as PAGU would obscure important distinctions. The most precise taxonomy supported by published work is therefore layered: explicit PAGU refers to policy-feedback-driven adjacency updating in TAGRL (Wang et al., 5 Sep 2025); broader PAGU-like methods include policy-controlled message passing, adaptive stopping, and policy-driven neighborhood selection (Deng et al., 2019).

An open direction suggested by the available papers is to integrate the narrow and broad senses. One plausible implication is that future PAGU systems may combine explicit adjacency updates with trajectory-based message control, node-wise adaptive stopping, or local top-$\mathcal N(i)$0 policy sampling. Another plausible implication is that graph adaptation may benefit from incremental, bounded updates rather than abrupt one-shot rewiring, because related adaptive-control work argues that one-step policy updates naturally limit variations caused by noisy model estimates (Zhao et al., 3 Jul 2026). These implications are consistent with the cited literature, but they remain extrapolations rather than established properties of PAGU itself.

In summary, PAGU denotes an explicit graph-topology adaptation mechanism in which policy feedback and reward signals govern retain/add updates of the working adjacency matrix (Wang et al., 5 Sep 2025). It also sits within a wider methodological shift in graph learning: inference schedules, propagation depth, and active neighborhoods are increasingly treated as learned control objects rather than fixed design choices (Deng et al., 2019). The explicit TAGRL formulation gives PAGU its clearest technical identity; the surrounding literature explains why such a mechanism emerged and what broader adaptive graph-computation agenda it belongs to.