GNARL: Graph Neural Algorithmic Reasoning

• GNARL is a framework that reimagines neural algorithmic reasoning by formulating graph algorithms as a sequential MDP, enabling stepwise valid solution construction.
• It integrates imitation learning from expert trajectories with reinforcement learning (PPO) to handle tasks where direct supervision is unavailable.
• Empirical results on tasks like BFS, DFS, TSP, and MVC demonstrate GNARL's enhanced graph accuracy and reliability over traditional NAR methods.

Searching arXiv for the GNARL framework paper and closely related background on neural algorithmic reasoning. {"query":"GNARL framework Graph Neural Algorithmic Reasoning reimagined through Reinforcement Learning arXiv (Schutz et al., 23 Sep 2025)", "max_results": 5} GNARL is a framework for Graph Neural Algorithmic Reasoning reimagined through Reinforcement Learning, introduced as a reformulation of algorithm learning on graphs from supervised trajectory prediction into sequential decision making in a Markov Decision Process (MDP) (Schutz et al., 23 Sep 2025). It is designed for settings in which a model must construct graph solutions step by step, rather than merely predict node- or edge-level labels. In this formulation, GNARL combines imitation learning (IL) when expert trajectories exist and reinforcement learning (RL) when they do not, while using action masking and structured state transitions to obtain valid-by-construction solution trajectories (Schutz et al., 23 Sep 2025).

1. Motivation and conceptual position

GNARL is motivated by three limitations attributed to standard Neural Algorithmic Reasoning (NAR). First, standard NAR predicts intermediate or output labels but may produce globally inconsistent results, often requiring post-processing, beam search, or other repair procedures. Second, existing NAR methods for NP-hard tasks are described as highly specialized, problem-specific, and engineering-heavy. Third, standard NAR depends on an expert algorithm capable of generating hints, which prevents direct application to problems for which no strong algorithm is available (Schutz et al., 23 Sep 2025).

The framework’s central move is to reinterpret algorithm execution as an MDP, thereby treating algorithmic reasoning as a sequential control problem rather than a pure supervised prediction problem. This makes it possible to use behavioral cloning from expert action distributions where such supervision exists, or PPO when it does not, while preserving a common graph-based execution model (Schutz et al., 23 Sep 2025).

A common misconception is to view GNARL as a minor variant of NAR. The paper instead presents it as a categorical shift from “predict the next hint/output” to “choose the next valid graph action in an MDP” (Schutz et al., 23 Sep 2025). Another possible confusion is terminological: GNARL, in this sense, is unrelated to the GNAR family of generalized network autoregressive models for network time series (Jiménez-Varón et al., 7 Oct 2025, Knight et al., 2019).

2. MDP formulation of graph algorithm learning

GNARL defines a generic MDP as

$\mathcal{M} = \langle S, A, T, R, h \rangle,$

with state space $S$, action space $A$, valid action set $A(s)\subseteq A$, transition function $T$, reward function $R$, and horizon $h$. A policy is

$\pi: S \to \Delta(A),$

and the optimal policy is

$$\pi^* = \arg\max_\pi \mathbb{E}\left[\sum_{t=1}^{h} R(s_t,a_t)\;\middle|\; a_t\sim \pi(\cdot\mid s_t)\right].$$

GNARL instantiates this abstract MDP on a graph $G=(V,E)$ to form a graph-algorithm MDP $S$0 (Schutz et al., 23 Sep 2025).

The state is decomposed as

$S$1

where $S$2 contains immutable input features and $S$3 contains mutable state features. This mirrors the distinction between CLRS input probes and hint probes, but with a key difference: GNARL encodes all input features at every step so that the Markov property holds even without recurrent latent state (Schutz et al., 23 Sep 2025).

The core action is typically selecting a node $S$4. Problems that are naturally edge-based or triangle-based are represented through multiple phases of node selection, tracked by a phase variable $S$5. The framework states that the maximum number of phases $S$6 depends on the problem: node-based algorithms use $S$7, edge-based algorithms use $S$8, and triangle-based algorithms use $S$9. Node state features $A$0 record which nodes were selected in each phase (Schutz et al., 23 Sep 2025).

The transition function is deterministic in the described applications: after an action is chosen, the environment updates the mutable state features according to the algorithm’s internal rule. When the objective is a terminal criterion $A$1, GNARL uses reward shaping

$A$2

which the paper explicitly ties to equivalence with terminal-objective optimization via the reward shaping theorem (Schutz et al., 23 Sep 2025).

This formulation has direct methodological consequences. It permits a fixed problem horizon $A$3, unlike NAR formulations in which the number of processor steps may depend on hints or a termination network. It also makes the notion of valid actions explicit, which underpins action masking and thus the framework’s emphasis on valid-by-construction trajectories (Schutz et al., 23 Sep 2025).

3. Encode-process-act architecture

GNARL replaces the standard NAR encode-process-decode pattern with encode-process-act (Schutz et al., 23 Sep 2025). The encoder separately linearly transforms each feature into a latent space of dimension $A$4, then aggregates them by location into node features $A$5, edge features $A$6, and graph features $A$7. Both input and state features are re-encoded at every step (Schutz et al., 23 Sep 2025).

The processor is a GNN $A$8 applying $A$9 rounds of message passing to produce node embeddings,

$A(s)\subseteq A$0

followed by a pooled graph embedding,

$A(s)\subseteq A$1

The implementations use modified MPNN or TripletMPNN variants in which latent recurrent embeddings from the previous step are removed, so the processor does not carry $A(s)\subseteq A$2 across iterations. The paper reports that this removal better aligns the architecture with the Markov interpretation (Schutz et al., 23 Sep 2025).

The actor produces a size-flexible distribution over nodes through a proto-action mechanism. First, the graph embedding is transformed,

$A(s)\subseteq A$3

then compared with each node embedding via

$A(s)\subseteq A$4

and normalized to yield the policy

$A(s)\subseteq A$5

When PPO is used, the critic is an MLP over the graph embedding that outputs a scalar state value (Schutz et al., 23 Sep 2025).

At inference time, evaluation uses greedy decoding, i.e. selecting the highest-probability valid action, while sampling is also supported and can generate multiple distinct solutions. Action masking excludes invalid actions outside $A(s)\subseteq A$6, which is the mechanism by which the framework enforces validity during rollout (Schutz et al., 23 Sep 2025).

4. Learning regimes and algorithmic scope

GNARL supports both IL and RL within the same formalism. For behavioral cloning from expert action distributions, the loss is

$A(s)\subseteq A$7

If only expert state-action pairs are available, the loss becomes

$A(s)\subseteq A$8

For RL, GNARL uses PPO with an actor-critic setup, and the critic minimizes

$A(s)\subseteq A$9

with $T$0 in all experiments (Schutz et al., 23 Sep 2025).

This dual training regime allows the same framework to span three cases. In the first, an expert algorithm exists and expert distributions can be used directly, as in the CLRS tasks. In the second, optimal or strong solutions exist but are expensive, allowing imitation from solvers such as Concorde or an ILP solver, as in TSP and MVC. In the third, no strong expert algorithm exists, so GNARL is trained directly with PPO, as in Robust Graph Construction (RGC) (Schutz et al., 23 Sep 2025).

The scope of applications described in the paper includes both classical algorithmic tasks and NP-hard combinatorial optimization. For CLRS-30, the evaluated tasks are BFS, DFS, Bellman-Ford, and MST-Prim. For NP-hard problems, the paper applies GNARL to Travelling Salesperson Problem (TSP), Minimum Vertex Cover (MVC), and Robust Graph Construction (RGC) (Schutz et al., 23 Sep 2025).

The treatment of multiple valid solutions is particularly notable. For CLRS tasks, expert policies are often multi-modal, assigning equal probability to all valid next actions in a state. The framework can therefore represent and sample from multiple correct continuations, rather than being tied to a single canonical trajectory. The paper formalizes temperature sampling as

$T$1

with $T$2 recovering greedy decoding and $T$3 approaching uniform random selection (Schutz et al., 23 Sep 2025).

5. Empirical results

On the CLRS-30 benchmark, the paper reports graph-level correctness at $T$4. For BFS, TripletMPNN and GNARL$T$5 both achieve $T$6. For DFS, TripletMPNN achieves $T$7 while GNARL$T$8 reaches $T$9. For Bellman-Ford, the corresponding figures are $R$0 and $R$1. For MST-Prim, they are $R$2 and $R$3 (Schutz et al., 23 Sep 2025). The paper emphasizes these results as evidence that GNARL improves graph accuracy / solution correctness, especially in tasks where node-level accuracy can obscure whether the final constructed solution is globally valid (Schutz et al., 23 Sep 2025).

For TSP, performance is measured as percentage above the optimum. The paper states that GNARL$R$4 and GNARL$R$5 both scale well, are close to the Christofides heuristic at moderate sizes, and outperform the earlier NAR method of Georgiev et al. on larger OOD graph sizes, despite using only 10% of their training data and no beam search (Schutz et al., 23 Sep 2025). For MVC, GNARL is reported as competitive with PDNAR, PDNAR$R$6, and an approximation baseline, with GNARL$R$7 surpassing the approximation baseline at larger scales relative to training size (Schutz et al., 23 Sep 2025). For RGC, GNARL is described as competitive with the specialized RL method RNet-DQN, and on BA graphs with targeted removal, GNARL$R$8 improves generalization over RNet-DQN (Schutz et al., 23 Sep 2025).

These results support the paper’s claim that the framework can operate across polynomial-time algorithmic tasks, NP-hard optimization problems, and settings in which no expert algorithm is available. A plausible implication is that the primary empirical advantage of GNARL is not a single architecture-specific gain, but the interaction between MDP formulation, valid-action constraints, and the ability to switch between IL and RL without changing the overall graph-policy design.

6. Limitations, failure modes, and interpretive context

The paper identifies several limitations explicitly. The MDP must be manually specified, including states, actions, and transitions. The framework also depends on stepping an environment during execution, which can become a runtime bottleneck; the authors suggest learned world models as a possible remedy. In addition, behavioral cloning suffers from compounding distribution shift, because a BC-trained model observes only states under $R$9, so off-distribution errors at test time can propagate. This is especially harmful for longer-horizon problems (Schutz et al., 23 Sep 2025).

Several failure modes and design observations are also noted. Greedy decoding can still fail if a poor action is selected early. Long trajectories are harder because errors accumulate over many steps. The paper remarks that evaluation should emphasize graph accuracy rather than node accuracy when many output configurations are valid. It also reports that Max aggregation is often beneficial for algorithmic alignment, but can cause state aliasing in some domains; for DFS and RGC, Sum aggregation is used instead (Schutz et al., 23 Sep 2025).

The multiple-solution setting introduces its own tension. GNARL’s ability to sample diverse valid solutions is described as a strength, but the paper also notes a tradeoff between strict solution matching and diversity of valid outputs (Schutz et al., 23 Sep 2025). This bears directly on how algorithmic reasoning systems are evaluated: if the target problem admits many correct trajectories, exact agreement with one expert trace may be less informative than validity and objective value.

In broader context, GNARL occupies a distinct place within graph-learning research. It is not a graph time-series model in the sense of GNAR (Jiménez-Varón et al., 7 Oct 2025, Knight et al., 2019), nor a graph-native language-model architecture such as NAG, which internalizes graph processing within a Transformer through topology-aware masking and positional recalibration (Gong et al., 30 Jan 2026). GNARL instead targets algorithm learning on graphs through sequential decision making. Its defining contribution is the formal translation of algorithm trajectories into actions in an MDP, yielding a unified framework for expert imitation, RL without an expert, and valid-by-construction graph solution generation (Schutz et al., 23 Sep 2025).