Neuron-Activated Graph (NAG)

• Neuron-Activated Graph (NAG) is a concept encompassing neuron-centric graph frameworks that represent, interpret, and modulate neural computations.
• It facilitates encoder-free text-graph modeling, target-oriented pretraining, and graph-adaptive activations across diverse neural architectures.
• NAG integrates structured neuron activation profiles and graph-conditioned nonlinearities to enhance interpretability and performance in neural network models.

Neuron-Activated Graph (NAG) is a neuron-centric graph concept whose published usage spans several distinct constructions. In explicit nomenclature, NAG denotes both “Native Architecture for Graphs” for encoder-free text-graph modeling in LLMs and “Neuron-Activated Graph Ranking” for target-oriented pretraining data selection; closely related papers use the same phrase to describe graph representations of neuron behavior, graph-conditioned activations, and active neuron-like subgraphs that function as memory engrams (Gong et al., 30 Jan 2026, Wang et al., 17 Apr 2026, Foote et al., 2023, Mantri et al., 2024, Wei et al., 2023). A plausible unifying interpretation is that NAG refers to any formalism in which neurons, neuron activations, or neuron-level computation define the nodes, edges, or latent structure of a graph used for representation, reasoning, or adaptation.

1. Terminological scope and recurring formal motifs

The literature does not present a single canonical NAG. Instead, it presents several technically distinct objects that share a neuron-centric graph interpretation. Two papers use the acronym directly in their titles, while several others supply mechanisms that the cited authors explicitly connect to a “Neuron-Activated Graph” idea: graph-conditioned activation hypernetworks, trie-structured neuron explanations, active-directed memory subgraphs, and graph-organized neuron populations (Wang et al., 17 Apr 2026, Gong et al., 30 Jan 2026, Foote et al., 2023, Mantri et al., 2024, Wei et al., 2023, Salomon, 27 Nov 2025).

Usage	Core object	Representative source
NAG	Encoder-free LM architecture for text-graphs	(Gong et al., 30 Jan 2026)
NAG-based Ranking	Sparse cross-layer neuron index set for data selection	(Wang et al., 17 Apr 2026)
Neuron graph / NAG-style	Trie or graph describing a single neuron’s behavior	(Foote et al., 2023)
Graph-activated neuron mechanism	Graph-conditioned activation function for GNN layers	(Mantri et al., 2024)

Across these variants, several motifs recur. First, the graph is often defined over neurons, neuron indices, or neuron-induced abstractions rather than over external entities alone. Second, the graph is typically sparse or structured, even when implemented over a complete candidate set. Third, the representation is usually cross-layer or context-dependent rather than fixed. Fourth, the graph often serves as an interpretable surrogate for internal computation rather than merely as an input data structure. A common misconception is therefore to treat NAG as the name of one architecture; the cited work instead shows a family of related, but non-identical, constructions.

2. Neuron-index graphs for interpretability and data selection

One prominent NAG lineage is explicitly neuron-centric. In “Neuron to Graph: Interpreting LLM Neurons at Scale”, a neuron graph is built from the top-activating examples of a single neuron by pruning each example to a minimal substring that preserves at least $50\%$ of the neuron’s pivot activation, applying perturbation-based saliency, augmenting salient context tokens with DistilBERT substitutions that also preserve at least $50\%$ activation, and inserting the resulting context-to-pivot patterns into a trie whose end nodes store normalized activation values (Foote et al., 2023). The core quantities are the pivot index

$e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$

the saliency score

$$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$

and the graph-based predictor

$$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$

The resulting representation is a trie internally and a compact DAG visually, with context nodes, ignore nodes, pivot nodes, and end nodes. On a 6-layer SoLU Transformer, average F1 for predicting neuron firing from the graph exceeds both token-lookup and 5-gram lookup baselines in every layer, with layer-0 F1 $0.68$ and layer-1 F1 $0.65$; the method scaled to all $18{,}432$ neurons of the model in approximately 48 hours on a single Nvidia Tesla T4 GPU.

A second neuron-centric formulation is NAG-based Ranking for target-oriented pretraining data selection. Here a neuron is a column of a projection matrix, neuron impact is

$$\text{Imp}(N_k \mid h_{\text{in}}) = \left| h_{\text{in}}^\top W_{:,k} \right|,$$

and the NAG of an input $c$ is the layer-wise set of top-$50\%$0 high-impact neurons,

$50\%$1

Similarity is defined by Dice-style overlap,

$50\%$2

This “graph” is explicitly described as loose: edges are implicit in the layered structure rather than modeled directly. Nevertheless, it functions as a compact cross-layer neuron profile for retrieval and ranking. With default width ratio $50\%$3 and top-20% corpus filtering, NAG-based Ranking improves target-oriented pretraining by $50\%$4 on average over random sampling, and deactivating only $50\%$5 of NAG-selected neurons causes a $50\%$6 performance collapse, while random deactivation of the same fraction changes accuracy by only $50\%$7 (Wang et al., 17 Apr 2026). Another important result is that restricting NAG to the final layer incurs a $50\%$8 average drop, which establishes that the relevant neuron pattern is distributed across layers rather than concentrated in the last representation alone.

Taken together, these two lines use NAG not as an external graph encoder but as an interpretable neuron-level summary. In one case the summary is symbolic and sequence-local; in the other it is sparse, cross-layer, and corpus-scalable. This suggests that “graph” in NAG can denote either explicit combinatorial structure or a structured set of neuron identities, provided that the object captures how internal neuron activity organizes computation.

3. Graph-conditioned nonlinearities and graph-adaptive activation mechanisms

A different usage of NAG arises in graph neural networks, where the activation itself is conditioned on graph structure. The general template appears in “Graph-Adaptive Activation Functions for Graph Neural Networks”, where the post-convolution nonlinearity is no longer pointwise. Instead, each activation aggregates multi-hop shifted signals over local neighborhoods,

$50\%$9

or, in the kernel variant,

$e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$0

The paper proves permutation equivariance for these activations and establishes Lipschitz stability for graph-adaptive max activations (Iancu et al., 2020). In this sense, the neuron’s activation is graph-local and graph-parameterized, making the nonlinearity itself a graph operator.

DiGRAF sharpens this idea by making the activation graph-adaptive, diffeomorphic, and layer-specific. The activation is a CPAB diffeomorphism $e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$1 over a bounded interval $e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$2, with parameters predicted by an auxiliary GNN: $e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$3 The layer then applies

$e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$4

with scalar action

$e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$5

The paper is explicit that this is graph-wise adaptive, not node-wise: one $e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$6 is shared across all nodes and channels within a layer for a given graph. It also states that the paper does not use the term “Neuron-Activated Graph,” but that DiGRAF aligns with such an idea because the graph determines the activation curve used by all neurons in a layer (Mantri et al., 2024).

The experimental record supports that interpretation. DiGRAF reports BlogCatalog accuracy $e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$7, Flickr accuracy $e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$8, and ZINC-12k MAE $e_i = \arg\max_k a(k, x^{(i)}, \ell, j),$9, compared with $$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$0 for GIN+ReLU and $$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$1 for DiGRAF without graph adaptivity. It also improves molhiv ROC-AUC to $$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$2 and molsol RMSE to $$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$3. A necessary clarification is that this is not a node-personalized activation system in its base form. The graph activates a layer-shared nonlinearity, so the mechanism is graph-conditioned rather than neuron-specific in the strictest sense.

4. Native graph computation inside LLMs

In “NAG: A Unified Native Architecture for Encoder-free Text-Graph Modeling in LLMs”, NAG is an explicit acronym for Native Architecture for Graphs. The central claim is that text-graphs need not be encoded by an external GNN and then aligned to a LLM through graph tokens or soft prompts. Instead, NAG flattens a text-graph into a token sequence with <n>, <e>, and <g> tags, enforces topology using a topology-aware self-attention mask, and removes serialization-order bias using recalibrated positional IDs (Gong et al., 30 Jan 2026).

The mask is constructed as

$$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$4

where the components separately encode intra-element semantics, inter-element structural flow, graph-level aggregation through </g>, and query access to the graph. Structural position calibration then assigns all semantic hub tokens the same positional ID,

$$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$5

so that relative distance under RoPE does not depend on arbitrary serialization order. This produces permutation-invariant graph reading at the element level, while preserving ordinary causal attention inside each node or edge text span.

Two implementations are introduced. NAG-LoRA inserts LoRA updates into attention projections and allows deeper structural adaptation. NAG-Zero freezes the backbone and inserts inter-layer gated adapters that operate only on special structural tokens, with

$$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$6

while non-special text tokens bypass the adapter exactly. The stated consequence is “absolute” preservation of the base model’s linguistic behavior on plain text for NAG-Zero.

Empirically, on nine synthetic topology tasks totaling 180k samples, NAG-LoRA is best in 8 of 9 tasks, including Node Count $$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$7 with $$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$8 absolute error, Edge Count $$\alpha_{k,k'} = 1 - \frac{a(k', y^{(i)}, \ell, j)}{a(k', \hat{y}^{(i)}, \ell, j)},$$9 with $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$0 absolute error, Triangle Count $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$1 with $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$2 absolute error, and Shortest Path $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$3 with $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$4 absolute error. On real semantic graph tasks, NAG-LoRA achieves ExplaGraphs $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$5, SceneGraphs $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$6, and WebQSP Hit@1 $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$7. The paper also proves that replacing recalibrated positional IDs with standard sequential IDs harms performance, including a Connected Nodes improvement from $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$8 to $$\hat{a}(i, x; G_{\ell j}) = \max_{\text{paths } p \text{ matching context around } x_i} \text{norm-activation}(p).$$9 under recalibration. Within the NAG family, this work is distinctive because the graph is not inferred from neurons; it is internalized into the LM’s own self-attention manifold.

5. Active memory graphs and dynamical neuron-population graphs

Another NAG-relevant tradition appears in memory and neuroscience-inspired graph dynamics. In “Autonomous and Ubiquitous In-node Learning Algorithms of Active Directed Graphs and Its Storage Behavior”, an active-directed graph is a graph in which each node has autonomous and independent behavior and relies only on information obtained within the local field of view to make decisions (Wei et al., 2023). The paper adopts engram theory directly: a memory engram is a connected subgraph formed among a set of co-activated nodes. With $0.68$0 denoting active nodes at time $0.68$1, the stable neuron-activated subgraph is

$0.68$2

Nodes maintain local index tables of the form $0.68$3, activate with fixed probability $0.68$4 when stimulated, reuse past traces by maximizing F1 similarity between current and stored fan-in patterns, and otherwise choose fan-out neighbors by weighted random selection biased against over-used paths. Capacity is not merely linear in node count; the paper reports that sparser graphs can store more samples because single-sample subgraphs decompose into multiple weakly connected components. In one Erdős–Rényi example with $0.68$5, $0.68$6 edges, $0.68$7, and single-sample size $0.68$8, the sparse graph stores about $0.68$9 samples with $0.65$0, while a denser graph with $0.65$1 edges stores only about $0.65$2.

The same neuron-graph perspective appears in “Decoding Spiking Mechanism with Dynamic Learning on Neuron Population”, which proposes a Neuron Activation Network for single-trial spike trains (Chen et al., 2019). Here neurons are graph nodes, hidden states $0.65$3 are node features, and a learned relation matrix $0.65$4 acts as effective adjacency. Messages are computed by

$0.65$5

followed by recurrent state updates

$0.65$6

and Poisson spike generation $0.65$7. On retinal ganglion cell data with $0.65$8 neurons, the model reports about $0.65$9 explainable variance, lower MAE than pyGLM, and zero-lag cross-correlation peaks for almost all neurons. The learned relation matrix separates OFF and ON cells despite no cell-type supervision.

These two works situate NAG in dynamical systems rather than representation learning alone. In one case the graph is a decentralized memory substrate whose active subgraphs are the memory trace; in the other it is a latent population graph whose node activations and message passing jointly decode spiking dynamics. A plausible implication is that NAG can serve equally as a representational abstraction and as a model of distributed neural computation.

6. Graph-regularized neuron organization and graph-organized intelligence

A further lineage organizes neurons themselves into graphs for interpretability or computation. Graph Spectral Regularization imposes a graph over the neurons of a hidden layer and penalizes non-smooth activation patterns with the Laplacian quadratic form

$18{,}432$0

The graph may be predefined, such as an $18{,}432$1 grid or fully connected capsule-dimension graphs, or learned from feature co-activations using the adaptive Gaussian kernel

$18{,}432$2

The paper shows that this graph structure yields localized and interpretable activation patches with little or no performance loss: on MNIST, test accuracy with GSR is $18{,}432$3, matching L2 regularization and exceeding the unregularized baseline $18{,}432$4 (Tong et al., 2018). In NAG terms, each sample induces a graph signal over a neuron graph, and interpretation proceeds by analyzing smooth activation regions, connected components, and learned feature-space topology.

At the architectural extreme, “Intelligent Neural Networks: From Layered Architectures to Graph-Organized Intelligence” replaces layered feature processing with a graph of stateful neurons (Salomon, 27 Nov 2025). Each intelligent neuron combines selective state-space dynamics, implemented by a Mamba block, with attention-based routing over a complete graph of neurons. The generic update is

$18{,}432$5

Unlike standard transformers, attention is applied over the neuron dimension rather than the token dimension. On Text8, the reported test BPC is $18{,}432$6, compared with $18{,}432$7 for a comparable Transformer and $18{,}432$8 for a parameter-matched Mamba Stack that fails to converge under the same training protocol. Ablations further report $18{,}432$9 BPC for a no-communication variant and $$\text{Imp}(N_k \mid h_{\text{in}}) = \left| h_{\text{in}}^\top W_{:,k} \right|,$$0 for static communication, indicating that learned inter-neuron routing is beneficial while non-learned communication can act as interference.

These works clarify a final sense in which NAG operates: not merely as a graph extracted from neurons or a graph-conditioned nonlinearity, but as a neuron-organizing principle for the architecture itself. The published record therefore supports three broad readings of NAG: a graph of neurons, a graph induced by neuron activation, and a graph used to modulate neuron computation. The absence of a single standardized definition is not a defect of the term; it reflects the breadth of current attempts to make neural computation graph-structured, interpretable, and neuron-resolved.