Learnable Reparameterized Graph Construction (LRGC)

• The paper introduces LRGC, which replaces fixed graph construction heuristics with a learnable, end-to-end optimized mechanism.
• It utilizes trainable key-query attention and soft-threshold reparameterization to dynamically prune and adapt graph connectivity.
• Empirical results reveal enhanced accuracy and efficiency on benchmarks, demonstrating improved performance over traditional graph methods.

to=python code to=python code Learnable Reparameterized Graph Construction (LRGC) denotes a class of graph-construction mechanisms in which graph topology is not fixed a priori but parameterized and optimized jointly with a downstream objective. In explicit nomenclature, the term is introduced in "ViG-LRGC: Vision Graph Neural Networks with Learnable Reparameterized Graph Construction" (Elsharkawi et al., 23 Sep 2025), where pairwise key-query attention and a soft-threshold reparameterization determine edges in each Grapher layer. In a broader lineage, "Dynamic Emotion Modeling with Learnable Graphs and Graph Inception Network" (Shirian et al., 2020) embodies the same principle by treating the adjacency matrix as a trainable object and learning it together with emotion classification. Across these formulations, the defining feature is end-to-end differentiable graph construction rather than reliance on manually specified adjacency rules.

1. Conceptual scope and lineage

LRGC is best understood as a response to a recurring limitation in graph neural architectures: the need to decide which nodes should communicate. Earlier pipelines often imposed this structure through non-learnable heuristics such as adjacency by temporal proximity, top-$k$ nearest neighbors, similarity thresholds, clustering-based hypergraphs, or axial constraints. LRGC replaces that design choice with trainable graph selection, so the topology can adapt to data and task loss rather than being fixed by hand (Elsharkawi et al., 23 Sep 2025).

In retrospective terms, the learnable graph construction used in L-GrIN is in the spirit of what later work would call LRGC. There, the model does not merely consume a graph; it learns the graph structure itself, and does so jointly with the classifier. The same basic principle reappears in ViG-LRGC, but with an explicit reparameterized sparsification rule and layer-specific learned thresholds (Shirian et al., 2020).

Work	Domain	Learnable graph element
L-GrIN (Shirian et al., 2020)	Dynamic emotion recognition	Adjacency matrix $\mathbf{A}$ and pooling vector $\mathbf{p}$
ViG-LRGC (Elsharkawi et al., 23 Sep 2025)	Vision Graph Neural Networks	Key-query edge scoring and learned threshold $\tau_l$

The two works differ in architecture and application domain, but they share a common methodological shift: graph construction is moved from preprocessing into the trainable core of the model.

2. Learnable adjacency in dynamic emotion modeling

In L-GrIN, dynamic emotion data from video, speech, and motion capture are modeled as structured graphs, with each temporal frame or short segment mapped to a node. Given a dynamic input sequence, the method constructs an undirected graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$ with $|\mathcal{V}|=M$ nodes, adjacency matrix $\mathbf{A}\in\mathbb{R}^{M\times M}$, and node-feature matrix

$$\mathbf{N} = [\mathbf{n}_1,\mathbf{n}_2,\cdots,\mathbf{n}_M]^T \in \mathbb{R}^{M\times P}.$$

The central design choice is that $\mathbf{A}$ is not assumed known. The paper explicitly contrasts this with manually constructed binary or weighted adjacency matrices and instead treats emotion recognition as a joint graph learning and graph classification problem (Shirian et al., 2020).

This learnable graph enters the convolution operator directly. L-GrIN replaces the linear projection in a conventional GCN with a nonlinear MLP and uses the trainable adjacency matrix inside message passing: $$\mathcal{G}^*(\mathbf{H}^{(k)})= \sigma\Big(\mathrm{MLP}^{(k)}\big(\mathrm{ReLU}(\mathbf{A})\mathbf{H}^{(k)}\big)\Big).$$ Here $\mathbf{A}$0 has two hidden layers with $\mathbf{A}$1 neurons each, and $\mathbf{A}$2 enforces non-negativity of edge weights. In LRGC terminology, this is a parameterized adjacency reoptimized end-to-end with the downstream classifier.

L-GrIN also introduces a graph inception layer to model temporal dynamics at multiple scales: $\mathbf{A}$3 The two graph convolutions use different hidden sizes, specifically $\mathbf{A}$4 and $\mathbf{A}$5, and the maxpool branch aggregates over each node’s directly connected 1-hop neighbors. Because the graph is initialized as fully connected, this initially resembles pooling across all nodes; as learning proceeds, the neighborhood structure changes with the learned adjacency.

Graph-level classification is performed through a combined fixed and learnable pooling scheme: $\mathbf{A}$6 The vector $\mathbf{A}$7 is trainable and learns how to combine node-level embeddings into a graph embedding. This makes graph construction and graph-to-vector aggregation jointly learnable.

3. Reparameterized edge selection in ViG-LRGC

The explicit LRGC formulation in ViG-LRGC was proposed for Vision Graph Neural Networks, where each layer must decide which image-patch nodes should communicate. The paper argues that earlier graph-building rules in ViG models are biased and brittle: $\mathbf{A}$8-NN forces every node to accept exactly $\mathbf{A}$9 neighbors, similarity-thresholded schemes still depend on fixed cutoffs, and clustering-based hypergraph construction introduces cluster-induced bias (Elsharkawi et al., 23 Sep 2025).

LRGC addresses this in two stages. First, it computes a learnable attention score for every candidate node pair. If the node features in Grapher layer $\mathbf{p}$0 are

$\mathbf{p}$1

then for a candidate directed edge $\mathbf{p}$2,

$\mathbf{p}$3

Here $\mathbf{p}$4 and $\mathbf{p}$5 are learnable linear layers.

Second, LRGC applies a differentiable soft-threshold reparameterization inspired by Soft Threshold Weight Reparameterization: $\mathbf{p}$6 which yields the combined edge score

$\mathbf{p}$7

If the pairwise attention is below the learned threshold, the edge is mapped to zero; otherwise it receives a positive score. The sigmoid on both attention and threshold is used for numerical stability, and each Grapher module has its own threshold parameter $\mathbf{p}$8.

This makes LRGC both learnable and hyper-parameter-free in the specific sense used by the paper: the threshold is not provided by grid search, but learned during training. The thresholds are initialized to $\mathbf{p}$9 for all layers, so the graph is fully connected in the first optimization step and then pruned during training.

4. Joint optimization and architectural embedding

In L-GrIN, graph learning is formalized as a composite optimization over classification, adjacency learning, and pooling regularization: $\tau_l$0

$\tau_l$1

with $\tau_l$2, and

$\tau_l$3

The first regularization term biases the learned adjacency toward temporal locality, the second regularizes adjacency magnitude, and the third regularizes the learnable pooling vector. Architecturally, L-GrIN contains two graph inception layers followed by pooling and a classifier, and the learned adjacency is shared in the sense that each graph sample has different node features but the same edge weights are assumed for a given dataset (Shirian et al., 2020).

In ViG-LRGC, the graph-construction module is inserted directly into the Grapher block of a ViG-Ti-like architecture. The model has a shallow CNN stem with five convolutional layers, a backbone of 12 repeated Grapher + FFN modules, and a classification head consisting of 2D average pooling followed by two fully connected layers. For the reported model, the embedding dimension is $\tau_l$4, and a $\tau_l$5 input produces a $\tau_l$6 grid, or $\tau_l$7 nodes per layer. Learnable positional embeddings preserve spatial information (Elsharkawi et al., 23 Sep 2025).

The LRGC-gated Grapher update is

$\tau_l$8

$\tau_l$9

$\mathcal{G}=(\mathcal{V},\mathcal{E})$0

$\mathcal{G}=(\mathcal{V},\mathcal{E})$1

The attention score $\mathcal{G}=(\mathcal{V},\mathcal{E})$2 acts as a gate: if $\mathcal{G}=(\mathcal{V},\mathcal{E})$3, the edge contributes nothing. In both L-GrIN and ViG-LRGC, therefore, graph construction is not an external preprocessing step but an internal, differentiable component of the model.

5. Empirical results and ablation evidence

L-GrIN was evaluated on five benchmark emotion recognition databases spanning video, audio, and motion capture. The reported results are 94.11% on RML, 87.49% on eNTERFACE, and 85.65% on RAVDESS for facial emotion recognition; 65.50% on IEMOCAP for speech emotion recognition; and 58.59% on MPI for body emotion recognition. These scores outperform BLSTM, standard GCN, PATCHY-SAN, PATCHY-Diff, SENet, CNN, CNN-LSTM, representation learning, LSTM-CTC, and several audiovisual baselines. The paper also emphasizes parameter efficiency: L-GrIN uses on the order of $\mathcal{G}=(\mathcal{V},\mathcal{E})$4 parameters, far fewer than the million-scale BLSTM and smaller than CNNs such as SENet, and on the speech task it outperforms LSTM-CTC while using about four times fewer parameters (Shirian et al., 2020).

The ablations in L-GrIN isolate the contribution of learnable graph construction. Starting from a standard GCN baseline at 76.57% on RML, progressively adding the new $\mathcal{G}=(\mathcal{V},\mathcal{E})$5 convolution, inception module, learned adjacency, and learned pooling raises performance to 94.11% when all components are enabled. Removing either learned adjacency or learned pooling lowers accuracy by 2.61%. The pooling study reports that learnable pooling outperforms maxpool, meanpool, and sortpool on RML, and the adjacency study shows that learnable adjacency outperforms binary and weighted hand-crafted graphs across RML, IEMOCAP, and MPI. The inception hyperparameter study identifies the pair $\mathcal{G}=(\mathcal{V},\mathcal{E})$6 as best for the two graph convolutions and finds that two inception layers are optimal.

ViG-LRGC was evaluated on ImageNet-1k. The reported comparison is 8.1M parameters, 1.5B FLOPs, 74.3% Top-1, and 92.3% Top-5 for ViG-LRGC; 7.2M parameters, 1.3B FLOPs, 70.5% Top-1, and 89.9% Top-5 for ViG-Ti; and 7.2M parameters, 1.3B FLOPs, 71.9% Top-1, and 90.8% Top-5 for SViG-Ti. Thus the LRGC variant improves over ViG-Ti by 3.8% Top-1 and 2.4% Top-5, and over SViG-Ti by 2.4% Top-1 and 1.5% Top-5 (Elsharkawi et al., 23 Sep 2025).

The empirical behavior of LRGC in ViG is also characterized by pruning dynamics. A plot of the average number of neighbors per node across epochs shows that the graph begins fully connected and becomes increasingly sparse during training, with different layers learning different effective sparsities. This is presented as evidence that the learned thresholds $\mathcal{G}=(\mathcal{V},\mathcal{E})$7 adapt pruning strength on a layer-by-layer basis.

6. Relation to heuristic graph construction, assumptions, and limits

A common misconception is to reduce LRGC to a variant of fixed similarity thresholding. The explicit ViG-LRGC formulation differs more fundamentally: both the scoring function and the pruning threshold are learnable. It therefore departs not only from $\mathcal{G}=(\mathcal{V},\mathcal{E})$8-NN, which forces fixed-degree neighborhoods, but also from similarity-thresholded construction, which still relies on fixed cutoffs, and from clustering-based hypergraph construction, which determines neighborhood structure through cluster assignment rather than pairwise learned relevance (Elsharkawi et al., 23 Sep 2025).

Another misconception is that a learned graph is necessarily unconstrained. L-GrIN provides the opposite example. Its adjacency is trainable, but the graph learning loss includes a structural matrix $\mathcal{G}=(\mathcal{V},\mathcal{E})$9 with entries $|\mathcal{V}|=M$0, which encourages nearby temporal segments to have stronger connectivity, along with Frobenius-norm regularization on $|\mathcal{V}|=M$1 and $|\mathcal{V}|=M$2 regularization on the learnable pooling vector $|\mathcal{V}|=M$3. The learned adjacency is therefore data-driven but not unregularized, and the paper notes that it visually concentrates mass near the diagonal, indicating that optimization discovers local temporal dependencies while still allowing departures from a rigid prior (Shirian et al., 2020).

The two formulations also make different modeling assumptions. In L-GrIN, the learned adjacency is shared across samples within a dataset, which is the simplifying assumption that makes graph structure learning tractable in that setting. In ViG-LRGC, graph construction is recomputed within each Grapher layer and begins from a fully connected candidate graph through initialization $|\mathcal{V}|=M$4. This offers the model maximal candidate connectivity at the start of training but increases memory use in the early stage; the reported experiments therefore use batch size 256 and note that the method is more memory-intensive initially (Elsharkawi et al., 23 Sep 2025).

From a computational standpoint, ViG-LRGC reports graph-construction complexity of roughly

$|\mathcal{V}|=M$5

which is simplified to approximately $|\mathcal{V}|=M$6 when $|\mathcal{V}|=M$7. The paper characterizes this as remaining in the same general regime as earlier ViG-style graph construction, with a slight FLOPs increase due to learnable projections. For L-GrIN, the practical regime includes fixed-length standardization by cyclic padding, graph sizes such as $|\mathcal{V}|=M$8 for video and $|\mathcal{V}|=M$9 for speech and motion capture, Adam with learning rate 0.01 and decay by 0.5 every 50 epochs, and regularization coefficients $\mathbf{A}\in\mathbb{R}^{M\times M}$0 and $\mathbf{A}\in\mathbb{R}^{M\times M}$1 (Shirian et al., 2020).

Taken together, these works establish LRGC as a technical pattern rather than a single architecture: graph topology is treated as a differentiable, trainable object; sparsity or structure is induced by learnable gating, reparameterization, or regularization; and graph construction is optimized jointly with representation learning and classification. A plausible implication is that LRGC is most useful where fixed neighborhood rules impose the wrong inductive bias, particularly when local and long-range dependencies must be selected adaptively rather than prescribed in advance.