Unified Graph Prompt Learning via Low-Rank Graph Message Prompting

Abstract: Graph Data Prompt (GDP), which introduces specific prompts in graph data for efficiently adapting pre-trained GNNs, has become a mainstream approach to graph fine-tuning learning problem. However, existing GDPs have been respectively designed for distinct graph component (e.g., node features, edge features, edge weights) and thus operate within limited prompt spaces for graph data. To the best of our knowledge, it still lacks a unified prompter suitable for targeting all graph components simultaneously. To address this challenge, in this paper, we first propose to reinterpret a wide range of existing GDPs from an aspect of Graph Message Prompt (GMP) paradigm. Based on GMP, we then introduce a novel graph prompt learning approach, termed Low-Rank GMP (LR-GMP), which leverages low-rank prompt representation to achieve an effective and compact graph prompt learning. Unlike traditional GDPs that target distinct graph components separately, LR-GMP concurrently performs prompting on all graph components in a unified manner, thereby achieving significantly superior generalization and robustness on diverse downstream tasks. Extensive experiments on several graph benchmark datasets demonstrate the effectiveness and advantages of our proposed LR-GMP.

• The paper presents a unified framework that integrates various graph data prompts into a general graph message prompting paradigm.
• It introduces a low-rank prompt representation that enhances efficiency, robustness, and accuracy in node and graph classification tasks.
• Empirical results demonstrate that LR-GMP outperforms traditional full fine-tuning and specialized GDPs, especially in few-shot and adversarial scenarios.

Unified and Efficient Graph Prompt Learning via Low-Rank Graph Message Prompting

Introduction and Motivation

Prompt-based adaptation has established itself as a central mechanism in GNN transfer learning, greatly reducing the parameter overhead in comparison to traditional full fine-tuning protocols. Graph Data Prompts (GDPs) have been developed to target specific graph components (e.g., node features, edge features, edge weights), but current GDP schemes suffer from two pronounced limitations: 1) their specialization to single graph components, restricting generalization and adaptability, and 2) the lack of a coherent, unified framework that can simultaneously integrate diverse prompting paradigms for downstream GNN tasks.

This paper addresses these limitations via two primary contributions: (1) a theoretical unification of previous GDP designs under the lens of Graph Message Prompting (GMP), where prompts are inserted at the level of the GNN message passing process, and (2) the proposal of a Low-Rank Graph Message Prompting (LR-GMP) scheme that instantiates efficient, expressive, and compact prompts through low-rank factorization, enabling joint modulation of both node and structural information.

Theoretical Foundation: Unification via Graph Message Prompts

GDPs—including node prompts, edge prompts, edge weight prompts, subgraph prompts, and hybrid approaches—are shown to be special cases of a general GMP formulation. In this paradigm, prompts are injected into the GNN's message construction step, rather than being separately assigned to raw node/edge/topology components. Specifically, any modification (addition, multiplication, or concatenation) to node or edge features, or changes in adjacency structure, can be recast as an additive or multiplicative modification to the message passed along $(u,v)$ in the GNN. Propositions with algebraic derivations rigorously demonstrate the equivalence of GDPs and GMPs.

Figure 1: Illustration contrasting canonical GDP insertion schemes with the unified GMP, highlighting how LR-GMP generalizes previous GDPs to operate over all graph components at once.

Figure 2: Schematic comparison between traditional component-wise GDP tuning and the unified LR-GMP framework, which performs prompt modulation in the multi-dimensional message space.

This theoretical unification simplifies the design of prompts and eliminates the need for intricate, hand-crafted protocols targeting specific graph portions. It also avoids the challenging optimization and combinatorial design decisions required by hybrid prompt schemes.

Low-Rank Graph Message Prompting: Method Details

The LR-GMP approach parameterizes the prompt as a low-rank matrix in the space of message dimensions. Letting $\mathbf{M}$ represent the message matrix, low-rank decomposition is performed as $\mathbf{P} = \mathbf{U}\mathbf{V}^\top$, where $\mathbf{U}$ and $\mathbf{V}$ are learnable factors, and $r$ (the rank) controls the trade-off between expressiveness and parameter efficiency. This form is memory- and compute-efficient even for large graphs, while remaining sufficiently powerful to generalize all previously proposed GDPs.

LR-GMP can be dynamically conditioned on the input (instance-specific prompts), supporting both transductive and inductive settings. The framework is inherently flexible to the choice of GNN backbone—GCN, GIN, GAT, APPNP, GCNII—all can seamlessly accommodate message-space prompting as an external, lightweight adaptation module.

Empirical Evaluation

Node and Graph Classification Performance

Extensive experiments on node and graph classification tasks validate that LR-GMP consistently outperforms both full fine-tuning and previous single-component or hybrid GDP-based prompt learning methods. Across benchmarks (Cora, CiteSeer, PubMed, Photo, Computers, Physics for nodes; BBBP, Tox21, SIDER, et al. for graphs), LR-GMP provides increased accuracy, particularly in few-shot settings where label efficiency and transferability are critical.

Figure 3: Performance of LR-GMP and leading baselines under various few-shot labeled data settings, demonstrating robust adaptation with limited supervision.

LR-GMP demonstrates robustness to adversarial/noisy graphs; its unified message prompt formulation provides strong regularization, outperforming models that only prompt node or edge components when the graph is randomly or adversarially perturbed.

Figure 4: LR-GMP maintains high performance under both random and targeted (Nettack) structural perturbations, with $p$ controlling the perturbation intensity.

Layer and Parameter Analysis

Prompt insertion can occur at arbitrary GNN layers (input, hidden, output, or all), and LR-GMP remains stable and performant across such configurations, verifying its layer-agnostic applicability.

Figure 5: Effect of inserting LR-GMP at different layers of the GNN stack, showing consistent gains irrespective of depth.

Variant studies on the rank parameter $r$ reveal favorable trade-offs: overly small $r$ limits capacity, while high $r$ brings diminishing returns and overfitting risks. In practice, moderate $\mathbf{M}$0 (2, 5, 10) suffices for state-of-the-art performance.

Figure 6: Validation curves depicting LR-GMP's performance for varying rank dimensions, guiding optimal selection for a balance between expressivity and parsimony.

Scalability and Flexibility

LR-GMP scales effectively with large-scale graphs and can replace GDPs in existing pipelines without architectural changes. The design is compatible with any GNN variant and does not interact adversely with message/attention structures, further supporting deployment in diverse real-world settings.

Implications and Future Directions

The introduction of unified message space prompt learning removes a persistent bottleneck in GNN adaptation research: the over-specialization of prompt designs and cumbersome compositional tuning. This opens up several lines for further exploration:

• Adversarially Robust Prompts: LR-GMP's unified formulation suggests novel prompt regularization for defending against sophisticated structural attacks.
• Automated Prompt Search: The reduction in design space complexity enables meta-learning and neural architecture search methods for prompt discovery.
• Few-shot/Continual Transfer: The ability to efficiently re-parametrize and re-tune only low-rank message prompts will benefit situations involving rapidly shifting or streaming graph domains.
• Cross-modal Graph Integration: Richer prompt spaces allow adaptation beyond classical attributional graphs, e.g., for vision-graph integration or multimodal GNN adaptation.

Conclusion

This work provides a comprehensive theoretical and empirical unification of graph prompt learning under the graph message prompting paradigm, eliminating the artificial separation imposed by prior GDP specialization. LR-GMP realizes efficient, expressive, and robust prompt tuning in the message space, setting a new standard for scalable and generalizable GNN adaptation.