---
title: Nested Graph Pseudo-Label Refinement (NeGPR)
url: https://www.emergentmind.com/topics/nested-graph-pseudo-label-refinement-negpr
type: topic
---

# Nested Graph Pseudo-Label Refinement (NeGPR)

Nested Graph Pseudo-Label Refinement (NeGPR) is a framework for **graph-level unsupervised domain adaptation under noisy source labels**, introduced for the setting in which a classifier trained on a labeled source graph corpus must be transferred to an unlabeled target graph corpus despite corruption in the source labels [2508.00716]. Its central premise is that **label noise and domain shift interact destructively** in graph adaptation: noisy supervision distorts source-side class boundaries and representations, after which adaptation can align target samples to incorrect regions of feature space and amplify negative transfer. NeGPR addresses this coupled failure mode through a three-part design comprising **noise-resilient dual-branch pretraining**, **nested cross-branch pseudo-label refinement**, and **a noise-aware regularizer that explicitly suppresses the effect of noisy pseudo labels** [2508.00716].

## 1. Problem setting and failure modes

The formulation follows standard unsupervised graph domain adaptation. The labeled source domain is
$$
\mathcal{D}^s = \{(G_i^s, y_i^s)\}_{i=1}^{n_s},
$$
where the labels $y_i^s$ may be noisy; the unlabeled target domain is
$$
\mathcal{D}^t = \{G_j^t\}_{j=1}^{n_t};
$$
and the two domains share the label space
$$
\mathcal{Y}=\{1,2,\dots,C\}.
$$
The task is to learn from $\mathcal{D}^s$ and $\mathcal{D}^t$ and achieve high target accuracy [2508.00716].

The method is motivated by three failure modes identified for graph domain adaptation with noisy source labels. First, **loss-based denoising becomes unreliable under domain shift**: methods developed for single-domain noisy-label learning, such as small-loss selection or robust losses, can align target features to corrupted source class prototypes, producing misaligned and noise-amplified embeddings. Second, **pseudo-labeling becomes contaminated**: if the source model has already been biased by noisy supervision, target pseudo labels inherit that bias, and in GNNs message passing can propagate the errors through neighborhoods. Third, **class-wise alignment is distorted**: when source supervision is untrustworthy, the alignment objective itself can pull samples toward incorrect class regions, making adaptation harmful rather than corrective [2508.00716].

These observations place NeGPR within a specific regime: not generic graph classification, but graph-level domain adaptation in which label noise and domain shift must be handled jointly rather than sequentially. This suggests that methods robust to noisy labels in a single domain, or methods designed for clean-label domain adaptation, are individually insufficient when both perturbations are present simultaneously.

## 2. Dual-branch architecture

NeGPR is organized around two complementary branches. The **Implicit branch (IB)** learns graph semantics through message passing, whereas the **Explicit branch (EB)** learns topology and structural patterns through graph kernels or substructure encoders [2508.00716]. The architectural rationale is that semantic similarity and higher-order structure offer different inductive biases; the framework treats this complementarity as a mechanism for reducing over-reliance on any single noisy supervision stream.

For the implicit branch, node embeddings are updated by a standard message-passing rule:
$$
\mathbf{h}_{u}^{l}= \operatorname{COM}\left(\mathbf{h}_{u}^{l-1}, \operatorname{AGG} \left(\mathbf{h}_{v\in \mathcal{N}(u)}^{l-1} \right)\right),
$$
and the graph representation is obtained via
$$
\mathbf{z}_{G}^{IB}=\operatorname{READOUT}\left(\left\{\mathbf{h}_{u}^{L}\right\}_{u \in \mathcal{V}\right).
$$
Here, $\mathcal{N}(u)$ denotes the neighborhood of node $u$, $\operatorname{AGG}$ aggregates neighbor states, $\operatorname{COM}$ combines self and neighborhood information, and $\operatorname{READOUT}$ pools node embeddings into a graph embedding. In the experiments, IB is instantiated with **GMT** (Graph Multiset Transformer), selected for strong semantic aggregation [2508.00716].

For the explicit branch, topological information is encoded more directly through high-order substructures:
$$
\mathbf{h}_v = \phi\left( \mathcal{S}_v\left(G\right) \right), \quad \forall v \in \mathcal{V},
$$
$$
\mathbf{z}_G^{EB} = \operatorname{READOUT}\left( \left\{ \mathbf{h}_v\right\}_{v\in \mathcal{V} \right).
$$
Here, $\mathcal{S}_v(G)$ is a set of substructures around $v$ such as shortest paths or subtree patterns, and $\phi(\cdot)$ maps each substructure to a latent vector. In the experiments, EB is instantiated with **PathNN**, described as a path-based graph model [2508.00716].

The architectural split is not presented as redundancy. The ablation results reported for removing either IB or EB indicate that performance drops sharply, which the paper interprets as evidence for semantic and structural complementarity. A plausible implication is that NeGPR relies on disagreement structure between branches as much as on agreement structure: if both branches made identical errors, the later refinement mechanism would have less corrective capacity.

## 3. Noise-resilient pretraining through neighborhood consistency

Before target adaptation begins, both branches are pretrained on the noisy source domain using supervised learning augmented with a **neighborhood consistency** objective [2508.00716]. Rather than trusting each noisy source label in isolation, the method constructs a semantic neighbor graph among source samples.

For a branch $B\in\{\text{IB}, \text{EB}\}$, the similarity between source graphs is defined by cosine similarity between graph embeddings:
$$
\alpha_{ij}=\frac{\mathbf{z}_{G_i}^B^{\top}\mathbf{z}_{G_j}^B}{\|\mathbf{z}_{G_i}^B\|\cdot\|\mathbf{z}_{G_j}^B\|}.
$$
Using the top-$k$ nearest source samples, the branch is encouraged to predict similarly to a weighted average of its neighbors’ predictions:
$$
\mathcal{L}_{\text{noise}^B} = \frac{1}{n_s} \sum_{i=1}^{n_s} \text{KL}\left( {\mathbf{z}_{G_i}^B} \Bigg\| \sum_{j \in top-k(G_i)} \alpha_{ij}\cdot \mathbf{z}_{G_j}^B \right).
$$
The source-side pretraining objective for each branch is
$$
\mathcal{L}_{\text{pre}^B} = \mathcal{L}_{\text{sup}^B} + \beta \mathcal{L}_{\text{noise}^B},
$$
with supervised term
$$
\mathcal{L}_{\text{sup}^B} = \frac{1}{n_s}\sum_{i=1}^{n_s} l(\sigma(\mathbf{z}_{G_i}^B),y_i),
$$
where $l$ is cross-entropy and $\sigma$ is softmax [2508.00716].

The paper notes that the notation in the neighborhood-consistency expression is somewhat garbled, but states that the intended meaning is clear: the model is penalized when its output for a source graph deviates from the prediction structure of its semantic neighborhood. Functionally, this is a local smoothness prior imposed on source predictions. The stated purpose is to reduce over-reliance on noisy labels by forcing local label and representation consistency [2508.00716].

In methodological terms, this pretraining stage is the first defense against noise. The source-side semantic neighborhood is treated as a more trustworthy supervision signal than an individual corrupted annotation. This suggests a shift from instance-wise trust to neighborhood-wise trust, which is particularly relevant in graph classification settings where semantically similar graphs may remain proximate even when some labels are corrupted.

## 4. Nested cross-branch pseudo-label refinement

After pretraining, NeGPR adapts to the target domain by alternating between the two branches. One branch produces pseudo labels for high-confidence target samples, and the other branch is updated using those samples; the roles then reverse [2508.00716]. This alternating procedure is the “nested” component of the method.

At each iteration, a branch $B$ predicts target probabilities:
$$
\hat{\mathbf{y}}_j = \operatorname{Softmax}(\mathbf{z}_{G_j}^B).
$$
High-confidence target graphs are selected using a threshold $\zeta$:
$$
\mathcal{T}_{\text{conf}^B} = \left\{ G_j^t \in \mathcal{D}^t \mid \max(\hat{\mathbf{y}}_j)\ge \zeta \right\}.
$$
The pseudo label is then assigned by
$$
\tilde{y}_j = \arg\max(\hat{\mathbf{y}}_j), \quad \forall G_j^t \in \mathcal{T}_{\text{conf}^B}.
$$
Confidence is therefore defined simply as the maximum predicted class probability [2508.00716].

The selected pseudo-labeled target samples from branch $B$ are used to fine-tune the opposite branch $B'$ via
$$
\mathcal{L}_{\text{refine}^{B'}} = \mathcal{L}_{\text{pre}^{B'}} - \frac{1}{|\mathcal{T}_{\text{conf}^B}|} \sum_{G_j^t \in \mathcal{T}_{\text{conf}^B}} \tilde{y}_j \log \sigma(\mathbf{z}_{G_j^t}^{B'}).
$$
The algorithm is summarized as follows: pretrain IB and EB on noisy source data; for iteration $i=1,\dots,T$, use branch $B$ to select high-confidence target samples, update branch $B'$ on those pseudo labels, then use $B'$ to select high-confidence samples and update $B$ [2508.00716].

The significance of this design lies in its departure from single-branch self-training. Instead of allowing one model to reinforce its own pseudo labels, NeGPR forces pseudo-label selection and pseudo-label consumption to occur in different branches. The paper characterizes this as a way for each branch to help correct the other while limiting the damage from noisy pseudo labels. A plausible implication is that the cross-branch arrangement reduces confirmation bias, provided that the branches are genuinely complementary rather than synchronized in error.

## 5. Noise-aware regularization and theoretical interpretation

Because high-confidence pseudo labels can still be incorrect, NeGPR introduces a **noise-tolerant regularization** term that couples the two branches during refinement [2508.00716]. The final refinement loss for branch $B'$ is
$$
\mathcal{L}_{\text{Re}^{B'}} = \mathcal{L}_{\text{refine}^{B'}} - \frac{\lambda}{|\mathcal{T}_{\text{conf}^B}|} \sum_{G_j^t \in \mathcal{T}_{\text{conf}^B}} \log\left( \left\langle \sigma(\mathbf{z}_{G_j^t}^{B'}), \sigma(\mathbf{z}_{G_j^t}^{B}) \right\rangle \right).
$$
This term encourages agreement between the predictive distributions of the two branches on target samples [2508.00716].

The paper gives two qualitative interpretations. When the pseudo label is correct, the cross-entropy gradient shrinks as the prediction matches the label; without extra regularization, the training signal from clean pseudo-labeled samples would vanish, allowing noisy examples to dominate later updates. The regularizer preserves a meaningful gradient contribution and maintains learning from clean target samples. When the pseudo label is incorrect, the cross-entropy term pushes the model toward the wrong class, while the agreement term contributes a counteracting gradient that suppresses this harmful update and reduces the effect of noisy pseudo labels [2508.00716].

The theoretical discussion is gradient-based rather than a full formal generalization analysis. Let $\Theta$ be the parameters of branch $B'$. The paper states
$$
\nabla_\Theta \mathcal{L}_{\text{Re}^{B'}} = \frac{1}{|\mathcal{T}_{\text{conf}^B}|} \sum_{G_j^t \in \mathcal{T}_{\text{conf}^B}} \nabla_\Theta \mathbf{z}_{G_j^t}^{B'} \cdot \left( \mathbf{p}_j - \tilde{y}_j + \lambda \cdot \mathbf{g}_j \right),
$$
where
$$
\mathbf{p}_j=\sigma(\mathbf{z}_{G_j^t}^{B'}), \qquad \mathbf{q}_j=\sigma(\mathbf{z}_{G_j^t}^{B}),
$$
and
$$
\mathbf{g}_j = \frac{1}{\langle \mathbf{p}_j, \mathbf{q}_j \rangle} \cdot \mathbf{J}_{\mathbf{p}_j}^{\top}\mathbf{q}_j,
$$
with softmax Jacobian
$$
[\mathbf{J}_{\mathbf{p}_j}]_{ck} = \frac{\partial p_{j,c}}{\partial z_{j,k}^{B'}} = p_{j,c}(\delta_{ck}-p_{j,k}).
$$
The appendix reportedly expands this further and provides a component-wise form of the regularizer gradient [2508.00716].

The theoretical claims rely on several explicit or implicit assumptions: the two branches are complementary rather than collapsing into identical errors; high-confidence selection is more likely to contain correct pseudo labels; and neighborhood consistency in the source domain captures meaningful local class structure even under noise [2508.00716]. The paper also states that the regularizer is intended to combat both pseudo-label noise introduced during target refinement and source overfitting caused by noisy labels during pretraining. It therefore functions as a stabilizer of the entire nested procedure, not merely as an auxiliary penalty.

## 6. Empirical evaluation, ablations, and scope

The empirical evaluation is conducted on TUDataset benchmarks under two kinds of domain shift: **structure-based shift** and **feature-based shift** [2508.00716]. Structure-based shift uses **MUTAGENICITY**, **NCI1**, **FRANKENSTEIN**, and **PROTEINS**, with graphs partitioned into four subdomains using structural statistics such as node density, edge density, and graph flux, yielding transfers such as $P0\to P1$ and $N0\to N1$. Feature-based shift uses **PROTEINS**, **DD**, **COX2**, **COX2_MD**, **BZR**, and **BZR_MD**. The paper states that dataset sizes and class counts are reported in a table and that all benchmark tasks are binary graph classification [2508.00716].

The comparison set spans graph kernels, GNNs, noisy-label methods, graph domain adaptation methods, and methods that address both noise and domain adaptation.

| Category | Methods |
|---|---|
| Graph kernels | WL, PathNN |
| GNNs | GCN, GIN, GAT, GMT |
| Noisy-label methods | Co-teaching, RTGNN, Taylor-CE, OMG, SPORT |
| Graph domain adaptation methods | DEAL, CoCo, SGDA, A2GNN, StruRW, PA-BOTH |
| Methods addressing both noise and DA | ROAD, ALEX |

Implementation details are specified as **PyTorch / PyTorch Geometric**, with **Adam**, learning rate $10^{-4}$, hidden size 256, weight decay $10^{-12}$, 4 GNN layers, default source label noise ratio $\alpha=0.3$, default pseudo-label threshold $\zeta=0.9$, and 5 random runs averaged. IB uses GMT and EB uses PathNN [2508.00716].

The reported performance trend is that NeGPR achieves the best or near-best results across almost all datasets and shift settings. Representative examples include **PROTEINS under semantic shift**, where NeGPR reaches values such as 77.1 on $P0\to P3$, **NCI1 under graph flux/edge-density shifts**, where it typically leads by several points across almost all transfer directions, and **FRANKENSTEIN**, where many methods are near chance in some settings but NeGPR still improves consistently [2508.00716]. The abstract reports **up to 12.7% in accuracy** gain over the strongest baseline under severe label noise [2508.00716].

Ablation studies remove the implicit branch, the explicit branch, the noise-resilient loss in pretraining (**NeGPR w/o NRL**), and the noisy pseudo-label tolerated regularization (**NeGPR w/o NTR**). The reported outcome is that removing either branch sharply hurts performance, removing NRL degrades robustness to noisy source supervision, and removing NTR degrades refinement quality and increases sensitivity to pseudo-label noise [2508.00716]. Sensitivity analyses further state that higher pseudo-label confidence thresholds $\zeta$ generally improve performance by filtering unreliable pseudo labels, performance degrades as source noise $\alpha$ increases though NeGPR remains stronger than baselines, and among backbones GMT tends to be the strongest IB backbone while PathNN is strongest for EB [2508.00716].

The paper also states several limitations. It does not provide a formal convergence proof for the full alternating procedure, and the theoretical treatment is described as mostly gradient-based intuition rather than a full generalization bound. It additionally notes dependence on a reasonably good initial pretraining stage, possible sparsity of the confidence filter under very high label noise, and computational overhead from dual branches and iterative refinement [2508.00716]. Accordingly, NeGPR is presented as most effective when source labels are noisy but not completely random, source and target domains differ while sharing class semantics, the two branches capture genuinely complementary information, and a meaningful subset of high-confidence target predictions exists to bootstrap refinement [2508.00716].

In aggregate, NeGPR is defined by the coupling of three mechanisms: **noise-resilient pretraining via neighborhood consistency**, **nested cross-branch pseudo-label refinement using confidence-thresholded target samples**, and **an agreement-based regularization term that suppresses pseudo-label noise** [2508.00716]. Its distinguishing claim is not merely that graph domain adaptation benefits from better pseudo labels, but that noisy source supervision and domain shift must be treated as a single interacting problem.

Source: https://www.emergentmind.com/topics/nested-graph-pseudo-label-refinement-negpr