Vertex Random Knowledge Graph (VRKG)

• VRKG is a probabilistic graph adapter where each class is represented as a Gaussian distribution, capturing intra-class textual diversity.
• The method constructs class nodes from multiple LLM-generated text descriptions and formulates edges using cosine similarity between class means.
• Empirical results show VRKG improves few-shot image classification and out-of-distribution robustness compared to deterministic adapters.

Vertex Random Knowledge Graph (VRKG) denotes, in the context of vision-LLM adaptation, a class-level probabilistic knowledge graph in which each class node is represented by a Gaussian distribution and inter-class edges are constructed from cosine similarity between class means (Jiang et al., 14 Jul 2025). Introduced as the textual-side graph model inside VRGAdapter, it is designed to replace a single deterministic class text feature with a distributional representation that captures both intra-class textual-description diversity and inter-class semantic relationships. The term is not uniform across the literature: in recommendation research, “VRKG” instead denotes “Virtual Relational Knowledge Graphs,” a relation-clustering construction unrelated to vertex-random node distributions (Lu et al., 2022).

1. Definition and conceptual scope

In the 2025 formulation, a VRKG is defined as a graph

$\mathcal{G} = (\mathcal{V}, \mathcal{E}),$

with class-node set

$\mathcal{V}=\{v_1,\dots,v_C\},$

where each node $v_i$ is not a fixed embedding but a Gaussian-distributed class representation

$$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$

The defining feature is therefore vertex-level randomness: the random object is the node representation itself, while the edge structure is deterministic once the node means are estimated (Jiang et al., 14 Jul 2025).

This design is motivated by a specific limitation of deterministic textual adapters for pre-trained VLMs. Prior adapters typically refine one handcrafted prompt, one class-name embedding, or one averaged text embedding per class. The VRKG formulation rejects that point-estimate view and instead treats each class as occupying a distribution in text-embedding space. In that sense, $\boldsymbol{\mu}_i$ functions as a semantic center and $\boldsymbol{\sigma}_i$ as a representation of semantic variability induced by multiple valid textual descriptions of the same class.

The graph is described as a knowledge graph because it encodes semantic knowledge extracted from text descriptions together with inter-class correlations. It is “vertex random” rather than a fully random graph because the edge weights are not sampled. The adjacency matrix is constructed deterministically as

$$\mathbf{A}_{ij} = \cos(\boldsymbol{\mu}_i, \boldsymbol{\mu}_j), \quad i,j \in [1,C].$$

A recurrent misconception is therefore that VRKG randomizes graph connectivity. In the formulation actually used, randomness is placed on class-node representations, not on edge existence or edge weight.

2. Construction pipeline

VRKG construction begins by generating multiple textual descriptions for each class using an LLM:

$$[\mathbf{P}_i^1, \mathbf{P}_i^2, \ldots,\mathbf{P}_i^M] = \mathrm{LLM}(\text{``Describe class } i \text{''}).$$

Each description is then encoded by the frozen CLIP text encoder:

$\mathbf{T}_i^m = f_T(\mathbf{P}_i^m).$

In experiments, the number of descriptions is $M=50$ (Jiang et al., 14 Jul 2025).

The Gaussian parameters of the class node are estimated empirically from these text embeddings:

$\mathcal{V}=\{v_1,\dots,v_C\},$0

$\mathcal{V}=\{v_1,\dots,v_C\},$1

The covariance is diagonal, so the model tracks per-dimension uncertainty but no cross-dimension covariance. The vertices are class/category nodes, not description nodes and not latent semantic prototypes.

The deterministic adjacency is then built from pairwise cosine similarity between class means:

$\mathcal{V}=\{v_1,\dots,v_C\},$2

This yields a graph $\mathcal{V}=\{v_1,\dots,v_C\},$3 whose node features are distributions and whose edges encode mean-based semantic affinity. The construction is practical rather than variational: the paper does not introduce a prior–posterior graphical model, an ELBO, or a KL regularizer for graph construction.

The framework also defines a deterministic limiting case. When $\mathcal{V}=\{v_1,\dots,v_C\},$4, the empirical variance is zero, so each node collapses to a point mass. The paper explicitly states that GraphAdapter is a special case of VRGAdapter with diversity $\mathcal{V}=\{v_1,\dots,v_C\},$5, which means that the deterministic graph adapter can be recovered as the zero-variance limit of VRKG (Jiang et al., 14 Jul 2025).

3. Probabilistic message propagation and adapter learning

Once constructed, VRKG is processed by a Gaussian-based graph convolution network that propagates the first two moments separately. Let $\mathcal{V}=\{v_1,\dots,v_C\},$6 and use the normalized propagation matrix $\mathcal{V}=\{v_1,\dots,v_C\},$7. The moment updates are

$\mathcal{V}=\{v_1,\dots,v_C\},$8

$\mathcal{V}=\{v_1,\dots,v_C\},$9

The initial conditions are $v_i$0 and $v_i$1. In implementation, ELU is used for means and ReLU for variances (Jiang et al., 14 Jul 2025).

This propagation gives each class a context-aware distribution representation. The mean channel aggregates semantic centers from neighboring classes, while the variance channel aggregates semantic diversity or uncertainty patterns. The resulting graph representation remains Gaussian at each node, but only in the diagonal-covariance sense.

After $v_i$2 graph layers, the model samples an adapted class feature via reparameterization:

$v_i$3

$v_i$4

The sampled feature is then combined with the original class semantic center:

$v_i$5

The full adapted textual classifier is

$v_i$6

VRKG therefore operates as a textual adapter attached after frozen text encoding, not as a modification of the CLIP text encoder itself. The image branch remains the frozen CLIP visual encoder, and the downstream task is classification through image–text matching. An important technical boundary is that the paper gives no dedicated graph regularizer, KL term, uncertainty regularizer, or calibration objective; the only explicit learning signal is downstream cross-entropy classification (Jiang et al., 14 Jul 2025).

4. Relation to earlier random-KG paradigms

VRKG belongs to a broader line of work in which randomness is used as a representational primitive on graphs, but it is distinct from earlier KG formulations. The closest precursor on the KG side is RR-GCN, which keeps both initial node representations and relation-specific transformations random and frozen, thereby constructing node embeddings by aggregating randomly transformed random representations from neighbors (Degraeve et al., 2022). RR-GCN is vertex-random in the sense of random initial entity states and random relation-aware propagation over typed directed KGs, but its node variables are fixed random vectors rather than Gaussian distributions, and its primary tasks are node classification and link prediction rather than VLM text adaptation.

A second nearby family uses random walks to induce auxiliary relational evidence. “Walk-and-Relate” augments sparse KGs by sampling random walks, extracting metapaths, and converting them into additional triples for downstream KG embedding models (Manchanda, 2022). The survey literature situates such methods alongside DeepWalk, node2vec, metapath2vec, regpattern2vec, and subgraph2vec, emphasizing different ways to bias or constrain random walks over knowledge graphs (Bozorgi et al., 2024). These approaches are vertex-centric and graph-randomizing in the sense of path sampling, but they do not endow vertices with distributional latent states.

By contrast, graph-level latent-variable generation on KGs, exemplified by the Relational Graph Variational Auto-Encoder, places a latent variable on a small KG subgraph or triple and jointly decodes adjacency, node labels, and edge labels (Wolf, 2021). That formulation is probabilistic, but the random object is the whole sampled relational graph rather than a persistent node distribution. A plausible implication is that VRKG occupies an intermediate position: more probabilistic than frozen random-feature encoders, but less generative than full graph VAEs.

These comparisons clarify the specific novelty of VRKG. It combines two elements that earlier lines typically separate: distributional node modeling for intra-class text diversity, and graph propagation for inter-class semantic coupling.

5. Empirical evidence and application profile

The immediate application of VRKG is few-shot image classification and out-of-distribution generalization within a VLM adaptation pipeline. The paper evaluates the resulting VRGAdapter on 11 few-shot image-classification datasets and on ImageNet-V2 and ImageNet-Sketch for OOD testing (Jiang et al., 14 Jul 2025).

An ablation on ImageNet-1K isolates the contribution of VRGAdapter itself. Baseline accuracies are reported as 59.42, 60.50, 61.69, 62.89, and 64.46 for 1-shot, 2-shot, 4-shot, 8-shot, and 16-shot settings, respectively. With VRGAdapter only, the corresponding accuracies are 62.49, 62.92, 63.57, 64.60, and 66.03. The reported absolute gains are therefore $v_i$7, $v_i$8, $v_i$9, $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$0, and $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$1.

A direct comparison with GraphAdapter under the same multi-branch setting further supports the vertex-random formulation. On DTD, GraphAdapter reports 54.08 / 56.15 / 63.89 / 67.73 / 72.87, whereas VRGAdapter reports 56.80 / 59.99 / 65.43 / 68.91 / 74.00 across the same shot settings. On ImageNet-1K, GraphAdapter reports 62.12 / 64.12 / 66.70 / 68.77 / 70.83, while VRGAdapter reports 63.88 / 65.53 / 67.54 / 69.31 / 71.35. The paper also studies the number of descriptions $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$2 and reports that performance improves as $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$3 increases from 2 to 50, peaking at 71.35% for $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$4 on 16-shot ImageNet-1K; for the mixing coefficient in $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$5, the best result occurs at $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$6.

The paper interprets these gains as evidence that modeling both semantic center and semantic variability is superior to using a single deterministic textual prototype. It argues for reliability primarily through robustness and accuracy improvements rather than through explicit calibration metrics.

6. Terminological ambiguity, limitations, and interpretation

The acronym “VRKG” is ambiguous in current usage.

Paper	Expansion	Core mechanism
(Jiang et al., 14 Jul 2025)	Vertex Random Knowledge Graph	Gaussian class nodes; deterministic mean-similarity edges
(Lu et al., 2022)	Virtual Relational Knowledge Graphs	Relation clustering into $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$7 virtual relations

In recommendation, VRKG4Rec uses “VRKG” to mean Virtual Relational Knowledge Graphs, where original KG relations are clustered into a small number $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$8 of virtual relations, then partitioned into $$\mathcal{H}_i \sim \mathcal{N}\big(\boldsymbol{\mu}_i, \operatorname{diag}(\boldsymbol{\sigma}_i)\big), \quad i \in [1,C].$$9 subgraphs and processed by Local Weighted Smoothing (Lu et al., 2022). That meaning is structurally different from vertex-random node distributions. The two uses share only the acronym and the general idea of reorganizing graph semantics for downstream learning.

Within the vertex-random formulation itself, several limitations are explicit. First, the uncertainty model is diagonal; full covariance structure is not propagated. Second, adjacency depends only on class means, not on similarities between full distributions. Third, the formulation is probabilistic but not fully Bayesian: the paper does not include an ELBO, KL-divergence term, graph consistency loss, or uncertainty regularizer. Fourth, reliability is argued through benchmark performance and OOD gains, while calibration metrics are not reported (Jiang et al., 14 Jul 2025).

These boundaries matter for interpretation. VRKG is most precisely understood not as a generic knowledge-graph formalism, but as a class-level probabilistic graph adapter for text-side VLM fine-tuning. Its central claim is narrower and more technical: a class should be modeled as a region in text-embedding space rather than a single point, and graph propagation over such random vertices can produce more informative textual prototypes than deterministic graph adapters.