---
title: Mixture of Ego-Graphs Fusion (MoEGF)
url: https://www.emergentmind.com/topics/mixture-of-ego-graphs-fusion-moegf
type: topic
---

# Mixture of Ego-Graphs Fusion (MoEGF)

Mixture of Ego-Graphs Fusion (MoEGF) is a fine-grained graph fusion mechanism designed for multi-view clustering within the Mixture of Ego-Graphs Contrastive Representation Learning (MoEGCL) framework. Diverging from traditional view-level fusion, MoEGF aggregates per-sample (ego-graph) structures from multiple data views using a Mixture-of-Experts (MoE) paradigm. This design enables adaptive, sample-specific fusion of ego-graphs to produce a fused adjacency matrix for downstream graph neural network (GNN) processing, substantially enhancing clustering performance by capturing localized multi-view interactions [2511.05876].

## 1. Mathematical Formulation of MoEGF

Given $M$ data views, each sample $i$’s representation in view $m$ is encoded as $z_i^m = f^m(x_i^m) \in \mathbb{R}^{d_\psi}$. For each view, a $k$-nearest-neighbor (KNN) adjacency matrix $S^m \in \{0,1\}^{N \times N}$ is built:

$$
S_{ij}^m = \begin{cases}
1 & \text{if } j \in K_i^m \\
0 & \text{otherwise}
\end{cases}
$$

The sample’s ego-graph in view $m$ is the binary vector $V_i^m := (S_{i1}^m,\,S_{i2}^m,\,\dots,S_{iN}^m) \in \{0,1\}^N$.

The concatenated embedding $z_i := [z_i^1; z_i^2; \ldots; z_i^M] \in \mathbb{R}^{M d_\psi}$ serves as the gating input to a two-layer MLP, yielding softmax weights $C_i \in \mathbb{R}^M$:

$$
C_i = \text{softmax}(\text{mlp}^{(1)}(z_i))
$$

The fused ego-graph vector for sample $i$ is then the convex combination:

$$
V_i = \sum_{m=1}^M C_i^m V_i^m
$$

The stacked set of $V_i$ forms the fused adjacency matrix $S \in \mathbb{R}^{N \times N}$.

To incorporate feature information, a two-layer GCN is applied:

$$
\tilde{S} = I_N + S;\quad \tilde{D}_{ii} = \sum_j \tilde{S}_{ij}
$$
$$
\tilde{Z} = \sigma\left( \tilde{D}^{-1/2} \tilde{S} \tilde{D}^{-1/2} \left( \sigma\left( \tilde{D}^{-1/2} \tilde{S} \tilde{D}^{-1/2} Z W^0 \right) W^1 \right) \right)
$$

where $Z = [z_1; \dots; z_N]$ and $W^0, W^1$ are learnable parameters.

## 2. Algorithmic Implementation

A typical training epoch for MoEGF within MoEGCL, assuming minibatch size $B$, comprises the following steps:

1. **Encoding:** For each sample and each view, $z_i^m = f^m(x_i^m)$; form concatenated $z_i$.
2. **KNN Construction:** For each view, build $V_i^m$ as the row of $S^m$.
3. **Gating:** Apply $\text{mlp}^{(1)}$ and softmax to $z_i$, outputting $C_i$.
4. **Fusion:** Compute $V_i$ as the weighted sum of $V_i^m$ using $C_i$.
5. **Adjacency Assembly:** Assemble $\{V_i\}$ into the fused adjacency $S_\text{batch}$.
6. **GCN Forward:** Apply two-layer GCN to obtain $\tilde{Z}_\text{batch}$.
7. **Projection Heads:** Apply separate MLPs to $\tilde{z}_i$ ($\hat{h}_i$) and $z_i^m$ ($h_i^m$).
8. **Loss Computation:** Compute autoencoder reconstruction loss $\mathcal{L}_\text{Rec}$ and ego-graph contrastive loss $\mathcal{L}_\text{Egc}$.
9. **Optimization:** Sum total loss $\mathcal{L} = \mathcal{L}_\text{Rec} + \lambda\mathcal{L}_\text{Egc}$, backpropagate, and update all parameters.

The dominant computational cost per batch is $O(B^2 M)$ for the fusion and $O(B d_\psi d_g)$ for the feature transformations.

## 3. Comparison to View-Level Fusion Paradigms

Traditional deep multi-view clustering approaches construct one graph per view and perform graph fusion at the view level, assigning global weights to views and yielding a mixture for all samples. In contrast, MoEGF outputs sample-wise fusion coefficients $C_i$, enabling personalized graph structures per sample.

Empirical results show substantial accuracy gains from this design. For example, removing MoEGF and instead concatenating $z_i^m$ leads to absolute clustering accuracy (ACC) drops of 37.6% (Caltech5V), 6% (RGBD), and 41% (WebKB). MoEGF outperforms state-of-the-art multi-view clustering baselines by over 8% ACC on WebKB and by 4–7% on RGBD [2511.05876].

## 4. Design Decisions, Hyperparameters, and Implementation Notes

Key implementation features and hyperparameter choices are summarized below:

| Component               | Setting                                             | Notes                                               |
|-------------------------|----------------------------------------------------|-----------------------------------------------------|
| Number of Experts       | $K = M$                                            | One per view                                        |
| Gating Network (mlp¹)   | Two-layer MLP, softmax output, dropout $p=0.1$    | Hidden dim not specified; $d_\psi=512$ used         |
| KNN Graph (per view)    | $k$-nearest neighbors, $k \in [5,10]$ typical      | Binary adjacency                                    |
| Embedding Dimensions    | $d_\psi = 512$, $d_\phi=128$                      | \\
| Batch Size              | $b = 256$                                          | \\
| Training Epochs         | $T_p = 200$ (pre-train), $T_f = 300$ (fine-tune)   | \\
| MoEGF Mixture Weights   | Dense softmax, no regularizer                      | \\

Implementation is amenable to minibatch parallelism and scales as $O(B^2 M)$ with batch size $B$ and number of views $M$, dominated by fusion and GCN costs. KNN adjacency and gating can be precomputed or batched for efficiency.

## 5. Integration Within the MoEGCL Framework

MoEGF operates immediately after per-view autoencoder embedding. It delivers the fused adjacency $S$ to a two-layer GCN, producing structure-aware node embeddings $\tilde{Z}$. The subsequent Ego Graph Contrastive Learning (EGCL) module aligns fused and per-view representations via the loss

$$
\mathcal{L}_{\mathrm{Egc}} = -\frac{1}{2N} \sum_{i=1}^N \sum_{m=1}^M \log\frac{ \exp ( \cos( \hat h_i, h_i^m ) / \tau ) }{ \sum_{j=1}^N \exp( (1 - S_{ij}) \cos( \hat h_i, h_j^m ) / \tau ) - \exp(1/\tau) }
$$

Gradients from $\mathcal{L}_{\mathrm{Egc}}$ propagate through the GNN layers and the MoEGF gating MLP, ensuring that the fused structure is optimized for cluster-aware representation learning.

## 6. Broader Context and Related Methods

MoEGF advances prior fusion mechanisms for multi-view graph data in deep clustering tasks by providing an alternative to global or view-level fusion. The approach is conceptually related to the class of Mixture-of-Experts graph methods, including MoG [2405.14260], which extend MoE strategies to graph sparsification and subgraph selection via per-node adaptive fusion. MoEGF, however, is specifically tailored for sample-level ego-graph combination, with direct integration into a contrastive clustering framework. Both approaches share the use of per-node/per-sample gating and fusion, but differ in fusion domains (ego-adjacency in MoEGF, Grassmannian spectral fusion in MoG).

A plausible implication is that the fundamental Mixture-of-Experts paradigm, when applied locally to ego-centric structures, generalizes beyond clustering to other graph learning problems, including efficient sparsification, node classification, and adaptive edge selection.

## 7. Empirical Performance and Observed Impact

On benchmark datasets, MoEGF within MoEGCL results in pronounced accuracy improvements over both naive and coarse-grained fusion strategies, as evinced by substantial drops in ACC upon ablation. The empirical findings underscore the significance of fine-grained, per-sample fusion in capturing the mutual reinforcement and complementarity of multi-view graph signals. The method’s flexible, differentiable construction further allows direct end-to-end optimization with contrastive learning objectives, significantly advancing state-of-the-art performance in multi-view clustering settings [2511.05876].

Source: https://www.emergentmind.com/topics/mixture-of-ego-graphs-fusion-moegf