Modality-Aware Cooperative Co-Evolution

• MACC is a multimodal optimization framework that uses modality-aware mechanisms to regulate cooperative co-evolution of heterogeneous graph subsystems.
• It integrates local modality-specific updates with global feedback to fuse structural, semantic, and task-driven information effectively.
• Empirical results show enhanced accuracy, robustness, and efficiency across tasks, though challenges remain in scalability and architectural flexibility.

Modality-aware Cooperative Co-evolution (MACC) denotes a family of multimodal optimization schemes in which heterogeneous modalities are treated as coupled but non-identical subsystems, and their interaction is regulated by modality-aware mechanisms during iterative joint optimization. In the current literature, MACC appears explicitly as a divide-and-conquer cooperative co-evolutionary framework for multimodal graph neural architecture search in MACC-MGNAS (Wang et al., 23 Sep 2025), and it is also used as an organizing principle for bidirectional topology–semantics co-evolution in CO-EVOLVE on heterophilous text-attributed graphs (Xing et al., 20 Mar 2026) and for task-aware modality–topology co-evolution in TMTE on multimodal-attributed graphs (Zhu et al., 29 Mar 2026). Across these formulations, the common premise is that static, unidirectional integration is inadequate when modalities differ in reliability, topology quality, or functional role.

1. Problem settings and conceptual scope

MACC is formulated over multimodal graph problems in which structural and attribute channels must be coordinated rather than merely concatenated. In CO-EVOLVE, the setting is heterophilous graph learning on a graph $G=(V,E)$ with node features $X$ and text $T$, where adjacent nodes often belong to different classes and the homophily assumption fails. The paper identifies three failure modes of static pipelines in this setting: semantic–structural dissonance, bidirectional error propagation, and a blind-leading-the-blind phenomenon (Xing et al., 20 Mar 2026). In TMTE, the setting is a multimodal-attributed graph $$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$ with shared topology and modality-specific features, and the motivating observation is that real-world MAGs exhibit noisy interactions, missing connections, and task-agnostic relational structures (Zhu et al., 29 Mar 2026). In MACC-MGNAS, the problem is not representation learning on a fixed multimodal graph alone, but architecture optimization over a discrete search space of multimodal graph neural networks for vulnerability co-exploitation prediction on the VulCE graph (Wang et al., 23 Sep 2025).

These formulations differ in optimization target. CO-EVOLVE co-evolves a GNN-view of topology and an LLM-view of semantics. TMTE co-evolves graph topology and multimodal representations toward a downstream task. MACC-MGNAS co-evolves modality-specific gene blocks and fusion genes during neural architecture search. A plausible implication is that MACC is best understood not as a single algorithm, but as a recurrent design principle for multimodal systems in which modality heterogeneity must be preserved while cross-modal cooperation remains necessary.

2. Modality awareness as a control principle

The modality-aware aspect of MACC is implemented through mechanisms that regulate when, how, and to what extent one view should influence another.

Framework	Modality-aware mechanism	Function
CO-EVOLVE (Xing et al., 20 Mar 2026)	Hard-Structure Conflict-Aware Contrastive Loss, Uncertainty-Gated Consistency, Adaptive Node Gating, Entropy-Aware Adaptive Fusion	Respect topological boundaries, align only confident views, personalize structural and predictive fusion
MACC-MGNAS (Wang et al., 23 Sep 2025)	Modality-specific chromosome blocks, coordinator-worker split, MADTS, SPDI	Capture modality heterogeneity, reduce evaluation cost, adaptively balance exploration and exploitation
TMTE (Zhu et al., 29 Mar 2026)	Multi-perspective similarities, softmax modality weighting, smoothness-regularized fusion, cross-modal alignment	Induce topology from modality embeddings and align modalities under task supervision

In CO-EVOLVE, modality awareness is explicit at both training and inference time. Structural embeddings from the GNN are projected into soft prompts for the LLM, while semantic embeddings induce a dynamic semantic graph that rewires the GNN. The fusion between hard/static structure and learned semantic structure is node-wise, with $$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$, and the consistency term is uncertainty-weighted through predictive entropy, $H(P)=-\sum_c P_c \log P_c$, so that only confident predictions dominate cross-view supervision (Xing et al., 20 Mar 2026).

In MACC-MGNAS, modality awareness is architectural and evolutionary. A chromosome is decomposed as $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$, where $C^{(m)}$ are modality gene blocks and $C^{(\mathrm{fus})}$ are fusion genes. Local workers evolve modality-specific blocks independently, whereas the coordinator recombines them for global evaluation. The framework further introduces a modality-aware dual-track surrogate, in which a Gaussian-process modality surrogate is fused with global fitness signals using variance-aware weighting, and a similarity-based population diversity indicator regulates crossover and mutation rates (Wang et al., 23 Sep 2025).

In TMTE, modality awareness appears in the topology learner itself. For each modality $m$, the model defines $X$0 learnable perspective vectors $X$1 and computes weighted cosine similarities

$X$2

which are aggregated within modality and then across modalities using softmax weights over $X$3. This makes topology induction itself modality-sensitive rather than imposed by a fixed graph prior (Zhu et al., 29 Mar 2026).

3. Cooperative co-evolution and update dynamics

The cooperative co-evolutionary core of MACC is an iterative feedback process in which each subsystem is updated using the current state of the others rather than via a single-pass pipeline.

CO-EVOLVE uses a Gauss–Seidel alternating optimization strategy. At iteration $X$4, the GNN first encodes the current fused topology $X$5 to produce structural context $X$6, which is projected into soft prompts $X$7 for the LLM. The LLM, conditioned on those prompts, outputs updated semantic embeddings $X$8 and predictions $X$9. From $T$0, the model constructs a dynamic semantic graph $T$1, fuses it with the hard structure $T$2, and then updates the GNN on the resulting graph. The training schedule freezes the GNN while updating the LLM, rebuilds $T$3, then freezes the LLM while updating the GNN. The paper characterizes this as a cyclic bidirectional feedback loop that stabilizes collaboration under heterophily and noise (Xing et al., 20 Mar 2026).

TMTE also implements a closed-loop procedure, but the evolved object is not a semantic graph for a GNN-LLM pair; it is a task-aware topology coupled to multimodal embeddings. At round $T$4, topology evolution computes node-anchor affinities $T$5 and the corresponding evolved adjacency $T$6 from current modality embeddings and the fused view. The model then blends the original normalized topology and the evolved topology as

$T$7

updates modality embeddings and a fused smooth representation under this current topology, applies cross-modal alignment and task loss, and stops when the relative change in $T$8 falls below $T$9 or after $$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$0 rounds (Zhu et al., 29 Mar 2026).

MACC-MGNAS instantiates cooperative co-evolution at the search level. The coordinator maintains a global population $$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$1, dispatches decomposed gene blocks to local workers, receives elites $$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$2 and $$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$3, and reassembles candidate chromosomes using

$$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$4

The coordinator then performs global evaluation, survivor selection, crossover, mutation, and incumbent update. Workers evolve blocks independently for $$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$5 local steps, but the full architecture is always judged by global validation performance, which preserves inter-modality compatibility (Wang et al., 23 Sep 2025).

These three cases occupy different granularities—feature/topology co-evolution, topology/representation co-evolution, and chromosome-block co-evolution—but all adopt the same structural idea: local specialization combined with global coupling.

4. Formal objectives and mathematical structure

CO-EVOLVE combines supervised prediction, conflict-aware semantic correction, and uncertainty-gated alignment in a single epoch-wise objective:

$$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$6

Its supervised term uses fused predictions $$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$7, with cross-entropy on labeled nodes. The Hard-Structure Conflict-Aware contrastive term is

$$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$8

where $$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$$9 are structural positives derived from Personalized PageRank diffusion $$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$0, and $$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$1 are hard-conflict negatives defined by high semantic similarity but low structural proximity. The consistency term is

$$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$2

This formalization makes modality awareness depend simultaneously on topology, semantic conflict, and predictive uncertainty (Xing et al., 20 Mar 2026).

TMTE formalizes topology evolution and modality evolution within a joint task-aware objective. Topology induction begins from multi-perspective similarities and yields an anchor-based affinity matrix $$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$3, from which the implicit evolved adjacency is recovered as

$$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$4

The smooth fused embedding is defined as the minimizer of

$$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$5

with closed-form solution

$$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$6

approximated by a truncated Neumann series. Cross-modal alignment uses an InfoNCE-style loss with stop-gradient on the fused branch, and the full objective is

$$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$7

Task supervision therefore enters both topology evolution and modality evolution rather than only the final prediction layer (Zhu et al., 29 Mar 2026).

MACC-MGNAS formulates MACC as discrete architecture optimization. An MGNN architecture is encoded as

$$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$8

and its chromosome representation is

$$\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$$9

Global fitness is

$H(P)=-\sum_c P_c \log P_c$0

while a modality worker uses a local score

$H(P)=-\sum_c P_c \log P_c$1

Its surrogate-fused worker objective is

$H(P)=-\sum_c P_c \log P_c$2

with

$H(P)=-\sum_c P_c \log P_c$3

Exploration and exploitation are regulated by the diversity measure

$H(P)=-\sum_c P_c \log P_c$4

which switches the crossover and mutation rates between high and low modes according to a threshold $H(P)=-\sum_c P_c \log P_c$5 (Wang et al., 23 Sep 2025).

5. Empirical performance, robustness, and ablations

CO-EVOLVE reports results on Reddit, Instagram, and WikiCS. It achieves Instagram best Acc $H(P)=-\sum_c P_c \log P_c$6 and F1 $H(P)=-\sum_c P_c \log P_c$7, WikiCS best Acc $H(P)=-\sum_c P_c \log P_c$8 and second-best F1 $H(P)=-\sum_c P_c \log P_c$9, and Reddit second Acc $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$0 with F1 $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$1. The paper reports average improvements of $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$2 in Accuracy and $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$3 in F1-score over state-of-the-art baselines. Under False Semantic Friends corruption at $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$4, Reddit Acc is $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$5 for CO-EVOLVE versus $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$6 for LLaGA, and under Missing Structural Links with $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$7 deletion on Reddit, GCN drops from $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$8 to $$C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$$9 whereas CO-EVOLVE drops only from $C^{(m)}$0 to $C^{(m)}$1. Ablations show that removing the Semantic Structure Learner causes the largest drop, removing Conflict-Aware Loss yields the second-largest degradation, and removing Soft Prompts or Uncertainty-Gated Consistency still causes $C^{(m)}$2–$C^{(m)}$3 drops (Xing et al., 20 Mar 2026).

MACC-MGNAS evaluates on VulCE with validation F1 as fitness, final retraining on train+val, and test evaluation across $C^{(m)}$4 independent runs. The full method reports mean F1 $C^{(m)}$5, best F1 $C^{(m)}$6, and GPU-hours $C^{(m)}$7, with statistically significant improvement over all baselines under paired $C^{(m)}$8-test with $C^{(m)}$9. Reported baselines include PSO, EDA, GA, BO, MyGO, C2RS, DC-NAS, and MAGCN. The abstract states an $C^{(\mathrm{fus})}$0 F1 improvement over the state-of-the-art competitor and $C^{(\mathrm{fus})}$1 reduction in computation cost. Ablation results identify MACC as the primary source of accuracy gains relative to centralized GA, MADTS as the major source of compute savings with $C^{(\mathrm{fus})}$2 GPU-hours relative to the version without MADTS, and SPDI as improving stability and convergence (Wang et al., 23 Sep 2025).

TMTE is evaluated on $C^{(\mathrm{fus})}$3 MAG datasets—RedditS, Movies, Grocery, DY, Bili Dance, SemArt, Flickr30k, Ele-fashion, Goodreads—and $C^{(\mathrm{fus})}$4 non-graph multimodal dataset, MVSA, across $C^{(\mathrm{fus})}$5 tasks: node classification, link prediction, node clustering, cross-modal retrieval, graph-to-text generation, and graph-to-image generation. Reported graph-centric gains include $C^{(\mathrm{fus})}$6 Acc and $C^{(\mathrm{fus})}$7 F1 over the runner-up in node classification, $C^{(\mathrm{fus})}$8 MRR and $C^{(\mathrm{fus})}$9 Hits@3 in link prediction, and $m$0 NMI and $m$1 ARI in clustering. For modality-centric tasks, the paper reports MRR $m$2 on Ele-fashion retrieval, BLEU-4 $m$3 on Flickr30k G2Text, and CLIP-Score $m$4 on SemArt G2Image. Ablations labeled One-shot TE, Only ME, and Task-agnostic E each cause notable drops, which the paper interprets as evidence for the necessity of iterative topology-plus-modality co-evolution and task-aware supervision (Zhu et al., 29 Mar 2026).

Taken together, these results indicate that MACC-style systems are evaluated not only by peak accuracy or F1, but also by robustness to semantic corruption, resilience to edge deletion, search efficiency, convergence stability, and transfer across graph-centric and modality-centric tasks.

6. Interpretation, limitations, and extensions

A recurring misconception is to treat MACC as a single fixed recipe. The cited literature instead assigns the label to several related constructions: cooperative co-evolutionary neural architecture search, GNN–LLM co-evolution on heterophilous graphs, and task-aware topology–modality co-evolution on MAGs (Wang et al., 23 Sep 2025, Xing et al., 20 Mar 2026, Zhu et al., 29 Mar 2026). What is shared is the insistence that modality heterogeneity should be explicitly modeled, and that cooperation should be gated rather than unconditional.

The principal limitations are also heterogeneous. MACC-MGNAS notes that evolutionary NAS remains expensive on very large graphs even with surrogates, that fixed chromosome length $m$5 limits discovery of novel macro-structures, and that the reported experiments assume static graphs rather than dynamic vulnerability data (Wang et al., 23 Sep 2025). TMTE notes its assumption of a shared topology across modalities, dependence on pre-trained encoders, possible difficulty on highly unaligned modalities or ultra-large graphs, and sensitivity to the number of perspectives $m$6 and anchor sampling strategy (Zhu et al., 29 Mar 2026). CO-EVOLVE describes failure cases and mitigations rather than a standalone limitations section: severe hallucinations may motivate increasing $m$7, strengthening the conflict-aware loss, or biasing the gate toward hard structure early in training; extremely noisy topology may motivate reducing $m$8, using higher $m$9 for $X$00, or raising $X$01; collapse of one modality may motivate freezing it longer and annealing $X$02 (Xing et al., 20 Mar 2026).

Extension directions are correspondingly broad. CO-EVOLVE explicitly sketches image–text and audio–text variants by replacing soft structural prompts, redefining semantic graphs, and grounding conflict mining in region, scene, spatial, instance, or temporal graphs (Xing et al., 20 Mar 2026). MACC-MGNAS states that the MACC paradigm is applicable to multimedia, biomedical, and cyber-physical systems by redefining modality-specific blocks and fusion genes and training modality surrogates on domain-specific local objectives (Wang et al., 23 Sep 2025). TMTE proposes future work on advanced graph learners, distributed or federated settings, richer metric learning, adaptive modality weighting, temporal or hypergraph MAGs, and more generative objectives (Zhu et al., 29 Mar 2026).

In this sense, MACC is not merely a fusion heuristic. It is a technical stance on multimodal graph systems: modality-specific structure should remain explicit; cross-modal exchange should be calibrated by topology, uncertainty, diversity, or task loss; and the joint system should evolve through iterative feedback rather than through a static chain.