--- title: Modality-Aware Cooperative Co-Evolution url: https://www.emergentmind.com/topics/modality-aware-cooperative-co-evolution-macc type: topic --- # Modality-Aware Cooperative Co-Evolution Modality-aware Cooperative Co-evolution (MACC) denotes a family of multimodal optimization schemes in which heterogeneous modalities are treated as coupled but non-identical subsystems, and their interaction is regulated by modality-aware mechanisms during iterative joint optimization. In the current literature, MACC appears explicitly as a divide-and-conquer cooperative co-evolutionary framework for multimodal graph neural architecture search in MACC-MGNAS [2510.07325], and it is also used as an organizing principle for bidirectional topology–semantics co-evolution in CO-EVOLVE on heterophilous text-attributed graphs [2603.19596] and for task-aware modality–topology co-evolution in TMTE on multimodal-attributed graphs [2603.27723]. Across these formulations, the common premise is that static, unidirectional integration is inadequate when modalities differ in reliability, topology quality, or functional role. ## 1. Problem settings and conceptual scope MACC is formulated over multimodal graph problems in which structural and attribute channels must be coordinated rather than merely concatenated. In CO-EVOLVE, the setting is heterophilous graph learning on a graph $G=(V,E)$ with node features $X$ and text $T$, where adjacent nodes often belong to different classes and the homophily assumption fails. The paper identifies three failure modes of static pipelines in this setting: semantic–structural dissonance, bidirectional error propagation, and a blind-leading-the-blind phenomenon [2603.19596]. In TMTE, the setting is a multimodal-attributed graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$ with shared topology and modality-specific features, and the motivating observation is that real-world MAGs exhibit noisy interactions, missing connections, and task-agnostic relational structures [2603.27723]. In MACC-MGNAS, the problem is not representation learning on a fixed multimodal graph alone, but architecture optimization over a discrete search space of multimodal graph neural networks for vulnerability co-exploitation prediction on the VulCE graph [2510.07325]. These formulations differ in optimization target. CO-EVOLVE co-evolves a GNN-view of topology and an LLM-view of semantics. TMTE co-evolves graph topology and multimodal representations toward a downstream task. MACC-MGNAS co-evolves modality-specific gene blocks and fusion genes during neural architecture search. A plausible implication is that MACC is best understood not as a single algorithm, but as a recurrent design principle for multimodal systems in which modality heterogeneity must be preserved while cross-modal cooperation remains necessary. ## 2. Modality awareness as a control principle The modality-aware aspect of MACC is implemented through mechanisms that regulate when, how, and to what extent one view should influence another. | Framework | Modality-aware mechanism | Function | |---|---|---| | CO-EVOLVE [2603.19596] | Hard-Structure Conflict-Aware Contrastive Loss, Uncertainty-Gated Consistency, Adaptive Node Gating, Entropy-Aware Adaptive Fusion | Respect topological boundaries, align only confident views, personalize structural and predictive fusion | | MACC-MGNAS [2510.07325] | Modality-specific chromosome blocks, coordinator-worker split, MADTS, SPDI | Capture modality heterogeneity, reduce evaluation cost, adaptively balance exploration and exploitation | | TMTE [2603.27723] | Multi-perspective similarities, softmax modality weighting, smoothness-regularized fusion, cross-modal alignment | Induce topology from modality embeddings and align modalities under task supervision | In CO-EVOLVE, modality awareness is explicit at both training and inference time. Structural embeddings from the GNN are projected into soft prompts for the LLM, while semantic embeddings induce a dynamic semantic graph that rewires the GNN. The fusion between hard/static structure and learned semantic structure is node-wise, with $\alpha_i^{(t)}=\sigma(\mathrm{MLP}_{\mathrm{gate}}(z_i^{(t)}))$, and the consistency term is uncertainty-weighted through predictive entropy, $H(P)=-\sum_c P_c \log P_c$, so that only confident predictions dominate cross-view supervision [2603.19596]. In MACC-MGNAS, modality awareness is architectural and evolutionary. A chromosome is decomposed as $C=(C^{(1)},C^{(2)},\dots,C^{(M)},C^{(\mathrm{fus})})$, where $C^{(m)}$ are modality gene blocks and $C^{(\mathrm{fus})}$ are fusion genes. Local workers evolve modality-specific blocks independently, whereas the coordinator recombines them for global evaluation. The framework further introduces a modality-aware dual-track surrogate, in which a Gaussian-process modality surrogate is fused with global fitness signals using variance-aware weighting, and a similarity-based population diversity indicator regulates crossover and mutation rates [2510.07325]. In TMTE, modality awareness appears in the topology learner itself. For each modality $m$, the model defines $K$ learnable perspective vectors $\{\mathbf{w}^{(m,p)}\}_{p=1}^K$ and computes weighted cosine similarities $$ a_{ij}^{(m,p)}=\mathrm{cos}\!\big(\mathbf{w}^{(m,p)}\odot \mathbf{x}_i^{(m)},\,\mathbf{w}^{(m,p)}\odot \mathbf{x}_j^{(m)}\big), $$ which are aggregated within modality and then across modalities using softmax weights over $\bar{\mathcal{M}}=\mathcal{M}\cup\{|\mathcal{M}|+1\}$. This makes topology induction itself modality-sensitive rather than imposed by a fixed graph prior [2603.27723]. ## 3. Cooperative co-evolution and update dynamics The cooperative co-evolutionary core of MACC is an iterative feedback process in which each subsystem is updated using the current state of the others rather than via a single-pass pipeline. CO-EVOLVE uses a Gauss–Seidel alternating optimization strategy. At iteration $t$, the GNN first encodes the current fused topology $A_t$ to produce structural context $H_{\text{struct},t}$, which is projected into soft prompts $S_t$ for the LLM. The LLM, conditioned on those prompts, outputs updated semantic embeddings $Z_t$ and predictions $P_{\mathrm{LLM},t}$. From $Z_t$, the model constructs a dynamic semantic graph $A_t^{\mathrm{sem}}$, fuses it with the hard structure $A_{\mathrm{hard}}$, and then updates the GNN on the resulting graph. The training schedule freezes the GNN while updating the LLM, rebuilds $A_t^{\mathrm{sem}}$, then freezes the LLM while updating the GNN. The paper characterizes this as a cyclic bidirectional feedback loop that stabilizes collaboration under heterophily and noise [2603.19596]. TMTE also implements a closed-loop procedure, but the evolved object is not a semantic graph for a GNN-LLM pair; it is a task-aware topology coupled to multimodal embeddings. At round $t$, topology evolution computes node-anchor affinities $\mathbf{R}^{(E_t)}$ and the corresponding evolved adjacency $\mathbf{A}^{E_t}$ from current modality embeddings and the fused view. The model then blends the original normalized topology and the evolved topology as $$ \mathbf{Q}^{E_t}=\lambda\,\tilde{\mathbf{A}}+(1-\lambda)\,\mathbf{A}^{E_t}, $$ updates modality embeddings and a fused smooth representation under this current topology, applies cross-modal alignment and task loss, and stops when the relative change in $\mathbf{R}^{(E_t)}$ falls below $\delta$ or after $T$ rounds [2603.27723]. MACC-MGNAS instantiates cooperative co-evolution at the search level. The coordinator maintains a global population $P_t=\{C_1,C_2,\dots,C_N\}$, dispatches decomposed gene blocks to local workers, receives elites $E_t^{(m)}$ and $E_t^{(\mathrm{fus})}$, and reassembles candidate chromosomes using $$ \widetilde{P}^{(t)}=\Big(\prod_{m=1}^M E_t^{(m)}\Big)\times E_t^{(\mathrm{fus})}. $$ The coordinator then performs global evaluation, survivor selection, crossover, mutation, and incumbent update. Workers evolve blocks independently for $T_{LS}$ local steps, but the full architecture is always judged by global validation performance, which preserves inter-modality compatibility [2510.07325]. These three cases occupy different granularities—feature/topology co-evolution, topology/representation co-evolution, and chromosome-block co-evolution—but all adopt the same structural idea: local specialization combined with global coupling. ## 4. Formal objectives and mathematical structure CO-EVOLVE combines supervised prediction, conflict-aware semantic correction, and uncertainty-gated alignment in a single epoch-wise objective: $$ L_{\text{total}}=L_{\text{task}}+L_{\text{conflict}}+L_{\text{cons}}. $$ Its supervised term uses fused predictions $Y_{\text{pred}}=\beta P_{\mathrm{LLM}}+(1-\beta)P_{\mathrm{GNN}}$, with cross-entropy on labeled nodes. The Hard-Structure Conflict-Aware contrastive term is $$ L_{\text{conflict}}=\frac{1}{N}\sum_{i=1}^{N}\Big[\sum_{j\in P_i}\max(0,\Delta^+-z_i^\top z_j)+\lambda\sum_{k\in H_i}\max(0,z_i^\top z_k-\Delta^-)\Big], $$ where $P_i$ are structural positives derived from Personalized PageRank diffusion $\Pi=\gamma(I-(1-\gamma)\hat{A})^{-1}$, and $H_i$ are hard-conflict negatives defined by high semantic similarity but low structural proximity. The consistency term is $$ L_{\text{cons}}=(1-H(P_{\mathrm{GNN}}))\,KL(P_{\mathrm{GNN}}\|P_{\mathrm{LLM}}) +(1-H(P_{\mathrm{LLM}}))\,KL(P_{\mathrm{LLM}}\|P_{\mathrm{GNN}}). $$ This formalization makes modality awareness depend simultaneously on topology, semantic conflict, and predictive uncertainty [2603.19596]. TMTE formalizes topology evolution and modality evolution within a joint task-aware objective. Topology induction begins from multi-perspective similarities and yields an anchor-based affinity matrix $\mathbf{R}^{E_t}$, from which the implicit evolved adjacency is recovered as $$ \mathbf{A}^{E_t}=(\Delta^{E_t})^{-1}\mathbf{R}^{E_t}(\Lambda^{E_t})^{-1}(\mathbf{R}^{E_t})^\top. $$ The smooth fused embedding is defined as the minimizer of $$ \min_{\hat{\mathbf{H}}}\ \|\hat{\mathbf{H}}-\bar{\mathbf{H}}\|_F^2 +\alpha\cdot \mathrm{tr}\big(\hat{\mathbf{H}}^\top(\mathbf{I}-\mathbf{Q}^{E_t})\hat{\mathbf{H}}\big), $$ with closed-form solution $$ \hat{\mathbf{H}}=\frac{1}{\alpha+1}\Big(\mathbf{I}-\frac{\alpha}{\alpha+1}\mathbf{Q}^{E_t}\Big)^{-1}\bar{\mathbf{H}}, $$ approximated by a truncated Neumann series. Cross-modal alignment uses an InfoNCE-style loss with stop-gradient on the fused branch, and the full objective is $$ \mathcal{L}=\mathcal{L}_{\text{mod}}+\eta\,\mathcal{L}_{\text{task}}. $$ Task supervision therefore enters both topology evolution and modality evolution rather than only the final prediction layer [2603.27723]. MACC-MGNAS formulates MACC as discrete architecture optimization. An MGNN architecture is encoded as $$ A=(\phi^{(m)},\psi^{(m)},\gamma,F,R,L), $$ and its chromosome representation is $$ C=(g_1,g_2,\dots,g_K),\qquad C^*=\arg\max_{C\in S} f(C). $$ Global fitness is $$ F_{\text{global}}(C)=\mathrm{Eval}(M(C);D_{\text{val}}), $$ while a modality worker uses a local score $$ F_{\text{local}}^{(m)}(C^{(m)})=\mathrm{ModuleEval}(C^{(m)};D_{\text{val}}^{(m)}). $$ Its surrogate-fused worker objective is $$ \widehat{F}_t^{(m)}(C^{(m)})=\alpha_t\cdot F_{\text{global}}(C)+(1-\alpha_t)\cdot F_{\text{local}}^{(m)}(C^{(m)}), $$ with $$ \alpha_t=\frac{\widehat{\sigma}_{\text{local},t}^2}{\widehat{\sigma}_{\text{local},t}^2+\widehat{\sigma}_{\text{global},t}^2+\epsilon}. $$ Exploration and exploitation are regulated by the diversity measure $$ \mathrm{SPDI}(P_t)=\frac{2}{N(N-1)}\sum_{i