---
title: Multi-Modality CFM Architecture
url: https://www.emergentmind.com/topics/multi-modality-conditional-flow-matching-cfm-architecture
type: topic
---

# Multi-Modality CFM Architecture

The term "PepMerge Benchmark" refers to advanced resources in both structural peptide–protein co-design and hyperparameter optimization for large language model merging. In the context of peptide design, PepMerge is a large-scale, systematically clustered benchmark dataset and evaluation suite for assessing the efficacy of generative models targeting peptide–protein binding. Separately, the term also denotes a surrogate-based hyperparameter optimization benchmark (SMM-Bench) for model merging, comprising rigorously defined parameter spaces, empirical surrogates, and reproducible evaluation protocols. This entry details both usages with a focus on rigorously extracted evaluation criteria, dataset construction, and baseline results as reported in the literature.

## 1. Dataset Composition and Clustering Criteria

PepMerge, as introduced in peptide–protein modeling, is a curated, non-redundant collection of 8,365 peptide–protein complexes [2601.04506]. Complexes are aggregated from PepBDB and Q-BioLip, with filtering steps ensuring:

- Peptide length between 3 and 25 residues.
- X-ray crystallographic resolution better than 4 Å.
- Non-redundancy via clustering at 40% receptor sequence identity, computed by MMseqs2.

This clustering yields 292 groups, of which 10 clusters (spanning 158 complexes) constitute a canonical held-out test set. The remaining 282 clusters form the pool for training and validation, with the precise allocation of train/validation splits as per Li et al. (2024). This structure ensures low sequence and structure redundancy between train and test, supporting robust model generalization evaluation.

## 2. Evaluation Metrics and Mathematical Formulation

All model assessments on PepMerge employ a unified suite of geometry, energy, and designability/diversity metrics [2601.04506]:

### Geometry Metrics

- **Amino Acid Recovery Rate (AAR):** Fraction of positions with exact residue identity between prediction and ground truth,
  $$
  \mathrm{AAR} = \frac{1}{N}\sum_{j=1}^N \mathbf{1}\{a_j^\mathrm{pred}=a_j^\mathrm{gt}\} \times 100\%.
  $$
- **C\(_\alpha\) RMSD:** Backbone C\(_\alpha\) root-mean-square deviation post-superposition,
  $$
  \mathrm{RMSD} = \sqrt{\frac{1}{N}\sum_{j=1}^N \bigl\|\mathbf{x}_j^\mathrm{pred}-\mathbf{x}_j^\mathrm{gt}\bigr\|^2} \;\text{(\AA)}
  $$
- **Secondary‐Structure Similarity Ratio (SSR):**
  $$
  \mathrm{SSR} = \frac{1}{N}\sum_{j=1}^N \mathbf{1}\{\mathit{SS}_j^\mathrm{pred}=\mathit{SS}_j^\mathrm{gt}\} \times 100\%.
  $$
- **Binding‐Site Overlap (BSR):** Jaccard overlap of predicted vs. ground truth peptide–receptor contact sets,
  $$
  \mathrm{BSR} = \frac{|B^\mathrm{pred}\cap B^\mathrm{gt}|}{|B^\mathrm{pred}\cup B^\mathrm{gt}|}\times100\%
  $$

### Energy Metrics

- **Complex Stability (Stb):** Proportion of generated complexes with Rosetta all-atom energies lower than the native,
  $$
  \mathrm{Stb} = \frac{1}{M}\sum_{i=1}^M \mathbf{1}\{E_\mathrm{pred}^{(i)}<E_\mathrm{gt}^{(i)}\}\times100\%
  $$
- **Binding Affinity Improvement (Aff):** Proportion with improved (lower) predicted binding affinity,
  $$
  \mathrm{Aff} = \frac{1}{M}\sum_{i=1}^M \mathbf{1}\{\Delta G_\mathrm{pred}^{(i)}<\Delta G_\mathrm{gt}^{(i)}\}\times100\%
  $$

### Design Metrics

- **Designability (Des):** Fraction of generated sequences for which an ESMFold refold returns to within 2 Å RMSD,
  $$
  \mathrm{Des}=\frac{1}{M}\sum_{i=1}^M \mathbf{1}\{\mathrm{RMSD}_\mathrm{fold}^{(i)}<2\,\text{\AA}\}\times100\%
  $$
- **Diversity (Div):** 1 minus mean pairwise TM-score among generated peptides,
  $$
  \mathrm{Div}=1-\frac{2}{M(M-1)}\sum_{i<j}\mathrm{TM}(P_i,P_j)
  $$

This formalism enables thorough, multidimensional assessment of sequence identity, atomic geometry, secondary structural integrity, receptor contact accuracy, stability, energetic favorability, structural recoverability, and novelty.

## 3. Baseline Methods and Quantitative Benchmarks

PepMerge's evaluation framework supports direct comparison across a spectrum of generative architectures for de novo peptide co-design. On the 158-complex test set, leading models were scored as summarized below [2601.04506]:

| Method     | AAR (%)↑ | RMSD (Å)↓ | SSR (%)↑ | BSR (%)↑ | Stb (%)↑ | Aff (%)↑ | Des (%)↑ | Div ↑ |
|------------|----------|-----------|----------|----------|----------|----------|----------|-------|
| Diffusion  | 47.04    | 3.28      | 74.89    | 49.83    | 15.34    | 17.13    | 48.54    | 0.57  |
| PepGLAD    | 50.43    | 3.83      | 80.24    | 19.34    | 20.39    | 10.47    | 75.07    | 0.32  |
| PPIFlow    | 48.35    | 3.59      | 68.13    | 25.94    | 15.77    | 12.08    | 46.53    | 0.51  |
| PepFlow    | 51.25    | 2.07      | 83.46    | 86.89    | 18.15    | 21.37    | 65.22    | 0.42  |
| SurfFlow   | **54.07**| **1.96**  | **85.11**|**87.38** | 22.46    |**22.51** |**73.60** |**0.61**|

These metrics demonstrate that SurfFlow, which models explicit surface geometry and chemistry, achieves the best mean reconstruction accuracy, contact overlap, and structure diversity. For instance, the improvement in AAR (54.1% vs. PepFlow’s 51.3%) and RMSD (1.96 Å vs. 2.07 Å) reflects high-fidelity sequence and structure capture by incorporating surface features.

## 4. Architectural and Methodological Innovations

SurfFlow exemplifies key modeling innovations underpinning modern PepMerge results [2601.04506]:

- **Multi-Modality Conditional Flow Matching:** Combines continuous flow matching (surface positions, normals) and discrete flow matching for categorical surface features, using multiple vector-field predictors and a composite loss function:
  $$
  \mathcal{L}=\lambda_\mathrm{pos}\mathcal{L}_\mathrm{pos}+\lambda_\mathrm{ori}\mathcal{L}_\mathrm{ori}+\lambda_\mathrm{con}\mathcal{L}_\mathrm{con}+\lambda_\mathrm{cat}\mathcal{L}_\mathrm{cat}+\lambda_\mathrm{str}\mathcal{L}_\mathrm{str}.
  $$
- **Equivariant Surface Geometric Network (ESGN):** Models both intra- and inter-surface interactions as a dynamic, heterogeneous graph. Edge features use radial basis function embeddings, while node features encode normal angle features via spherical Fourier-Bessel expansions. SE(3)-equivariance is enforced.
- **Classifier-Free Guidance:** This mechanism enables direct conditional generation (e.g., enforcing cyclic or disulfide constraints) without auxiliary classifiers.

These approaches facilitate fine-grained alignment of peptide–receptor surface complementarity, capturing both geometric and physicochemical binding determinants.

## 5. PepMerge as a Surrogate Optimization Benchmark (Model Merging)

In model merging and hyperparameter optimization, “PepMerge” (SMM-Bench) is a self-contained surrogate benchmark for efficient HPO development [2509.02555]:

- **Two Search Spaces:**
  - **Parameter-Space (PS):**
    $$
    h = (w_1,\,w_2,\,\dots,\,w_{64}) \in [0,1]^{64}
    $$
    with layer-wise continuous weights, $\mathrm{Perf}_{\rm PS}(h) = \mathrm{accuracy}(\text{merged model}(h))$.
  - **Data-Flow-Space (DFS):**
    $$
    h = (c_1,\dots,c_{32}) \in \{0,1,2\}^{32},\quad (s_2,\dots,s_{64})\in [0.4,1.5]^{63}
    $$
    with categorical layer insertion choices and interface scaling factors.

- **Surrogate Models:** For each space and evaluation dataset (gsm8k-ja, MGSM), a LightGBM regression model $f_{\rm sur}$ is trained on tens of thousands of $(h, \mathrm{Perf})$ pairs, supporting millisecond function evaluation and rapid optimizer benchmarking.
- **Metrics:**
  - Mean Squared Error (MSE)
  - $R^2$ (coefficient of determination)
  - Kendall’s Tau (rank correlation)

Standardized API calls enable seamless integration into AutoML HPO workflows. Surrogates capture real LLM merge evaluations, providing high-fidelity optimization testing and reproducibility.

## 6. Usage Protocols and Best Practices

For both the peptide design and surrogate optimization instantiations, reproducibility and methodological rigor are prioritized:

- Strictly enforce input domains:
  - $[0,1]^{64}$ for PS
  - $\{0,1,2\}^{32}\times[0.4,1.5]^{63}$ for DFS.
- In peptide design, adhere to the prescribed data splits and evaluation metrics for fair comparison.
- Protocols emphasize the need to account for possible surrogate overestimation in DFS, and recommend retraining on uniform data subsets for bias mitigation [2509.02555].
- Optimizer trajectory reporting, including best-so-far performance and random seeds, is recommended for full reproducibility.

## 7. Impact and Extensions

PepMerge, in both its molecular design and surrogate HPO instantiations, serves as a robust, reproducible benchmark for advancing generative modeling and optimizer development. The biological benchmark supports unbiased comparison of full-atom and surface-based generative architectures, highlighting the importance of explicit molecular surface modeling for high-precision peptide binder design [2601.04506]. Meanwhile, the surrogate formulation enables rapid, computationally lightweight development of novel hyperparameter optimization techniques, especially valuable for expensive LLM merging tasks [2509.02555]. A plausible implication is that the methodologies and metric formalism of PepMerge may be broadly applicable for benchmarking generative models and optimizers in other large-scale molecular and neural network design settings.

Source: https://www.emergentmind.com/topics/multi-modality-conditional-flow-matching-cfm-architecture