Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Multi-Modal Contrastive Diffusion Model for Therapeutic Peptide Generation (2312.15665v2)

Published 25 Dec 2023 in q-bio.QM, cs.LG, and q-bio.BM

Abstract: Therapeutic peptides represent a unique class of pharmaceutical agents crucial for the treatment of human diseases. Recently, deep generative models have exhibited remarkable potential for generating therapeutic peptides, but they only utilize sequence or structure information alone, which hinders the performance in generation. In this study, we propose a Multi-Modal Contrastive Diffusion model (MMCD), fusing both sequence and structure modalities in a diffusion framework to co-generate novel peptide sequences and structures. Specifically, MMCD constructs the sequence-modal and structure-modal diffusion models, respectively, and devises a multi-modal contrastive learning strategy with intercontrastive and intra-contrastive in each diffusion timestep, aiming to capture the consistency between two modalities and boost model performance. The inter-contrastive aligns sequences and structures of peptides by maximizing the agreement of their embeddings, while the intra-contrastive differentiates therapeutic and non-therapeutic peptides by maximizing the disagreement of their sequence/structure embeddings simultaneously. The extensive experiments demonstrate that MMCD performs better than other state-of-theart deep generative methods in generating therapeutic peptides across various metrics, including antimicrobial/anticancer score, diversity, and peptide-docking.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models. arxiv:2205.15019.
  2. Structured Denoising Diffusion Models in Discrete State-Spaces. In Advances in Neural Information Processing Systems, volume 34, 17981–17993. Curran Associates, Inc.
  3. A Survey on Generative Diffusion Model. arxiv:2209.02646.
  4. Machine Learning Designs Non-Hemolytic Antimicrobial Peptides. Chem Sci, 12(26): 9221–9232.
  5. PyRosetta: A Script-Based Interface for Implementing Molecular Modeling Algorithms Using Rosetta. Bioinformatics, 26(5): 689–691.
  6. A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597–1607. PMLR.
  7. Ib-M6 Antimicrobial Peptide: Antibacterial Activity against Clinical Isolates of Escherichia Coli and Molecular Docking. Antibiotics, 9(2): 79.
  8. Deep Attention Based Variational Autoencoder for Antimicrobial Peptide Discovery.
  9. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, volume 33, 6840–6851. Curran Associates, Inc.
  10. A Fresh Look at the Ramachandran Plot and the Occurrence of Standard Structures in Proteins. 1(3-4): 271–283.
  11. Equivariant Diffusion for Molecule Generation in 3D. In Proceedings of the 39th International Conference on Machine Learning, 8867–8887. PMLR.
  12. What makes multi-modal learning better than single (provably). Advances in Neural Information Processing Systems, 34: 10944–10956.
  13. The Survey: Text Generation Models in Deep Learning. J King Saud Univ-com, 34(6, Part A): 2515–2528.
  14. Current Trends of Bioactive Peptides—New Sources and Therapeutic Effect. Foods, 9(7): 846.
  15. What Can Machine Learning Do for Antimicrobial Peptides, and What Can Antimicrobial Peptides Do for Machine Learning? Interface Focus, 7(6): 20160153.
  16. Machine Learning-Enabled Discovery and Design of Membrane-Active Peptides. Bioorgan Med Chem, 26(10): 2708–2718.
  17. Structural Basis of Lipopolysaccharide Extraction by the LptB2FGC Complex. Nature, 567(7749): 486–490.
  18. De novo peptide and protein design using generative adversarial networks: an update. Journal of Chemical Information and Modeling, 62(4): 761–774.
  19. A Text-guided Protein Design Framework. arxiv:2302.04611.
  20. Design Guidelines for Prompt Engineering Text-to-Image Generative Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, 1–23. New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-9157-3.
  21. Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures.
  22. Reliable and Accurate Solution to the Induced Fit Docking Problem for Protein–Ligand Binding. J Chem Theory Comput, 17(4): 2630–2639.
  23. modlAMP: Python for Antimicrobial Peptides. Bioinformatics, 33(17): 2753–2755.
  24. Recurrent Neural Network Model for Constructive Peptide Design. J Chem Inf Model, 58(2): 472–479.
  25. Trends in peptide drug discovery. Nature reviews Drug discovery, 20(4): 309–325.
  26. AMPGAN v2: Machine Learning Guided Design of Antimicrobial Peptides.
  27. E(n) Equivariant Graph Neural Networks. In Proceedings of the 38th International Conference on Machine Learning, 9323–9332. PMLR.
  28. Protein Sequence and Structure Co-Design with Equivariant Translation. arxiv:2210.08761.
  29. Deep Unsupervised Learning Using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, 2256–2265. PMLR.
  30. Generative Modeling by Estimating Gradients of the Data Distribution. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
  31. Discovering Highly Potent Antimicrobial Peptides with Deep Generative Model HydrAMP. Nat Commun, 14(1): 1453.
  32. Discovering highly potent antimicrobial peptides with deep generative model HydrAMP. Nature Communications, 14(1): 1453.
  33. MLACP 2.0: An Updated Machine Learning Tool for Anticancer Peptide Prediction. Comput Struct Biotec, 20: 4473–4480.
  34. APPTEST Is a Novel Protocol for the Automatic Prediction of Peptide Tertiary Structures. Brief Bioinform, 22(6): bbab308.
  35. Diffusion Probabilistic Modeling of Protein Backbones in 3D for the Motif-Scaffolding Problem. arxiv:2206.04119.
  36. Generating Ampicillin-Level Antimicrobial Peptides with Activity-Aware Generative Adversarial Networks. ACS Omega, 5(36): 22847–22851.
  37. Visualizing data using t-SNE. Journal of machine learning research, 9(11).
  38. DiGress: Discrete Denoising Diffusion for Graph Generation. arxiv:2209.14734.
  39. Deep Generative Models for Peptide Design. Digital Discovery, 1(3): 195–208.
  40. Detecting overfitting of deep generative networks via latent recovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11273–11282.
  41. Protein Structure Generation via Folding Diffusion. arxiv:2209.15611.
  42. Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2748–2760.
  43. Protein sequence design with deep generative models. Current Opinion in Chemical Biology, 65: 18–27.
  44. Fast protein structure comparison through effective representation learning with contrastive graph neural networks. PLoS computational biology, 18(3): e1009986.
  45. Structural and functional analysis of protein. In Bioinformatics, 189–206. Elsevier.
  46. Accelerating the Discovery of Anticancer Peptides Targeting Lung and Breast Cancers with the Wasserstein Autoencoder Model and PSO Algorithm. Brief Bioinform, 23(5): bbac320.
  47. Diffusion Models: A Comprehensive Survey of Methods and Applications. arxiv:2209.00796.
  48. Multimodal contrastive training for visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6995–7004.
  49. Deep Learning-Based Bioactive Therapeutic Peptide Generation and Screening. J Chem Inf Model, 63(3): 835–845.
  50. Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction. arxiv:2301.12068.
  51. Label Anchored Contrastive Learning for Language Understanding. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1437–1449.
  52. Weakly Supervised Contrastive Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10042–10051.
  53. Discrete contrastive diffusion for cross-modal and conditional generation. arXiv preprint arXiv:2206.07771.
  54. Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation. arxiv:2206.07771.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now