Papers
Topics
Authors
Recent
2000 character limit reached

Holographic-(V)AE: an end-to-end SO(3)-Equivariant (Variational) Autoencoder in Fourier Space (2209.15567v2)

Published 30 Sep 2022 in cs.LG and physics.bio-ph

Abstract: Group-equivariant neural networks have emerged as a data-efficient approach to solve classification and regression tasks, while respecting the relevant symmetries of the data. However, little work has been done to extend this paradigm to the unsupervised and generative domains. Here, we present Holographic-(Variational) Auto Encoder (H-(V)AE), a fully end-to-end SO(3)-equivariant (variational) autoencoder in Fourier space, suitable for unsupervised learning and generation of data distributed around a specified origin in 3D. H-(V)AE is trained to reconstruct the spherical Fourier encoding of data, learning in the process a low-dimensional representation of the data (i.e., a latent space) with a maximally informative rotationally invariant embedding alongside an equivariant frame describing the orientation of the data. We extensively test the performance of H-(V)AE on diverse datasets. We show that the learned latent space efficiently encodes the categorical features of spherical images. Moreover, H-(V)AE's latent space can be used to extract compact embeddings for protein structure microenvironments, and when paired with a Random Forest Regressor, it enables state-of-the-art predictions of protein-ligand binding affinity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Geiger M, Smidt T (2022) e3nn: Euclidean Neural Networks arXiv:2207.09453 [cs].
  2. (2020) One molecular fingerprint to rule them all: drugs, biomolecules, and the metabolome Journal of Cheminformatics 12:43.
  3. (2018) 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data arXiv:1807.02547 [cs, stat].
  4. (2018) Spherical CNNs arXiv:1801.10130 [cs, stat].
  5. Thomas N, et al. (2018) Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds arXiv:1802.08219 [cs].
  6. (2018) Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network arXiv:1806.09231 [cs, stat].
  7. (2020) Spin-Weighted Spherical CNNs arXiv:2006.10731 [cs].
  8. (2020) SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks arXiv:2006.10503 [cs, stat].
  9. (2022) Geometric and Physical Quantities Improve E(3) Equivariant Message Passing arXiv:2110.02905 [cs, stat].
  10. Musaelian A, et al. (2022) Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics arXiv:2204.05249 [cond-mat, physics:physics].
  11. (2022) E(n) Equivariant Graph Neural Networks arXiv:2102.09844 [cs, stat].
  12. Liao YL, Smidt T (2022) Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs arXiv:2206.11990 [physics].
  13. Batzner S, et al. (2022) E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials Nature Communications 13:2453 Number: 1 Publisher: Nature Publishing Group.
  14. Kramer MA (1992) Autoassociative neural networks Computers & Chemical Engineering 16:313–328.
  15. Kingma DP, Welling M (2019) An Introduction to Variational Autoencoders Foundations and Trends® in Machine Learning 12:307–392 arXiv:1906.02691 [cs, stat].
  16. Einstein A (1916) Die Grundlage der allgemeinen Relativitätstheorie Annalen der Physik pp 770–822.
  17. Tung WK (1985) Group Theory in Physics.
  18. Schmidt E (1907) Zur Theorie der linearen und nichtlinearen Integralgleichungen Mathematische Annalen 63:433–476.
  19. Kingma DP, Welling M (2013) Auto-Encoding Variational Bayes.
  20. Higgins I, et al. (2022) beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework.
  21. Cobb OJ, et al. (2021) Efficient Generalized Spherical CNNs arXiv:2010.11661 [astro-ph].
  22. (2016) Layer Normalization arXiv:1607.06450 [cs, stat].
  23. (2020) UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction arXiv:1802.03426 [cs, stat].
  24. Deng L (2012) The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] IEEE Signal Processing Magazine 29:141–142 Conference Name: IEEE Signal Processing Magazine.
  25. Aldenderfer M, Blashfield R (1984) Cluster Analysis (SAGE Publications, Inc., 2455 Teller Road, Thousand Oaks California 91320 United States of America).
  26. Rosenberg A, Hirschberg J (2007) V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure (Association for Computational Linguistics, Prague, Czech Republic), pp 410–420.
  27. Lohit S, Trivedi S (2020) Rotation-Invariant Autoencoders for Signals on Spheres arXiv:2012.04474 [cs].
  28. (2022) Unsupervised Learning of Group Invariant and Equivariant Representations arXiv:2202.07559 [cs].
  29. Su M, et al. (2019) Comparative Assessment of Scoring Functions: The CASF-2016 Update Journal of Chemical Information and Modeling 59:895–913 Publisher: American Chemical Society.
  30. Townshend R, et al. (2021) ATOM3D: Tasks on Molecules in Three Dimensions Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1.
  31. (year?) SHREC 2017: Large-scale 3D Shape Retrieval from ShapeNet Core55.
  32. Berman HM, et al. (2000) The Protein Data Bank Nucleic Acids Research 28:235–242.
  33. Pun MN, et al. (2022) Learning the shape of protein micro-environments with a holographic convolutional neural network Pages: 2022.10.31.514614 Section: New Results.
  34. AlQuraishi M (2019) ProteinNet: a standardized data set for machine learning of protein structure BMC Bioinformatics 20:311.
  35. Shu Z, et al. (2018) Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance arXiv:1806.06503 [cs].
  36. (2020) Rate-Invariant Autoencoding of Time-Series pp 3732–3736 ISSN: 2379-190X.
  37. Mehr E, et al. (2018) Manifold Learning in Quotient Spaces pp 9165–9174 ISSN: 2575-7075.
  38. (2011) Transforming Auto-Encoders, Lecture Notes in Computer Science eds Honkela T, Duch W, Girolami M, Kaski S (Springer, Berlin, Heidelberg), pp 44–51.
  39. (2019) Stacked Capsule Autoencoders arXiv:1906.06818 [cs, stat].
  40. Feige I (2022) Invariant-equivariant representation learning for multi-class data.
  41. Drautz R (2019) Atomic cluster expansion for accurate and transferable interatomic potentials Physical Review B 99:014104 Publisher: American Physical Society.
  42. Musil F, et al. (2021) Physics-Inspired Structural Representations for Molecules and Materials Chemical Reviews 121:9759–9815 Publisher: American Chemical Society.
  43. Uhrin M (2021) Through the eyes of a descriptor: Constructing complete, invertible descriptions of atomic environments Physical Review B 104:144110 arXiv:2104.09319 [cond-mat].
  44. (2021) Rotation-Equivariant Deep Learning for Diffusion MRI arXiv:2102.06942 [cs].
  45. (2022) Predicting Immune Escape with Pretrained Protein Language Model Embeddings (PMLR), pp 110–130 ISSN: 2640-3498.
  46. Boyd JP, Yu F (2011) Comparing seven spectral methods for interpolation and for solving the Poisson equation in a disk: Zernike polynomials, Logan–Shepp ridge polynomials, Chebyshev–Fourier Series, cylindrical Robert functions, Bessel–Fourier expansions, square-to-disk conformal mapping and radial basis functions Journal of Computational Physics 230:1408–1438.
  47. Bowman SR, et al. (2016) Generating Sentences from a Continuous Space arXiv:1511.06349 [cs].
  48. (1996) Effect of data standardization on neural network training Omega 24:385–397.
  49. (2008) Extracting and composing robust features with denoising autoencoders, ICML ’08 (Association for Computing Machinery, New York, NY, USA), pp 1096–1103.
  50. Kingma DP, Ba J (2017) Adam: A Method for Stochastic Optimization arXiv:1412.6980 [cs].
  51. Pedregosa F, et al. (2011) Scikit-learn: Machine Learning in Python Journal of Machine Learning Research 12:2825–2830.
  52. (2018) DeepDTA: deep drug–target binding affinity prediction Bioinformatics 34:i821–i829.
  53. (2019) DeepAffinity: interpretable deep learning of compound–protein affinity through unified recurrent and convolutional neural networks Bioinformatics 35:3329–3338.
  54. (2019) Cormorant: Covariant Molecular Neural Networks arXiv:1906.04015 [physics, stat].
  55. Elnaggar A, et al. (2022) ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning IEEE transactions on pattern analysis and machine intelligence 44:7112–7127.
  56. Gainza P, et al. (2020) Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning Nature Methods 17:184–192 Number: 2 Publisher: Nature Publishing Group.
  57. Nguyen T, et al. (2021) GraphDTA: predicting drug–target binding affinity with graph neural networks Bioinformatics 37:1140–1147.
  58. (2021) Learning from Protein Structure with Geometric Vector Perceptrons arXiv:2009.01411 [cs, q-bio, stat].
  59. (2022) Multi-Scale Representation Learning on Proteins arXiv:2204.02337 [cs, q-bio].
  60. Aykent S, Xia T (2022) GBPNet | Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
  61. (2022) When Geometric Deep Learning Meets Pretrained Protein Language Models arXiv:2212.03447 [cs, q-bio].
  62. Wu F, et al. (2022) Pre-training of Equivariant Graph Matching Networks with Conformation Flexibility for Drug Binding Advanced Science 9:2203796 arXiv:2204.08663 [cs].
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up