Symphony: Equivariant 3D Molecule Generator

• Symphony is an E(3)-equivariant autoregressive generative model that builds 3D molecular geometries by iteratively placing atoms relative to a chosen focus atom.
• It employs a novel point-centered spherical harmonic parameterization to directly model the angular distribution of the next atom’s position, overcoming limitations of rotationally invariant features.
• The architecture leverages two equivariant message-passing networks and discretized radial shells to predict focus, species, and spatial distributions, leading to improved validity and sampling efficiency.

Searching arXiv for the specified Symphony paper and closely related 3D molecular generation work. First, retrieving the main paper by arXiv ID. Now searching for related autoregressive and diffusion baselines mentioned in the paper. Symphony is an $E(3)$-equivariant autoregressive generative model for 3D molecular geometries that iteratively builds a molecule from molecular fragments, specifically by placing atoms one at a time relative to a chosen focus atom. Its central contribution is a “point-centered” spherical harmonic parameterization of the conditional distribution of the next atom’s 3D position, learned from higher-degree equivariant message-passing features. In contrast to earlier autoregressive approaches that rely on rotationally invariant features, Symphony uses higher-degree $E(3)$-equivariant features, allowing local orientation to be inferred from the environment around a single atom and enabling direct modeling of angular structure on $S^2$ together with discretized radial shells (Daigavane et al., 2023).

1. Definition and design objective

Symphony is motivated by the requirement that realistic 3D molecule generation must capture strong local geometric correlations, chemical constraints, and Euclidean symmetries, while retaining the flexibility of sequential growth and fragment completion. The model learns the probabilistic mapping $S^n \to S^{n+1}$ by factorizing each generation step into a focus distribution, a target-species distribution, and a target-position distribution, and then sampling these conditionals autoregressively until a global STOP token is emitted or a step limit is reached (Daigavane et al., 2023).

A central limitation of prior autoregressive models such as G-SchNet and G-SphereNet is that they use rotationally invariant features. With only invariants, local orientation cannot be inferred from a single point, which forces triangulation using at least three atoms and auxiliary unphysical tokens to break symmetry, and can make predictions sensitive to distant atoms or to discrete triangulation choices. Symphony replaces this invariant-only strategy with higher-degree equivariant features and a spherical harmonic representation centered at one focus atom, so that atom placement can be expressed directly in the local frame induced by the equivariant representation.

Training sequences are constructed from ground-truth molecules by a minimum spanning tree procedure. At each step, the next atom is the closest atom to the current fragment $S^n$; the focus atom $f_{n+1}$ lies in $S^n$, the next species is $Z_{n+1}$, and the model predicts the relative position of the new atom with respect to that focus. At test time, generation starts from one atom and proceeds atom-by-atom; Symphony places atoms rather than larger functional groups, and uses $N_{\max}=35$ as the generation limit.

2. Equivariance and spherical harmonic representation

The representation-theoretic core of Symphony is the use of higher-degree $E(3)$-equivariant features. A feature of degree $E(3)$0 transforms under rotations by the irreducible representation $E(3)$1 of $E(3)$2:

$E(3)$3

Degree-$E(3)$4 features are rotational invariants, whereas higher-degree features carry directional information. Features are translation-invariant by construction (Daigavane et al., 2023).

Real spherical harmonics $E(3)$5 form an orthonormal basis on $E(3)$6. Grouping all orders at fixed degree gives

$E(3)$7

with the equivariance relation

$E(3)$8

Any angular function can therefore be expanded as

$E(3)$9

Symphony applies this expansion not to a global field but to the conditional logits of the next atom position around a chosen focus atom. For each discretized radial shell $S^2$0, the model represents the angular logits as

$S^2$1

and defines

$S^2$2

The coefficients are predicted from the degree-$S^2$3 focus features and the target species embedding:

$S^2$4

To bypass truncation limits at finite degree, Symphony sums multiple channels of spherical signals after a nonlinearity:

$S^2$5

Two channels suffice empirically.

3. Architecture and sequential generation

Symphony uses two $S^2$6-equivariant message-passing neural networks. E3SchNet, described as a higher-degree generalization of SchNet, is used for predicting the focus and atom type. NequIP is used for predicting the position distribution. Both return degree-$S^2$7 features $S^2$8 for each atom $S^2$9 up to a chosen maximum degree $S^n \to S^{n+1}$0 (Daigavane et al., 2023).

Neighborhoods are cutoff-based:

$S^n \to S^{n+1}$1

Message passing proceeds for $S^n \to S^{n+1}$2 iterations. The NequIP update is

$S^n \to S^{n+1}$3

with learned $S^n \to S^{n+1}$4 and tensor products decomposed into irreps. E3SchNet uses

$S^n \to S^{n+1}$5

where the scalar weights are

$S^n \to S^{n+1}$6

The stepwise factorization is

• $S^n \to S^{n+1}$7 over atoms in $S^n \to S^{n+1}$8 plus STOP,
• $S^n \to S^{n+1}$9,
• $S^n$0.

The first two are computed from scalar features:

$S^n$1

$S^n$2

Inference samples $S^n$3, then $S^n$4, then the position; for the latter, the model first marginalizes over angles to pick $S^n$5, then samples $S^n$6 on that sphere.

Symphony does not explicitly enforce valence or bond types during generation. Bonds are assigned post hoc by xyz2mol, OpenBabel, or distance-based lookup. This design keeps the generative mechanism focused on geometry rather than explicit graph constraints.

4. Symmetry guarantees, radial parameterization, and training

The model is constructed to satisfy three symmetry requirements. First, $S^n$7 and $S^n$8 are $S^n$9-invariant. Second, $f_{n+1}$0 is $f_{n+1}$1-equivariant. Third, the conditionals satisfy the appropriate permutation equivariance or invariance properties with respect to atom indexing (Daigavane et al., 2023).

The key equivariance theorem for the position distribution is

$f_{n+1}$2

assuming the embedder is $f_{n+1}$3-equivariant and translation-invariant. The proof uses both equivariant feature transport,

$f_{n+1}$4

and the corresponding equivariance of spherical harmonics.

Radially, Symphony discretizes $f_{n+1}$5 to 64 uniformly spaced values from $f_{n+1}$6 to $f_{n+1}$7, covering QM9 bond lengths. For training the position head, the target distribution is a smooth Dirac-delta approximation around the target direction together with a radial Gaussian, truncated at spherical harmonic cutoff $f_{n+1}$8. Training uses teacher forcing with ground-truth fragment sequences. All distributions are trained by minimizing KL divergence to their empirical or constructed targets:

$f_{n+1}$9

where $S^n$0.

Numerical integration uses a uniform radial grid and Spherical Gauss-Legendre quadrature over angles. Fast evaluation over the spherical grid uses FFTs in e3nn-jax. Zero-centered Gaussian noise with variance $S^n$1 is added to input positions. Parameters are trained with Adam at learning rate $S^n$2.

5. Implementation, benchmarks, and empirical results

Symphony is evaluated on QM9 after removing 3054 “uncharacterized” molecules, leaving 130,831 molecules split into 100k train, 13,083 validation, and 17,748 test examples. The reported architecture uses NequIP with $S^n$3, $S^n$4, $S^n$5, and $S^n$6; E3SchNet uses $S^n$7 and the same cutoff; position modeling uses 2 channels in practice. Best validation loss is selected over 8,000,000 training steps with batch size 16 fragments. The paper notes that Symphony was trained for approximately 80 epochs, versus approximately 330 for G-SchNet and approximately 1100 for EDM (Daigavane et al., 2023).

On sampling efficiency, measured on an NVIDIA RTX A5000, Symphony requires 0.293 sec/molecule, compared with 0.930 sec/mol for EDM, 0.011 sec/mol for G-SchNet, and 0.006 sec/mol for G-SphereNet. The model is therefore approximately $S^n$8 faster than EDM at sampling, although slower than earlier autoregressive baselines because of tensor-product operations.

On validity and uniqueness, Symphony consistently improves on earlier autoregressive models. Using xyz2mol, validity is 83.50 for Symphony, 86.74 for EDM, 74.97 for G-SchNet, and 26.92 for G-SphereNet. Using OpenBabel, Symphony reaches 74.69, versus 77.75 for EDM, 61.83 for G-SchNet, and 9.86 for G-SphereNet. Using lookup-table bond assignment, Symphony reaches 68.11, while EDM reaches 90.77. Uniqueness via xyz2mol is 97.98 for Symphony, 99.16 for EDM, 96.73 for G-SchNet, and 21.69 for G-SphereNet.

PoseBusters sanity checks show high physical plausibility: 99.92 for All Atoms Connected, 99.56 for Reasonable Bond Angles, 98.72 for Reasonable Bond Lengths, 100.00 for Aromatic Ring Flatness, 99.07 for Double Bond Flatness, 95.65 for Reasonable Internal Energy, and 98.16 for No Internal Steric Clash. On training-set statistics, Symphony achieves the lowest bispectra MMD for several local environments, including C: C3,H1 and N: C2,H1, indicating strong angular fidelity; bond-length distributions are wider than QM9 and EDM, though the model performs best on single C–H and N–H among the bond categories discussed.

Fragment completion also generalizes well. For fragments with one H removed, valid completion rate on train fragments decreases from 98.53 to 95.57 over training progression, compared with 97.91 for G-SchNet. On test fragments, Symphony decreases from 98.66 to 95.43, compared with 98.15 for G-SchNet.

6. Ablations, limitations, and related context

Ablation studies identify higher-degree equivariant features as important for validity. Increasing NequIP degree from $S^n$9 to $Z_{n+1}$0 yields a large validity jump, with further increases giving smaller gains while retaining $Z_{n+1}$1. Increasing the number of position channels helps notably when $Z_{n+1}$2; two channels suffice in practice. By contrast, increasing E3SchNet degree from $Z_{n+1}$3 to $Z_{n+1}$4 has minimal effect on the focus and species tasks. Sampling-grid resolution is stable across a wide range of angular resolutions, including grids 10 times smaller than those used in training, and temperature exhibits the expected tradeoff: lower temperatures improve validity but reduce uniqueness (Daigavane et al., 2023).

The main reported limitation is radial discretization. Coarse radial shells produce wider bond-length distributions than EDM or G-SchNet, which hurts validity when bonds are assigned by distance-based lookup. A future extension proposed in the paper is to use normalizing flows to model continuous radial densities without discretization. A second limitation is computational cost: higher-degree tensor products increase training and inference cost relative to scalar- or vector-only architectures.

Within the broader landscape of 3D molecule generation, Symphony is positioned between rotationally invariant autoregressive methods and equivariant diffusion approaches. Rotationally invariant autoregressive models require triangulation and auxiliary tokens, whereas equivariant diffusion models such as EDM denoise entire point clouds, achieve strong validity and distributional metrics, but require expensive training, long sampling chains of approximately 1000 network evaluations per molecule, and fully connected interactions that may hinder scalability. Symphony demonstrates that higher-degree $Z_{n+1}$5-equivariant features and spherical harmonic projections can directly parameterize probability distributions of positions, improving autoregressive generation quality while preserving practical advantages for sequential fragment completion. A plausible implication is that this representation opens a path toward larger motif placement, hybrid flow or diffusion extensions, and applications to larger molecules, proteins, or materials, all of which are explicitly identified as future directions in the paper.