HyperRNA: Geometric Hypergraph RNA Design

• HyperRNA is a generative framework that integrates geometric deep learning and hypergraph convolution to solve the RNA inverse folding problem.
• It employs a three-bead coarse-grained representation and attention embedding to capture higher-order spatial and chemical dependencies.
• Its autoregressive decoder refines nucleotide sequences through iterative flow-matching, achieving improved RNA recovery, structural fidelity, and diversity.

The HyperRNA model is a generative framework leveraging hypergraphs and geometric deep learning for the RNA inverse folding problem, where the objective is to generate nucleotide sequences that adopt predefined secondary and tertiary structures. HyperRNA integrates geometric representations, attention-based feature embedding, hypergraph convolution, and autoregressive decoding, enabling the modeling of higher-order dependencies essential for accurate RNA design, particularly in protein–RNA complexes. Empirical results demonstrate that HyperRNA achieves improved accuracy and diversity relative to prior RNA design models while maintaining structural fidelity (Yang et al., 3 Dec 2025).

1. Three-Bead Coarse-Grained Representation and Graph Construction

HyperRNA's data preprocessing module applies a 3-bead coarse-grained (CG) molecular representation to each residue, encoding essential geometric and chemical information for deep learning:

• Protein backbones are mapped using tuples

$$\mathcal{P}_i = [\mathbf{p}_i^N,\;\mathbf{p}_i^{C_\alpha},\;\mathbf{p}_i^C]\in\mathbb{R}^{3\times 3}$$

• RNA backbones represent each nucleotide $j$ by

$$\mathcal{R}_j = [\mathbf{r}_j^P,\;\mathbf{r}_j^{C'_4},\;\mathbf{r}_j^{N_{1/9}}]\in\mathbb{R}^{3\times 3}$$

• All $n$ backbone atoms (protein and RNA) are treated as nodes $\{v_1,...,v_n\}$.

Spatial adjacency is defined by $k$-nearest neighbor (kNN) relationships in 3D Euclidean space, forming an adjacency matrix $A_{ij}$. Each node receives:

• Scalar features $\mathbf{s}_i$ (distances, angles, torsions, radial basis descriptors)
• Vector features $\mathbf{v}_i'\in\mathbb{R}^{d_v\times3}$ (bond orientation unit vectors)

These define the initial point cloud and geometric graph for further processing.

A hypergraph $\mathcal G = (\mathcal V, \mathcal E, \mathbf W)$ is constructed:

• $\mathcal V$: nodes as above
• $\mathcal E$: higher-order hyperedges capturing multi-residue or multi-atom interactions
• $\mathbf W$: edge weights
• Incidence matrix $H\in\{0,1\}^{n\times m}$ with vertex and edge degrees $d(v_i)$ and $\delta(e_\alpha)$, used in subsequent convolutions

The geometric CG mapping reflects techniques introduced in previous molecular modeling works, notably the 3-bead-per-nucleotide models that allow for scalable, time-efficient molecular dynamics simulations while preserving structural accuracy (Paliy et al., 2010).

2. Encoder: Attention Embedding and Hypergraph Convolution

2.1 Attention Embedding

• Vector features are flattened and processed with multi-head scaled dot-product attention, using head-specific projections:

$$V^i = \mathrm{softmax}\left(\frac{(\mathbf{v}'W_q^i)(\mathbf{v}'W_k^i)^\top}{\sqrt{3d_v/h}}\right)\mathbf{v}'W_v^i$$

• Outputs from all heads are concatenated and reshaped, yielding refined vector embeddings $\mathbf{v}'_a$.
• Scalar features are partitioned into five semantic groups, weighted and pooled via learnable attention:

$$\gamma_i = \frac{\exp(\mathbf w_p^i\cdot\mathbf s_i)}{\sum_{j=1}^5\exp(\mathbf w_p^j\cdot\mathbf s_j)}\,,\quad \mathbf s_a = \sum_{i=1}^5\gamma_i\mathbf s_i$$

The combination $(A,\mathbf{s}_a,\mathbf{v}'_a)$ defines the attention-augmented input for the convolution module.

2.2 Hypergraph Convolution

The core of HyperRNA's geometric deep learning is its $L$-layer hypergraph convolution, enabling higher-order information propagation:

$$\left(\mathbf s_a^{(i+1)},\,\mathbf v_a'^{(i+1)}\right) = \sigma\left(D_v^{-1/2} H W D_e^{-1} H^\top D_v^{-1/2} [\mathbf s_a^{(i)},\,\mathbf v_a'^{(i)}]\;\Theta^{(i)}\right)$$

where $\Theta^{(i)}$ are layer parameters and $\sigma$ is an elementwise nonlinearity.

After $L$ steps and normalization, node-level encodings $(\mathbf s_e,\mathbf v'_e)$ are summarized (pooling/read-out) to yield global representations:

• $\mathbf s_p\in\mathbb{R}^{n\times d_e}$ (scalar, structural)
• $\mathbf v'_p\in\mathbb{R}^{n\times d_v\times3}$ (vector, orientation)

This design permits explicit modeling of multifaceted spatial and chemical dependencies, moving beyond standard pairwise edge constraints.

3. Autoregressive Decoder for Sequence Generation

The decoder reconstructs the nucleotide sequence, integrating geometric and context signals:

• Stack $L_d$ GVP (Geometric Vector Perceptron) layers, each consuming $(\mathbf{s}_p,\mathbf{v}'_p)$ and a one-hot encoding of previously decoded nucleotides.
• At each step $t$, GVPs produce hidden states $\mathbf h_t$ yielding unnormalized logits over nucleotides $(A, C, G, U)$:

$$p(n_t| n_{<t}, \mathbf s_p, \mathbf v'_p) = \mathrm{softmax}(\mathbf h_t/\tau)$$

with $\tau$ as the softmax temperature.

• Teacher-forcing is used during training; at inference, either sampling or greedy decoding via $\arg\max$ is employed.
• Trajectory-to-Seq flow-matching refinement: During inference, backbone conformation is iteratively refined and $p(n_t|\cdot)$ updated, improving physical plausibility.

This decoder architecture ensures that the generated RNA sequence is both compositionally valid and structurally compatible with the targeted fold.

4. Training Objectives, Optimization, and Hyperparameters

Training is end-to-end and uses a composite objective:

$$\mathcal L_{\mathrm{total}} = \mathcal L_{\mathrm{seq}} + \mathcal L_{\mathrm{str}}$$

• $\mathcal L_{\mathrm{seq}}$: categorical cross-entropy over the sequence labels.
• $\mathcal L_{\mathrm{str}}$: mean-squared error (MSE) between actual and predicted backbone coordinates after structure prediction via RF2NA.

Adam is used for optimization with:

• Initial learning rate $1\times10^{-4}$
• Weight decay $0$
• Dropout $0.1$
• No other regularization

Training schedules:

• PDBBind: 100 epochs
• RNAsolo: 50 epochs (pretraining or fine-tuning)

This objective enforces both sequence fidelity and geometric realization, aligning with metrics directly relevant to downstream structural biology and molecular engineering.

5. Quantitative Evaluation and Comparative Performance

HyperRNA is evaluated on:

• PDBBind: protein-binding RNA inverse folding
• RNAsolo: unconditional backbone and sequence generation

Key metrics:

• RMSD: root-mean-square deviation (Å) of backbone atom positions
• RNA recovery: proportion of positions with correct nucleotide identity
• lDDT: local distance difference test on $C'_4$ atoms
• Validity: for RNAsolo, proportion of sequences with backbone scTM-score $\geq 0.45$ (via RhoFold)
• Diversity: fraction of unique backbone clusters (qTMclust, TM $\geq 0.45$)
• Novelty: average TM-score difference to training set (US-align, $C'_4$ atoms)

Model	RMSD (PDBBind)	Recovery ↑	lDDT ↑	Validity (RNAsolo)	Diversity	Novelty ↓
gRNAde	13.51 ± 1.26	0.28 ± 0.08	0.51	0.27 ± 0.011	0.43 ± 0.005	0.57 ± 0.009
gRNAde+Hypergraph	12.46 ± 0.75	0.28 ± 0.03	0.54	0.24 ± 0.017	0.46 ± 0.009	0.53 ± 0.009
HyperRNA	12.56 ± 0.99	0.29 ± 0.03	0.56	0.24 ± 0.012	0.47 ± 0.008	0.53 ± 0.007

HyperRNA demonstrates consistent improvements in RNA recovery and lDDT, competitive or reduced RMSD, and achieves highest diversity and matching lowest novelty for unconditional RNA generation (Yang et al., 3 Dec 2025).

6. Significance and Relation to Broader Research

HyperRNA extends prior coarse-grained molecular modeling (Paliy et al., 2010) by embedding geometric priors in a generative deep learning framework, leveraging hypergraph architectures for biomolecular sequence design. Its encoder-decoder structure interpolates between traditional physical/chemical modeling and end-to-end sequence optimization.

This approach distinguishes itself from related hypergraph/graph neural models for RNA analysis (e.g., ncRNA classification in (An et al., 24 Sep 2025)) by focusing on generative tasks, higher-order spatial encoding, and explicit coordinate+sequence supervision. The integration of attention, geometric vector representations, and hypergraph convolution enables the capture of complex RNA–protein and intra-molecular interactions.

A plausible implication is that this methodological template could be extended to protein sequence design, hybrid biomolecular systems, or other settings where structural constraints and high-order dependencies require explicit modeling beyond pairwise interactions.

---

References:

• "Harnessing Hypergraphs in Geometric Deep Learning for 3D RNA Inverse Folding" (Yang et al., 3 Dec 2025)
• "Coarse Graining RNA Nanostructures for Molecular Dynamics Simulations" (Paliy et al., 2010)
• "A HyperGraphMamba-Based Multichannel Adaptive Model for ncRNA Classification" (An et al., 24 Sep 2025)