---
title: Curriculum-Augmented GFlowNets for mRNA Design
url: https://www.emergentmind.com/topics/curriculum-augmented-gflownets-cagfn
type: topic
---

# Curriculum-Augmented GFlowNets for mRNA Design

Searching arXiv for the target paper and closely related GFlowNet references.
Curriculum-Augmented GFlowNets (CAGFN) are a variant of Generative Flow Networks introduced for de novo multi-objective mRNA sequence generation. In this formulation, mRNA design is cast as sequential construction of codon sequences on a directed acyclic graph, with terminal sequences sampled proportionally to a scalarized biological reward. CAGFN combine a preference-conditional GFlowNet with a length-based curriculum that progressively adapts training across protein-length regimes, with the stated aim of mitigating sparse, long-horizon rewards and multi-objective trade-offs in therapeutic sequence design. The framework was introduced in “Curriculum-Augmented GFlowNets For mRNA Sequence Generation” [2510.03811].

## 1. Problem formulation and scope

CAGFN are defined for the setting in which a target protein sequence is given and the goal is to generate plausible synonymous mRNA sequences that optimize multiple biological objectives simultaneously. The underlying challenge is described as the exploration of a vast space of possible nucleotide combinations while balancing sequence properties such as stability, translation efficiency, and protein expression [2510.03811].

The method frames mRNA sequence design as a stochastic policy over a directed acyclic graph of partial codon sequences. Each state \(s \in \mathcal{S}\) is a prefix of codons \((c_1,\dots,c_t)\), \(0 \le t \le L\), or the special sink state \(s_f\). The initial state \(s_0\) is the empty prefix. If \(s=(c_1,\dots,c_t)\) with \(t<L\), then the action space is restricted to synonymous codons encoding the \((t+1)\)th amino acid of the target protein; if \(t=L\), the only forward action is \(\texttt{exit}\), which transitions to \(s_f\). Backward actions remove the last codon [2510.03811].

A full episode is a trajectory
\[
\tau=(\,s_0 \xrightarrow{a_0} s_1 \xrightarrow{a_1}\dots \xrightarrow{a_{L-1}} s_L \xrightarrow{a_{\mathrm{exit}}} s_f)\,.
\]
The flow-based formulation posits a nonnegative flow function \(F\) on edges. For every non-terminal state \(s\), incoming and outgoing flows satisfy conservation,
\[
\sum_{s^-: s^-\to s} F(s^-\!\to s)
\;=\;
\sum_{s^+: s\to s^+} F(s\!\to s^+)\,,
\]
and for each terminating state \(x\),
\[
F(x\!\to s_f)=R(x)\,.
\]
The associated forward and backward policies are
\[
P_F(s'\mid s)=\frac{F(s\!\to s')}{\sum_{u}F(s\to u)},\quad
P_B(s\mid s')=\frac{F(s\!\to s')}{\sum_{v}F(v\!\to s')}\,.
\]

This construction makes CAGFN a sequence-generation method in which sampling probabilities are aligned with terminal rewards rather than solely with one-step likelihood objectives. A plausible implication is that the framework is intended to search broadly over synonymous codon realizations while preserving controllable pressure toward biologically desirable sequence-level outcomes.

## 2. GFlowNet objective and sequence-generation mechanics

The training objective used in CAGFN is the Sub-trajectory Balance (SubTB) objective, chosen to stabilize learning on long sequences [2510.03811]. For a partial trajectory \(\tau_{0:k}=(s_0,\dots,s_k)\), the loss is
\[
\mathcal{L}_{\rm SubTB}(\tau_{0:k})
=
\Bigl(\,\log\frac{F(s_0)\,\prod_{t=0}^{k-1}P_F(s_{t+1}\!\mid s_t)}
                 {F(s_k)\,\prod_{t=0}^{k-1}P_B(s_t\!\mid s_{t+1})}\Bigr)^{2}.
\]
For full trajectories terminating at \(x \to s_f\), this reduces to the Trajectory Balance loss with learned normalizer \(Z=F(s_0)\):
\[
\mathcal{L}_{\rm TB}(\tau)
=
\Bigl[\log\frac{Z\prod_{t=0}^{n-1}P_F(s_{t+1}\mid s_t)}
               {R(x)\,\prod_{t=0}^{n-1}P_B(s_t\mid s_{t+1})}\Bigr]^2.
\]

Within the proposed environment, non-terminal rewards are zero and only terminal states receive reward. The design environment, named **CodonDesignEnv**, represents states as length-\(L\) integer vectors with \(-1\) for unfilled codons. Dynamic action masking ensures that, at each position, only synonymous codons for the relevant amino acid are valid, plus a final “exit” action at \(t=L\) [2510.03811].

This structure is central to the method’s use in codon design. Since forward actions are constrained to synonymous codons, generated sequences preserve the target amino-acid sequence while varying the nucleotide realization. This suggests that the optimization target is not protein identity itself, but the biological quality of synonymous mRNA realizations under a specified objective mixture.

## 3. Length-based curriculum and adaptive task selection

The distinctive feature of CAGFN is the incorporation of curriculum learning into multi-objective GFlowNet training. Because mRNA sequences may be very long, the method introduces an adaptive, length-aware curriculum following the Teacher–Student Curriculum Learning framework [2510.03811].

Protein lengths are partitioned into \(K=5\) intervals:
\[
\{[25,40],\,[45,60],\,[65,80],\,[85,120],\,[125,180]\}\,.
\]
Each task corresponds to training on proteins whose length lies in one of these intervals. At each evaluation step \(t\), the mean reward \(m_{i,t}\) is recorded for each task \(i\), and learning progress is tracked by an exponential moving average:
\[
\Delta m_{i,t}=m_{i,t}-m_{i,t-1},\quad
\mathcal{LP}_{i,t}
=(1-\beta)\,\mathcal{LP}_{i,t-1}+\beta\,\Delta m_{i,t},
\]
with \(\beta \approx 0.05\).

Task sampling is then determined by
\[
P(i\mid t)=\frac{\max(0,\mathcal{LP}_{i,t})+\varepsilon}
{\sum_{j=1}^K\bigl[\max(0,\mathcal{LP}_{j,t})+\varepsilon\bigr]},
\]
with \(\varepsilon=0.15\) to ensure exploration. At each training step, the Teacher samples a task index \(k \sim P(\cdot)\), draws a protein of the corresponding length, instantiates the environment, and runs \(I_{\rm task}=200\) GFlowNet updates before re-sampling. Every \(I_{\rm eval}=5\) outer steps, tasks are re-evaluated to update \(\mathcal{LP}\) [2510.03811].

The stated rationale is progressive adaptation from easier to harder subproblems. In operational terms, the curriculum does not merely sort tasks by length once; it reweights them according to measured learning progress. A plausible implication is that CAGFN treat protein length as the principal source of horizon difficulty, and use task scheduling to concentrate optimization where marginal learning remains positive.

## 4. Multi-objective reward design

CAGFN optimize three biological objectives simultaneously and scalarize them via a weighted sum. For a complete mRNA sequence \(x\), the objective vector is
\[
\phi(x)=\bigl(\,\text{CAI}(x),\,-\text{MFE}(x),\,\text{GC\%}(x)\bigr)\in\mathbb{R}^3,
\]
where CAI is the Codon Adaptation Index, MFE is the Minimum Free Energy of secondary structure, and GC\% is the fraction of G or C nucleotides [2510.03811].

Given a weight vector \(w \in \Delta^2\), sampled from \(\mathrm{Dirichlet}(1,1,1)\) in conditional training, the scalar reward is
\[
R(x\mid w)=w^\top \phi(x)\,.
\]
By conditioning on \(w\), one GFlowNet is trained to learn the entire Pareto front [2510.03811].

This reward specification makes CAGFN explicitly preference-conditional. Rather than train separate models for different objective trade-offs, a single policy is conditioned on sampled weights over the three objectives. In the terminology of the source, the method is therefore both multi-objective and curriculum-augmented. This suggests that diversity over biological trade-offs is intended to be handled by conditional reward scalarization, whereas diversity over sequence space is handled by the GFlowNet sampling mechanism.

## 5. Algorithmic implementation and model architecture

The high-level CAGFN procedure initializes policy \(\pi_\theta\), uniform task sampling \(P(i)=1/K\), and \(\mathcal{LP}_i=0\). For each outer iteration, a task and corresponding protein sequence are sampled, the environment \(\mathcal{E}=\text{CodonDesignEnv}(p)\) is instantiated, and \(I_{\rm task}\) inner updates are executed. During each inner update, a preference vector \(w \sim \mathrm{Dirichlet}(1,1,1)\) is sampled, a batch of \(B=64\) trajectories is generated from \(\pi_\theta(\cdot\mid \mathcal{E},w)\), the SubTB loss is computed, and parameters are updated with Adam at learning rate \(5\times10^{-3}\). Every \(I_{\rm eval}=5\) outer steps, tasks are evaluated to update learning progress and recompute the task distribution [2510.03811].

The policy network is identified as the “PF module.” It takes as input a sequence of codon indices of length up to 180, embedded to dimension 32. The core model is a 4-layer Transformer with hidden size 256 and 8 attention heads. At each step \(t\), a linear head produces logits over 65 actions, which are masked to valid codons plus exit and normalized by softmax to yield \(P_F(\cdot\mid s,w)\) [2510.03811].

The source also reports the following implementation hyperparameters and resources.

| Component | Specification |
|---|---|
| Task intervals | [25,40], [45,60], [65,80], [85,120], [125,180] AA |
| Outer iterations | \(I_{\rm total}=100\) |
| Inner updates per task | \(I_{\rm task}=200\) |
| Evaluation frequency | Every 5 steps |
| Optimizer | Adam |
| Learning rate for \(\theta\) | \(5\times10^{-3}\) |
| Learning rate for \(\log Z\) | \(1\times10^{-1}\) |
| Batch size | \(B=64\) |
| Sampled trajectories per evaluation | 100 |
| Dirichlet prior | \(\alpha=(1,1,1)\) |
| Teacher EMA smoothing | \(\beta=0.05\) |
| Teacher exploration floor | \(\varepsilon=0.15\) |
| Action masking exploration floor | \(\varepsilon=0.25\) |
| Action-to-distribution | “GreedyProp” (ε-greedy proportional), a2d_ε=0.15 |
| Attention-computation | “LP” or Mastering-Rate variants |
| Library and hardware | torchgfn; standard GPU hardware such as 2 × NVIDIA A100 |

All experiments are reported as having been run with the torchgfn library on standard GPU hardware, with end-to-end runs completing in under 24 h [2510.03811].

## 6. Experimental evaluation and reported findings

The reported dataset is based on CodonTransformer data and includes proteins of lengths 25–180 amino acids, split into the five curriculum intervals. Benchmarks are reported for small proteins (25–60 AA) and medium proteins (85–120 AA). Baselines include SGFN (short-only), LGFN (long-only), ROFN (random order), reinforcement-learning methods MOReinforce and PPO, and a multi-objective GFlowNet without curriculum [2510.03811].

Evaluation uses Top-\(K\) Reward and Top-\(K\) Diversity with \(K=50\), Pareto performance defined as the fraction of generated sequences covering the Pareto front, uniqueness as the percentage of unique sequences, and training time and loss dynamics [2510.03811].

The principal quantitative findings are summarized below.

| Setting | Reported outcome |
|---|---|
| Small proteins | CAGFN achieves Top-K reward \(\sim 0.61\) vs 0.58 for SGFN, with 100% diversity and Pareto coverage up to 0.22 vs 0.19 |
| Medium proteins | Pareto performance up to 0.21 for CAGFN vs 0.17 for LGFN and ROFN |
| RL comparison | GFlowNets with or without curriculum maintain 100% uniqueness vs up to 88% duplicates in PPO |
| Speed | CAGFN trains \(\sim 4\times\) faster than LGFN and \(\sim 2.4\times\) faster than ROFN |
| OOD generalization | CAGFN generalizes effectively to unseen proteins across all length ranges while maintaining Pareto and diversity metrics |

The source further states that CAGFN improve Pareto performance and biological plausibility while maintaining diversity, and that they reach higher-quality solutions faster than a GFlowNet trained with random sequence sampling and enable generalization to out-of-distribution sequences [2510.03811].

These results position the curriculum mechanism as the main differentiator from non-curriculum GFlowNet training. The reported smoother loss curves and stable credit assignment suggest that the curriculum is presented not only as a data-ordering heuristic but as a mechanism for stabilizing optimization under long horizons.

## 7. Interpretation, relation to nearby methods, and limitations of scope

Within the reported comparison set, CAGFN are distinct from SGFN and LGFN because those baselines are specialist models tied to restricted length regimes, and from ROFN because that baseline uses random order rather than adaptive curriculum. They are also distinguished from PPO and MOReinforce by the use of GFlowNet objectives and by the emphasis on sequence diversity and Pareto coverage rather than policy optimization alone [2510.03811].

A recurrent misconception in sequence design is that improved scalar reward necessarily implies broad coverage of competing design trade-offs. The CAGFN formulation explicitly separates these concerns: scalarized rewards are conditioned on sampled preference vectors \(w\), while evaluation includes Pareto performance and diversity. This suggests that the method is intended to produce not a single optimum, but a set of plausible trade-off solutions across the objective simplex.

Another possible misconception is that the curriculum simply truncates sequence length and therefore changes the target problem. The description instead presents a task-selection mechanism over protein-length intervals, with dynamic reallocation based on learning progress. The objective remains generation of complete mRNA sequences for a specified target protein. The curriculum changes the ordering and frequency of training exposures rather than redefining the terminal design task [2510.03811].

The scope of the claims is limited to the stated environment and objectives: CAI, \(-\)MFE, and GC\%. No broader therapeutic efficacy claims are made in the source. Likewise, “biological plausibility” is reported as an empirical outcome, but the detailed operationalization of that phrase is not expanded beyond the objective design and benchmark metrics in the provided material. A plausible implication is that the paper treats these three objectives as a biologically motivated proxy rather than an exhaustive biological model.

In summary, CAGFN denote a curriculum-augmented, preference-conditional GFlowNet framework for synonymous mRNA sequence generation in which protein-length scheduling is used to stabilize training over long horizons, and multi-objective scalarization is used to learn diverse Pareto trade-offs within a single policy [2510.03811].

Source: https://www.emergentmind.com/topics/curriculum-augmented-gflownets-cagfn