Latent Grammar Flow: A Neuro-Symbolic Framework

• LGF is a neuro-symbolic framework that embeds grammar constraints into a discrete latent space to ensure syntactically valid and behaviorally meaningful equation discovery.
• It employs a grammar quantisation autoencoder and discrete flow model to guide candidate generation, optimizing both symbolic structure and continuous parameters.
• LGF demonstrates competitive performance in ODE discovery, robot design automation, and grammar induction by combining domain-specific constraints with latent-space organization.

Latent Grammar Flow (LGF) denotes a class of grammar-constrained latent-variable methods in which symbolic structures are embedded into a latent space and then manipulated by a learned generative or optimization process. In the most explicit usage of the term, LGF is a neuro-symbolic generative framework for discovering ordinary differential equations (ODEs) from data; it combines a context-free grammar for equation syntax, a discrete latent embedding learned by a grammar quantisation autoencoder, a behavioural loss that organizes latent codes by dynamical similarity, and a discrete flow model that guides sampling of candidate equations (Yu et al., 17 Apr 2026). Closely related uses of the same underlying idea appear in robot design automation, where grammar-generated robot graphs are embedded into a latent space for Bayesian optimization (Hu et al., 2022), and in grammar induction, where a continuous latent variable modulates sentence-specific grammar rule probabilities in a compound probabilistic context-free grammar (Kim et al., 2019). This suggests that “latent grammar flow” is best understood not as a single architecture, but as a broader design pattern in which a grammar defines valid symbolic objects while a latent mechanism reshapes search, inference, or generation over that grammar-constrained space.

1. Terminology and conceptual scope

In its direct formulation, LGF is introduced for ODE discovery as a framework that combines “a grammar-based symbolic representation that guarantees syntactic validity of equations” with “a discrete latent flow model that can efficiently sample and guide candidate equations toward good fits and physically meaningful behavior” (Yu et al., 17 Apr 2026). The grammar specifies admissible symbolic forms, while the latent model supports guided generation.

Related work uses the same conceptual ingredients without always using the exact label. In GLSO, robot designs are generated by graph grammar rules, embedded by a graph variational autoencoder into a low-dimensional latent space, and then optimized with Bayesian Optimization rather than by direct combinatorial search (Hu et al., 2022). In compound PCFGs, a sentence-level continuous latent variable $z$ modulates grammar rule probabilities, so that each sentence receives its own grammar instance while exact dynamic programming remains possible once $z$ is fixed (Kim et al., 2019). These formulations differ in domain and inference machinery, but they share the same organizing principle: grammar provides structural validity, and a latent representation or latent-conditioned transformation changes how the structured space is explored.

A common source of confusion is acronym overlap. “GF + MMT = GLF -- From Language to Semantics through LF” introduces GLF, the Grammatical Logical Framework, which combines GF and MMT for symbolic natural-language understanding, not Latent Grammar Flow (Kohlhase et al., 2019). Another possible confusion concerns the word “flow.” In the ODE setting, LGF uses a discrete flow model in latent space (Yu et al., 17 Apr 2026). By contrast, the robot-design paper explicitly states that its “latent grammar / latent flow” interpretation is not a flow model in the normal normalizing-flow sense (Hu et al., 2022).

2. Grammar-constrained symbolic representation

LGF represents equations with a context-free grammar $$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$, where terminals, nonterminals, production rules, and the start symbol define the admissible language of expressions (Yu et al., 17 Apr 2026). A production rule is written as

$\upalpha\rightarrow\upbeta$

with $\upalpha\in\mathcal{N}$ and $\upbeta\in(\mathcal{N}\cup\mathcal{T})^*$, and recursive application from $\mathcal{S}$ yields terminal strings

$$\mathcal{L}(G) = \{w\in\mathcal{T}^*| \mathcal{S}\rightarrow^*w\}.$$

The grammar-based view is used because it enforces syntax, naturally supports nested expressions, and avoids learning the syntax from scratch (Yu et al., 17 Apr 2026).

A notable design choice is that scalar constants are not grammar-expanded as long rule sequences. Instead, LGF uses a placeholder symbol $C$, and the actual scalar values are optimized later (Yu et al., 17 Apr 2026). This separates symbolic structure from continuous parameter fitting and yields a two-level representation: equation skeleton first, constants second.

The same grammar-first principle appears in adjacent work. GLSO generates robot morphologies by starting from a symbol $S$ and repeatedly applying grammar rules until only terminal symbols remain, producing valid robot graphs by construction (Hu et al., 2022). Compound PCFGs likewise preserve the tree-structured generative process of a PCFG and then modulate rule probabilities through a latent variable (Kim et al., 2019). A plausible implication is that LGF-style methods rely on grammar not merely as a parser or specification device, but as the mechanism that restricts the hypothesis class to structurally valid objects before any latent-space shaping occurs.

3. Latent embedding and behaviour-aware organization

The embedding component of LGF is the grammar quantisation autoencoder (GQAE), which maps a grammar-derived rule sequence into a discrete latent space using finite scalar quantisation (Yu et al., 17 Apr 2026). An expression is parsed into a sequence of rules, represented as a one-hot matrix

$z$0

with padding rules added when necessary. The encoder $z$1 produces a latent representation, the latent vector $z$2 is quantised to $z$3, and the decoder $z$4 reconstructs the rule matrix (Yu et al., 17 Apr 2026). The paper emphasizes that discrete quantisation reduces dimensionality and avoids some issues associated with continuous latent spaces, including poorly defined latent neighborhoods and excessive rejection sampling.

LGF does not restrict the latent space to structural similarity. It introduces a behavioural latent distance derived from sampled dynamics, motivated by the observation that structurally similar equations can behave very differently (Yu et al., 17 Apr 2026). For ODEs up to second order, the sampled variables are $z$5, $z$6, and $z$7, using a grid with

$z$8

points per sample. Each ODE structure is evaluated 25 times with random scalars in $z$9, and mean and standard deviation statistics are computed and clipped for stability (Yu et al., 17 Apr 2026). The resulting Wasserstein-based distance is incorporated through

$$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$0

and the total autoencoding loss is

$$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$1

This behaviour-aware shaping has close analogues elsewhere. GLSO augments graph reconstruction with a property prediction network trained on world-space features, specifically flat-ground contact pattern or contact locations represented as $$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$2, to make the latent space smoother and more useful for downstream optimization (Hu et al., 2022). Compound PCFGs use a sentence-level latent code to induce marginal dependencies beyond standard context-free assumptions, thereby coordinating global grammatical choices through $$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$3 (Kim et al., 2019). This suggests a general LGF principle: latent geometry is made meaningful not only by reconstruction of symbolic structure, but also by supervision tied to semantics, behaviour, or global coordination.

4. Flow-guided generation and inference

After learning a discrete latent representation, LGF uses a discrete flow model to sample candidate equations in latent space (Yu et al., 17 Apr 2026). Because the latent space is discrete, the method uses discrete flow matching based on continuous-time Markov chains and a rate matrix $$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$4, with a denoising predictor

$$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$5

Guidance is incorporated through conditional predictors, yielding a guided conditional rate matrix

$$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$6

approximated with learned predictors, and extended to multiple predictors as

$$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$7

The discrete flow model is trained from $$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$8 to $$G = \{\mathcal{T},\mathcal{N},\mathcal{P},\mathcal{S}\}$$9, then decoded into equation skeletons (Yu et al., 17 Apr 2026).

Inference proceeds recursively: start from a simple or noisy discrete state, use the flow model to move toward the target latent distribution, decode latent states into rule sequences, parse them into candidate equations, optimize constants, and rank the resulting expressions (Yu et al., 17 Apr 2026). The method is therefore characterized as a guided generator rather than a pure search method.

The meaning of “flow” varies across related papers. In the ODE-discovery LGF, the flow is an explicit discrete generative mechanism over latent codes (Yu et al., 17 Apr 2026). In compound PCFGs, the continuous latent variable $\upalpha\rightarrow\upbeta$0 can be viewed as a sentence-level control vector that modulates all rule probabilities, and the paper explicitly relates this to latent-flow-style grammar induction (Kim et al., 2019). In GLSO, by contrast, the latent-space process is optimization-driven rather than flow-based: graph-structured robot designs are encoded into a continuous latent manifold, and continuous optimizers such as Bayesian Optimization operate there (Hu et al., 2022). A separate latent-space flow literature places a normalizing flow prior in the latent space of a top-down generator and uses short-run Langevin dynamics as approximate posterior inference, but this is not grammar-based and serves a different purpose (Xie et al., 2023).

5. Discovery loop, constraints, and objective functions

LGF uses a two-stage nested discovery process consisting of an outer loop that generates equation skeletons and an inner loop that optimizes scalar constants (Yu et al., 17 Apr 2026). Once the GQAE, denoiser, and static predictors are trained, the framework samples skeletons and can iteratively refine them by sampling an initial population, evaluating candidates, training dynamic predictors, sampling new populations, and repeating until convergence (Yu et al., 17 Apr 2026). Convergence is defined using both the number of newly discovered unique expressions and stagnation of the top-$\upalpha\rightarrow\upbeta$1 objective.

For each skeleton, constants are optimized with Nelder-Mead in two stages: a coarse stage with $\upalpha\rightarrow\upbeta$2, $\upalpha\rightarrow\upbeta$3, and a maximum of 50 iterations, followed, when $\upalpha\rightarrow\upbeta$4, by a refinement stage with $\upalpha\rightarrow\upbeta$5 and $\upalpha\rightarrow\upbeta$6 (Yu et al., 17 Apr 2026). Accuracy is measured either by the ODE residual $\upalpha\rightarrow\upbeta$7 or by the solution loss $\upalpha\rightarrow\upbeta$8, with the framework using $\upalpha\rightarrow\upbeta$9 first and switching to $\upalpha\in\mathcal{N}$0 when $\upalpha\in\mathcal{N}$1 (Yu et al., 17 Apr 2026). To favor compact interpretable expressions, candidates are ranked with the complexity-penalized objective

$\upalpha\in\mathcal{N}$2

Domain knowledge enters in two forms. First, the grammar itself encodes admissible syntax, including balanced parentheses, valid compositions of terms, explicit or implicit equation forms, and system-specific allowed operators (Yu et al., 17 Apr 2026). Second, conditional predictors guide the flow toward equations satisfying known properties, including system order, stability, and the optimization objective (Yu et al., 17 Apr 2026). Stability is estimated by Lyapunov’s indirect method via linearization, conversion to first-order state-space form, and eigenvalues of the Jacobian at equilibrium points; for non-autonomous systems, boundedness and decay tendencies are checked numerically (Yu et al., 17 Apr 2026). The paper treats stability as guidance rather than a post hoc filter.

The broader significance of this design is that grammar restrictions and latent guidance are complementary rather than interchangeable. GLSO reports an analogous division of labor: grammar-guided data generation biases the decoder toward valid robot structures, while property supervision makes nearby latent points correspond more closely to similar physical capabilities (Hu et al., 2022).

6. Empirical performance, related formulations, and limitations

LGF is evaluated on three noisy ODE benchmarks using 5% Gaussian noise, with relative $\upalpha\in\mathcal{N}$3 error as the main metric (Yu et al., 17 Apr 2026). On a benchmark of 30 explicit one-dimensional first-order ODEs, PySR is reported as best in accuracy overall, while LGF performs similarly to ProGED and better than ODEFormer and GODE; LGF’s complexity is close to ground truth and comparable to grammar-based methods (Yu et al., 17 Apr 2026). On a benchmark of 10 ODEs, comprising 5 linear and 5 nonlinear cases with known order and stability, LGF beats all other methods in mean relative $\upalpha\in\mathcal{N}$4 error for $\upalpha\in\mathcal{N}$5 and $\upalpha\in\mathcal{N}$6, performs well on $\upalpha\in\mathcal{N}$7, and uses only half the population size of GODE (Yu et al., 17 Apr 2026). On seven partially observable second-order dynamical systems, including the pendulum, Duffing oscillator, Van der Pol oscillator, damped oscillator, exponential stiffness, and nonlinear damping cases, LGF outperforms PySR in most examples, especially when domain knowledge matters (Yu et al., 17 Apr 2026).

The paper’s comparisons isolate several claimed contributions. Relative to GODE, LGF uses discrete quantisation rather than a continuous latent space; relative to ProGED, it replaces Monte Carlo search with learned discrete flow guidance; relative to ODEFormer, it is described as more structured and cheaper while preserving interpretability; and relative to PySR, it emphasizes implicit equations, domain constraints, and trajectory-level validation when appropriate (Yu et al., 17 Apr 2026). A central analytical point is that structural similarity alone is insufficient and that behavioural shaping of the latent space organizes equations by what they do rather than only by how they look (Yu et al., 17 Apr 2026).

Related formulations exhibit the same general pattern in other domains. Compound PCFGs achieve the strongest results in their study, with PTB test $\upalpha\in\mathcal{N}$8 of 55.2 and CTB test $\upalpha\in\mathcal{N}$9 of 36.0, by combining a structural prior, neural parameter sharing, sentence-level latent variation, and exact tree marginalization conditioned on $\upbeta\in(\mathcal{N}\cup\mathcal{T})^*$0 (Kim et al., 2019). GLSO reports that it consistently finds higher-reward robot designs within a budget of 500 objective evaluations, outperforming random search, Monte Carlo Tree Search, Graph Heuristic Search, and a genetic algorithm on four locomotion tasks, while reusing the same trained VAE across tasks without retraining (Hu et al., 2022). These neighboring results support the interpretation that LGF-style methods are valuable when the symbolic hypothesis space is combinatorial, evaluations are expensive, and validity constraints must be enforced before optimization or generation.

The limitations stated for the ODE-discovery LGF are specific. The experiments are validated on first- and second-order one-dimensional ODEs; stability assessment for non-autonomous systems is approximate; behavioural distance is based on sampled vector fields and is therefore an approximation of true dynamical similarity; and performance depends on choices such as grammar, latent dimension $\upbeta\in(\mathcal{N}\cup\mathcal{T})^*$1, number of finite scalar quantisation levels, and guidance predictors (Yu et al., 17 Apr 2026). Proposed extensions include higher-dimensional systems, systems of ODEs, additional dynamical properties such as autonomy, and broader classes of dynamical systems (Yu et al., 17 Apr 2026). A plausible implication is that the future scope of LGF will depend less on the abstract grammar-plus-latent principle, which already appears across multiple fields, than on how well each instantiation can encode domain constraints and define a task-relevant notion of semantic closeness.