Diagrams-to-Dynamics (D2D) Overview

• Diagrams-to-Dynamics (D2D) is a framework that converts static diagrams—such as sketches, causal loop diagrams, and formal graph representations—into explicit dynamical systems with preserved structure and semantics.
• It applies techniques like diffeomorphic morphing in robotic motion synthesis, linear ODE construction from causal diagrams, and dynamic state evolution in graph inference to yield robust, stable models.
• D2D methods balance expressive modeling and computational stability by enforcing mathematical constraints, leveraging uncertainty quantification, and employing categorical syntax–semantics mappings.

Searching arXiv for recent and relevant papers on “Diagrams-to-Dynamics (D2D)” and closely related uses of the term. Diagrams-to-Dynamics (D2D) denotes a family of formalisms that convert diagrammatic representations into explicit dynamical objects. Across the literature, the term refers to at least three distinct but structurally related programs: transforming a sketched geometric diagram into an orbitally asymptotically stable robot motion policy (Zhi et al., 2023), transforming a causal loop diagram into an exploratory system dynamics model for intervention analysis under uncertainty (Uleman et al., 30 Jul 2025), and, in a broader interpretive sense, transforming static diagrams into evolving internal graph states for relational inference (Kim et al., 2017). A more abstract categorical formulation generalizes the same idea by treating diagrams as syntax and dynamical systems as semantics, linked by functors and composed via structured cospans (Li et al., 22 Sep 2025). These usages differ in domain, mathematical substrate, and intended outputs, but they share a central commitment: a diagram is not treated merely as a visualization, but as an input to a principled dynamical construction.

1. Scope and meanings of the term

The most explicit robotics formulation defines the diagrams-to-dynamics problem as follows: from a single sketched 2D closed curve on the camera image, build an orbitally asymptotically stable 3D dynamical system whose limit cycle on the surface corresponds, via projection, to the sketch, and whose transversal dynamics drive the end-effector toward that surface and orbit (Zhi et al., 2023). In that setting, the diagram is an egocentric sketch over an RGB(-D) image, and the output is a continuous-time, time-invariant dynamical system on the end-effector position $x \in \mathbb{R}^3$,

$\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$

A second usage arises in system dynamics, where D2D is defined as a workflow and computational framework that turns a causal loop diagram into a family of system dynamics models, then uses those models to simulate interventions and explore leverage points under deep structural and parametric uncertainty (Uleman et al., 30 Jul 2025). Here the input diagram is a CLD with signed directed links, and the output is a coupled ODE plus algebraic system, reduced in the linear case to

$\frac{dx}{dt} = A x + b.$

A third, looser usage appears in diagram understanding. The Dynamic Graph Generation Network processes a static diagram image through a GRU-driven state evolution over a dynamic adjacency tensor memory, so that the inference procedure itself can be interpreted as a diagram-induced dynamical system over evolving graph states (Kim et al., 2017). This is not labeled D2D in the original title, but the interpretation is explicitly developed in the detailed exposition.

A broader mathematical unification is proposed in compositional system dynamics, where CLDs, SSDs, and SFDs are treated as categorical syntax, and dynamical systems—ODEs, stochastic transitions, and related semantics—are produced by syntax–semantics functors (Li et al., 22 Sep 2025). This suggests that “D2D” can be understood either narrowly, as a named method in robotics or CLD analysis, or more generally as a paradigm of executable semantics for diagrammatic objects.

2. Diagrammatic Teaching and robot motion synthesis

In the robotics formulation, Diagrammatic Teaching is an interface in which the robot provides an egocentric RGB(-D) image of the scene and the user draws a single closed 2D sketch directly on this image, representing the desired cyclic path of the end-effector on a surface (Zhi et al., 2023). The learned policy must satisfy three conditions: from arbitrary initial end-effector positions in 3D it must approach the surface, once on the surface it must converge to a stable periodic motion whose limit cycle matches the sketch after projection to 3D, and it must maintain that cycle robustly against perturbations.

The relevant stability notion is orbital asymptotic stability. The paper states that a system $\dot{x}=f(x)$ is orbitally asymptotically stable if there exists a periodic solution $\bar{x}(\tau)$ with period $T>0$ such that, for any initial condition $x_0$ in a basin of attraction $\mathcal{X}$,

$$\lim_{t \to \infty} \; \min_{\tau \in [0, T]} \|\; x(t) - \bar{x}(\tau) \;\| = 0.$$

This notion is appropriate for cyclic tasks because convergence is required to the orbit rather than to a particular phase on the orbit (Zhi et al., 2023).

The implemented framework is the Stable Diffeomorphic Diagrammatic Teaching (SDDT) framework. It proceeds in four steps. First, the user sketch $\{p_i\}_{i=1}^n$ is ray-traced into 3D points $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$0 on the physical surface using camera intrinsics, extrinsics, and surface depth. Second, a hand-designed 3D O.A.S. base system $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$1 is constructed, with a circular limit cycle on the plane $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$2, radial convergence in-plane, constant angular velocity, and exponential convergence in the orthogonal direction. Third, a parameterized diffeomorphism $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$3, implemented by an invertible neural network, morphs the circular in-plane limit cycle into a new closed curve while keeping the normal coordinate unchanged. Fourth, the transformed system

$\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$4

is trained by minimizing the Hausdorff distance between the morphed limit cycle and the projected sketch (Zhi et al., 2023).

The base system is defined in polar coordinates on the $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$5-plane plus the normal coordinate $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$6: $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$7 with $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$8 and $\dot{x}(t) = f(x(t)), \qquad x(0) = x_0.$9. In Cartesian coordinates this becomes

$\frac{dx}{dt} = A x + b.$0

with limit cycle

$\frac{dx}{dt} = A x + b.$1

The diffeomorphism acts only on the in-plane coordinates, via

$\frac{dx}{dt} = A x + b.$2

Because diffeomorphisms are topological conjugacies, the orbital asymptotic stability of the transformed system is preserved (Zhi et al., 2023).

Training uses the discrete Hausdorff distance between the projected sketch point set and a discretized representation of the transformed orbit: $\frac{dx}{dt} = A x + b.$3 augmented by a regularizer toward the identity,

$\frac{dx}{dt} = A x + b.$4

Optimization is performed with PyTorch and FrEIA using Adam, and no special differentiable ODE solver is needed for the limit cycle because the base orbit is closed-form (Zhi et al., 2023).

The framework also includes a shape expressivity proposition: any smooth, non-self-intersecting closed curve in $\frac{dx}{dt} = A x + b.$5 is diffeomorphic to the unit circle. Combined with the universality of coupling-based INNs as diffeomorphism approximators, this implies that any smooth, simple closed sketch can in principle be realized arbitrarily well by morphing the base circular orbit (Zhi et al., 2023).

3. From causal loop diagrams to exploratory system dynamics

In the system dynamics literature, D2D is motivated by the limitations of causal loop diagrams. CLDs encode variables, directed links, link polarity, and feedback loops, but they are static and qualitative: they do not specify time scales, functional forms, or magnitudes, and they do not directly support intervention simulation (Uleman et al., 30 Jul 2025). The method is specifically positioned against direct use of network centrality analysis on CLDs, which is criticized because it is purely structural, ignores essential causal semantics such as polarity and accumulation, and can lead to false inference.

The D2D workflow begins with a CLD and a minimal additional labeling protocol. Each variable is classified as a stock, flow/auxiliary, or constant, using timescale separation relative to a chosen base time unit and timeframe. Stocks accumulate or deplete on the base time scale or slower, auxiliaries adjust essentially instantaneously relative to the base time unit, and constants change only more slowly than the full timeframe or are exogenous (Uleman et al., 30 Jul 2025).

Given a set of variables $\frac{dx}{dt} = A x + b.$6, directed signed links $\frac{dx}{dt} = A x + b.$7, and a decomposition into stocks $\frac{dx}{dt} = A x + b.$8, auxiliaries $\frac{dx}{dt} = A x + b.$9, and constants $\dot{x}=f(x)$0, D2D constructs an SDM by assuming additive linear relationships for all links. For each stock $\dot{x}=f(x)$1,

$\dot{x}=f(x)$2

and for each auxiliary $\dot{x}=f(x)$3,

$\dot{x}=f(x)$4

After algebraic elimination of auxiliaries, the system is reduced to a linear ODE

$\dot{x}=f(x)$5

with analytical solution

$\dot{x}=f(x)$6

Link polarity determines the sign of the parameter range rather than a fixed coefficient. Positive links are sampled from positive uniform ranges, negative links from negative uniform ranges, and ambiguous links from symmetric ranges (Uleman et al., 30 Jul 2025).

Parameters are interpreted in standardized units. Variables are conceptualized as Z-scores, so a coefficient $\dot{x}=f(x)$7 in an auxiliary equation means that a 1 SD change in the source corresponds to 0.1 SD difference in the target, whereas in a stock equation it means a 0.1 SD change per base time unit. The method recommends choosing $\dot{x}=f(x)$8 by comparing expected total VOI change within the timeframe to the number of base time units. In the Alzheimer’s example, an expected decline of about 2 SD over 5 years with quarterly time steps yields $\dot{x}=f(x)$9, while $\bar{x}(\tau)$0 is chosen based on the largest empirically estimated auxiliary parameter (Uleman et al., 30 Jul 2025).

Interventions are implemented as one-standard-deviation perturbations relative to equilibrium baseline. If the intervention variable is a stock, its initial condition is shifted by $\bar{x}(\tau)$1. If it is an auxiliary, a constant shift of $\bar{x}(\tau)$2 is added to its algebraic equation. If it is a constant, its value is set to $\bar{x}(\tau)$3 for the entire simulation horizon. The resulting endpoint effect on a variable of interest is recorded across many independent parameter samples, producing a distribution rather than a point estimate (Uleman et al., 30 Jul 2025).

Leverage points are then ranked using the distribution of intervention effects. The method uses the median effect as a primary ranking statistic and computes pairwise probabilistic dominance: $\bar{x}(\tau)$4 with bootstrap confidence intervals. Sensitivity analysis is performed using Spearman’s rank correlation between each parameter and the VOI outcome, either overall or conditional on a specific intervention. This identifies which empirical associations would most reduce uncertainty in leverage-point ranking if data were collected (Uleman et al., 30 Jul 2025).

The method is implemented in a Python package, systemdynamics, and a Streamlit web application. The primary input format is a Kumu-compatible Excel workbook with sheets for Elements, Connections, and optionally Interactions. The Elements sheet contains Label, Type, Tags, and Description; the Connections sheet encodes From, To, and Type for link polarity (Uleman et al., 30 Jul 2025).

4. Uncertainty, ranking, and the Alzheimer’s disease case

The published case study uses a CLD of Alzheimer’s disease previously developed in related work and compares D2D with both centrality analysis and a data-driven calibrated SDM built from the same CLD and variable labeling (Uleman et al., 30 Jul 2025). The CLD includes variables across brain health, physical health, psychosocial factors, and risk factors, with reinforcing loops and mixed positive and negative polarities.

The chosen timeframe is 5 years with a base time unit of 3 months, yielding 20 time steps. The labeling assigns several variables as stocks, including Cognitive functioning, Brain atrophy, Neuronal dysfunction, Cerebral endothelial dysfunction, Amyloid beta burden, Daily functioning, Morbidity burden, Depressive symptoms, Obesity, Blood pressure, and Tau burden. Variables such as Engagement in cognitively demanding tasks, Healthy dietary patterns, Physical activity, Sleep quality, Experienced stress, Systemic inflammation, Brain perfusion, Oxidative stress, Neuroinflammation, and Neuronal connectivity are treated as auxiliaries. Head trauma, ApoE-4 carriership, Education level, Diabetes, Dyslipidaemia, Social relationships, Hearing loss, Smoking, Excessive alcohol use, and Motor function are treated as constants under the chosen time-scale assumptions (Uleman et al., 30 Jul 2025).

Fifteen modifiable factors are explored as intervention variables. Decreasing interventions are applied to Obesity, Dyslipidaemia, Blood pressure, Diabetes, Smoking, Depressive symptoms, Hearing loss, Head trauma, and Excessive alcohol use; increasing interventions are applied to Education level, Physical activity, Social relationships, Healthy dietary patterns, Sleep quality, and Engagement in cognitively demanding tasks (Uleman et al., 30 Jul 2025). Using $\bar{x}(\tau)$5 independent parameter samples, the method evaluates change in Cognitive functioning after 5 years relative to baseline.

The reported median ranking from largest to smallest beneficial effect is:

Rank	Intervention
1	Sleep quality (+1)
2	Depressive symptoms (−1)
3	Social relationships (+1)
4	Physical activity (+1)
5	Healthy dietary patterns (+1)
6	Head trauma (−1)
7	Smoking (−1)
8	Hearing loss (−1)
9	Excessive alcohol use (−1)
10	Diabetes (−1)
11	Education level (+1)
12	Obesity (−1)
13	Dyslipidaemia (−1)
14	Blood pressure (−1)
15	Engagement in cognitively demanding tasks (+1)

The paper emphasizes that many of these distributions overlap substantially, so many pairwise distinctions are not decisive (Uleman et al., 30 Jul 2025). By contrast, some extremes are robust. The probability that Sleep quality yields greater cognitive benefit than Depressive symptoms is reported as 73% with bootstrap interval [67%, 79%]; against Physical activity, 79% [73%, 85%]; against Obesity, Blood pressure, Dyslipidaemia, and Engagement in cognitively demanding tasks, 100% [100%, 100%]. The intended interpretation is therefore not a sharp total ordering but a probabilistic differentiation of clearly high-ranked versus clearly low-ranked leverage points.

Sensitivity analysis further identifies parameters to which predicted cognitive outcomes are most sensitive. Across all interventions, the strongest reported associations are Neuronal dysfunction $\bar{x}(\tau)$6 Cognitive functioning, Depressive symptoms $\bar{x}(\tau)$7 Cognitive functioning, Hearing loss $\bar{x}(\tau)$8 Cognitive functioning, Neuronal connectivity $\bar{x}(\tau)$9 Cognitive functioning, Brain atrophy $T>0$0 Cognitive functioning, Neuronal dysfunction $T>0$1 Neuronal connectivity, and Sleep quality $T>0$2 Cognitive functioning (Uleman et al., 30 Jul 2025). For the Sleep-quality intervention specifically, the dominant sensitivities are Sleep quality $T>0$3 Cognitive functioning, Depressive symptoms $T>0$4 Cognitive functioning, Sleep quality $T>0$5 Depressive symptoms, Depressive symptoms $T>0$6 Sleep quality, and Neuronal dysfunction $T>0$7 Cognitive functioning. This suggests that D2D functions not only as an exploratory ranking device but also as a data-prioritization instrument.

5. Comparison with centrality analysis and related alternatives

A major theme of the CLD-based D2D work is that dynamic simulation under polarity-constrained uncertainty differs fundamentally from graph-theoretic centrality. Closeness centrality,

$T>0$8

and betweenness centrality,

$T>0$9

are treated as insufficient proxies for causal leverage because they assume influence propagates along shortest paths, often ignore polarity, and do not distinguish stocks, auxiliaries, and constants (Uleman et al., 30 Jul 2025).

In the Alzheimer’s example, betweenness centrality ranks Physical activity highest and cannot score many constants because it is only defined for variables with both in-degree and out-degree. Closeness centrality ranks Social relationships and Physical activity above Sleep quality and Depressive symptoms, and assigns relatively high scores to Obesity and Education level, which are low leverage in both D2D and the calibrated SDM (Uleman et al., 30 Jul 2025). The paper’s conclusion is not that centrality is useless, but that it is liable to false inference when used as a surrogate for dynamic leverage on causal diagrams.

The robotics D2D literature presents an analogous contrast between structurally constrained dynamics and generic neural dynamics. In simulation, SDDT is compared with Neural ODEs and the unmorphed base circular system. Using Hausdorff distance between the portion of the end-effector trajectory on the surface and the projected sketch, the reported results are:

Method	O.A.S. (basin size)	Star HD
SDDT (ours)	51	0.011
Neural ODEs	55	0.040
Base System (circle)	51	0.133

For the knight and arrow sketches, the corresponding Hausdorff distances are 0.011 and 0.017 for SDDT, 0.035 and 0.063 for Neural ODEs, and 0.201 and 0.209 for the base system (Zhi et al., 2023). The paper interprets this as evidence that enforcing structure—orbital asymptotic stability plus diffeomorphic morphing of a stable template—is beneficial relative to learning a vector field from scratch.

A plausible implication is that D2D methods often trade unrestricted expressivity for constrained semantic faithfulness. In the CLD setting, this means respecting direction, polarity, and accumulation; in the robotics setting, it means preserving orbital stability while learning geometric shape.

6. Generalized and abstract formulations

The phrase “Diagrams-to-Dynamics” also admits more abstract interpretations in which the central object is not a specific sketch or CLD but a broader translation from diagrammatic syntax to dynamical semantics.

In diagram understanding, the Dynamic Graph Generation Network defines a dynamic adjacency tensor memory

$x_0$0

whose first channel stores edge probabilities and whose remaining channels store edge hidden states (Kim et al., 2017). For a candidate edge between objects $x_0$1 and $x_0$2, the retrieve step computes a graph-conditioned hidden state

$x_0$3

which is then updated by GRU equations and written back into memory. The resulting internal process can be interpreted as a time-discrete dynamical system whose state is the evolving graph memory $x_0$4, driven by candidate pair features and global diagram context (Kim et al., 2017). This suggests a broader D2D reading in which a static diagram is translated into a dynamical state-estimation process rather than directly into a physical or causal process model.

An even more general formulation is provided by compositional system dynamics. There, diagram types such as CLDs, SSDs, and SFDs are formalized as attributed C-sets, that is, functors

$x_0$5

with morphisms given by natural transformations (Li et al., 22 Sep 2025). The D2D mapping is then a syntax–semantics functor from a category of diagrams to a category of dynamical systems. For SFDs, the canonical semantics maps stocks to state variables, flows to terms in stock derivatives, and auxiliaries and parameters to algebraic expressions, recovering the standard law

$x_0$6

This categorical approach supports modularity via structured cospans

$x_0$7

and composition via pushouts in the diagram category (Li et al., 22 Sep 2025). It also supports mappings between diagram types, such as $x_0$8, $x_0$9, and $\mathcal{X}$0. The same syntactic object can therefore be given multiple semantics: ODE systems, stochastic transition systems, eigenvalue elasticities, loop gains, and, prospectively, hybrid systems and temporal sheaves. This suggests that D2D is not only a method family but also a general methodological schema: diagrams are syntax, dynamics are semantics, and the translation between them should preserve structure.

7. Limitations, misconceptions, and open problems

Several misconceptions recur across the D2D literature. One is that a diagram alone determines a unique dynamical system. The published methods do not support that claim. In robotics, the single user sketch specifies the target orbit geometry, but the resulting system also depends on the choice of base O.A.S. system, the diffeomorphism class, and optimization under Hausdorff loss (Zhi et al., 2023). In CLD-based D2D, the CLD is insufficient without additional labeling of variables as stocks, auxiliaries, or constants, plus assumptions about parameter magnitudes, timeframe, and intervention direction (Uleman et al., 30 Jul 2025). In the categorical formulation, a diagram acquires dynamics only relative to a chosen semantics functor (Li et al., 22 Sep 2025).

A second misconception is that D2D eliminates uncertainty. The opposite is true in the CLD-based setting: D2D is explicitly designed for data-poor contexts and relies on broad polarity-constrained priors, prior predictive simulation, and probabilistic ranking (Uleman et al., 30 Jul 2025). Wide uncertainty intervals are treated as informative outputs rather than failures. A similar structural conservatism appears in robotics, where the emphasis is on preserving orbital asymptotic stability by construction rather than on unconstrained function approximation (Zhi et al., 2023).

The main limitations are domain-specific. SDDT assumes a planar contact surface aligned with $\mathcal{X}$1, a single sketch corresponding to a single limit cycle, and no explicit force control; extending to curved surfaces, multi-cycle behaviors, or diagrammatic force and impedance specification remains open (Zhi et al., 2023). CLD-based D2D assumes structural adequacy of the CLD, causal interpretability of links, linearity and additivity unless interactions are added, correct polarity, meaningful timescale separation, independent parameter priors, and stationarity over the timeframe (Uleman et al., 30 Jul 2025). Mislabeling a variable as stock, auxiliary, or constant can substantially change the resulting dynamics. The categorical framework, while mathematically robust, is primarily a formal foundation; its practical implications depend on the design of concrete semantics and software implementations (Li et al., 22 Sep 2025).

Future directions stated in the sources include extending CLD-based D2D with Bayesian calibration, more flexible functional forms such as saturating links and explicit delays, and additional case studies across domains (Uleman et al., 30 Jul 2025); extending robotics D2D to curved surfaces, more general attractors, and richer task classes (Zhi et al., 2023); and developing alternative semantics, dimensional annotations, hybrid and agent-based extensions, and temporal-sheaf-based temporal reasoning within the categorical framework (Li et al., 22 Sep 2025).

Taken together, these strands establish D2D as a cross-disciplinary program rather than a single algorithm. Its unifying principle is that diagrammatic objects—sketches, causal maps, or formalized system diagrams—can be treated as structured inputs to the construction of dynamical systems. What varies is the nature of the diagram, the semantics of the transformation, and the guarantees sought: orbital stability in robot motion (Zhi et al., 2023), uncertainty-aware exploratory leverage analysis in causal systems (Uleman et al., 30 Jul 2025), evolving graph-state inference in diagram understanding (Kim et al., 2017), or compositional functorial semantics in abstract system dynamics (Li et al., 22 Sep 2025).