X-Dyna: Simulation & Animation

• X-Dyna is a dual-framework system that integrates ab initio XFEL-driven molecular dynamics and a diffusion-based pipeline for zero-shot human image animation.
• It employs a Monte Carlo particle dynamics approach with XATOM-derived rates and a symplectic integrator to accurately simulate electron–ion interactions under extreme conditions.
• It harnesses latent diffusion with modular conditioning—using ControlNet and Dynamics-Adapter layers—to generate lifelike human animations with coherent motion and expressions.

X-Dyna encompasses two distinct research directions in computational simulation and generative modeling. First, it denotes XMDYN (sometimes called X-Dyna), a particle-based ab initio molecular dynamics code for simulating non-equilibrium matter under extreme x-ray irradiation. Second, X-Dyna refers to an expressive diffusion-based pipeline for zero-shot human image animation, integrating dynamic context from both subjects and background. Both are advanced, modular systems at the intersection of physics-driven and deep-learning-based simulation, leveraging sophisticated architectures, on-the-fly adaptation, and compositionality.

1. Frameworks and Scope

XMDYN ("X-Dyna") is a Monte Carlo molecular dynamics (MD) framework designed for simulating coupled electron–ion dynamics in solid-density samples subjected to x-ray free-electron-laser (XFEL) pulses, typically at photon energies from a few hundred electronvolts to tens of kiloelectronvolts and pulse durations of a few femtoseconds. The architecture combines classical particle motion under mutual Coulomb interactions with quantum-derived electronic transition rates and cross sections, sampled dynamically using data from the XATOM atomic physics toolkit. This allows modeling of both light and heavy atomic species without a priori truncation of electronic state space, under fully three-dimensional periodic boundary conditions (Abdullah et al., 2017).

The generative X-Dyna pipeline ("^^^^1^^^^") is a zero-shot, diffusion-based framework for realistic video animation from a single reference image plus driving video pose/appearance cues. It is based on a frozen Stable Diffusion 1.5 (SD) latent-diffusion UNet backbone augmented with temporal attention layers (e.g., from AnimateDiff) and several trainable modules: a Dynamics-Adapter for injecting appearance context, a pose ControlNet for body pose, and a local ControlNet for identity-disentangled facial expressions. It is trained on a diverse corpus including human motion content and natural scene dynamics (Chang et al., 17 Jan 2025).

2. Mathematical and Algorithmic Foundations

XMDYN: Coupled MD–Stochastic Dynamics

At each time step $\Delta t$, the code updates all particle positions and velocities using a symplectic velocity-Verlet integrator: $$r_i(t+\Delta t) = r_i(t) + v_i(t)\Delta t + \frac{1}{2m_i}F_i(t)\Delta t^2$$

$$v_i(t+\Delta t) = v_i(t) + \frac{1}{2m_i}\left(F_i(t) + F_i(t+\Delta t)\right)\Delta t$$

where the force $F_i$ for each particle incorporates all pairwise Coulomb terms (sum over all particles and their periodic images).

Electronic processes (photoionization, Auger decay, fluorescence, collisional ionization, recombination) are sampled on-the-fly using cross sections $\sigma$ and decay rates $\Gamma$ retrieved from XATOM for each atom/ion's instantaneous state. Probabilities per $\Delta t$ are computed as: $P_{\text{event}} = 1 - \exp(-\sigma \Phi \Delta t)$ with $\Phi$ the local photon flux for photoionization. The collisional ionization check uses energy-dependent impact parameter criteria.

X-Dyna Human Animation: Diffusion and Modular Conditioning

The generative X-Dyna pipeline operates in the latent space of a frozen Stable Diffusion 1.5 UNet:

• Inputs: one reference human image $I_R$ (encoded as latent $z_R$), a series of pose skeletons $\{P_i\}$, and face crops $\{F_i\}$ (latent-encoded).
• For each DDPM step $t$, the UNet receives the noisy latent $z_t$, pose features from ControlNet $C_P(P_i)$, face features from ControlNet $C_F(F_i)$, and appearance context via the Dynamics-Adapter $\mathcal{D}(z_R)$.
• The Dynamics-Adapter modifies spatial self-attention:
  • In each UNet block $i$, original attention is $$A_i = \operatorname{softmax}(Q_i K_i^T / \sqrt{d}) V_i$$.
  • A cross residual $A'_i$ is computed as $$A'_i = \operatorname{softmax}(Q'_i K_R^T / \sqrt{d})V_R$$ with $Q'_i = W_{Q}' z_i$.
  • Outputs fuse: $\text{Out}_i = A_i W_O + A'_i W'_O$.
• All modules except the Dynamics-Adapter's $W_{Q}'$ and $W_O'$, and the ControlNet branches, are frozen.

Motion propagation leverages temporal attention and 3D convolutional layers, matching driving pose/expression while allowing full-scene dynamic synthesis. The pipeline is optimized with standard denoising diffusion loss: $$L_{\text{simple}} = \mathbb{E}_{t,z_0,\epsilon}\left[\| \epsilon - \epsilon_\theta(z_t, t, \text{cond}) \|^2\right]$$ with $\text{cond} = \{P_i, F_i, z_R\}$.

3. Modular Architecture and Data Integration

Module	XMDYN Functionality (Abdullah et al., 2017)	X-Dyna Human Animation Role (Chang et al., 17 Jan 2025)
Main Engine	MD with MC sampling, periodic BCs	SD UNet 1.5 with temporal/3D-conv blocks
Appearance Adapter	n/a	Dynamics-Adapter $\mathcal{D}(z_R)$, spatial attn mod
Control Branches	n/a	C_P: pose ControlNet, C_F: face ControlNet
Temporal Modeling	n/a	AnimateDiff, SVD temporal attention, 3D conv layers
Data Sources	XFEL pulse, lattice initial config	Human/scene video: 900h motion + 3k timelapse clips
Evaluation Metrics	Temp., charge, energy, distributions	L1, PSNR, SSIM, LPIPS, FID, face-cosine, DTFVD, etc.

In the XMDYN regime, the only required external data are atomic cross sections/rates (from XATOM) and sample geometry. In the X-Dyna human animation system, extensive, harmonically mixed human and natural scene videos are employed for pretraining and finetuning, with module-specific initialization and staged training.

4. Workflow and Training Procedures

XMDYN simulation begins by specifying a supercell (e.g., 512 atoms in diamond) and initializing all species in neutral states. The XFEL temporal profile dictates the photon flux over the simulation window. The molecular dynamics loop steps through force computations, event sampling (photoionization, decay, collisions, recombination), and dynamic update of species and observables (energy, charge, electron distributions), with outputs ready for subsequent benchmarking (e.g., against average-atom or Boltzmann models).

The X-Dyna animation pipeline involves two-phase training:

• Stage 1: All modules except the face ControlNet are trained on the mixed corpus for 5 epochs.
• Stage 2: All but C_F are frozen; C_F is finetuned on human videos for 2 epochs, reinforcing expression disentanglement and transfer. During inference, the model directly encodes actual face crops (no external reenactment net) and applies zero-shot animation to an unseen reference image and arbitrary pose/expression sequence.

5. Evaluation, Benchmarks, and Limitations

XMDYN Performance

Validation is performed via:

• Comparison with Average-Atom (AA) models using Maxwell-Boltzmann fits to electron velocities and energetic matching—temperatures agree to within $\sim$10 eV up to $10^{11}$ ph/$\mu$m$^2$ fluences.
• Comparison with continuum Boltzmann approaches for charge evolution in diamond, yielding close matches in absorbed energy per atom and final charge states. Small residual quantitative differences are attributed to different treatments of collisional ionization and many-body recombination.
• Scalability is $O(N_{\text{particles}}^2)$ per step, but large configuration spaces are tractable, and single runs for 1000-atom supercells complete in $\sim$45 minutes on GPU with typical statistics from 20–50 trajectories.

X-Dyna Animation Metrics

Evaluation leverages several quantitative and perceptual metrics:

• Foreground prediction: L1, PSNR, SSIM, LPIPS.
• Identity preservation: Face-Cosine similarity (AdaFace), Face-Det (detection rate).
• Video realism: FID, content-debiased FVD.
• Dynamics specificity: Dynamic-Texture FVD (on video/foreground/background).

X-Dyna demonstrates improved synthesis of lifelike hair, cloth motion, water, fire, and background scene dynamics over ReferenceNet and SVD-based baselines. Facial expression transfer is identity-preserving, with precise alignment of lip and eyebrow movement.

Limitations include reliance on extensive pretraining corpora, sensitivity to the quality of pose/expression extraction, and the frozen nature of the diffusion backbone, which constrains further adaptation beyond the light-weight plugin modules.

6. Notable Applications and Contributions

XMDYN enables ab initio simulation and diagnostics of XFEL-driven nonequilibrium dynamics in complex condensed matter and molecular samples, including treatment of mixed heavy and light elements, and realistic plasma formation scenarios (Abdullah et al., 2017).

X-Dyna ("Expressive Dynamic Human Image Animation") offers a unified, zero-shot framework for high-fidelity, context-aware video animation from a single image, capturing both human and environment dynamics. Its modular design, particularly the Dynamics-Adapter and identity-disentangling local face ControlNet, allows synthesis of fluid, intricate, and highly realistic motion and background evolution, establishing a new state of the art for single-image video animation benchmarks (Chang et al., 17 Jan 2025).