--- title: Latent Laplace Diffusion (LLapDiff) url: https://www.emergentmind.com/topics/latent-laplace-diffusion-llapdiff type: topic --- # Latent Laplace Diffusion (LLapDiff) Searching arXiv for the LLapDiff paper and closely related latent/Laplace diffusion work to ground the article in current papers. Latent Laplace Diffusion (LLapDiff) is a generative framework for irregular multivariate time series that models the target as a low-dimensional latent trajectory and performs horizon-wide generation without step-by-step integration over physical time. It combines a latent target encoder/decoder, a gap-aware history summarizer, and a latent diffusion model whose reverse process is guided by a stable modal parameterization motivated by stochastic port-Hamiltonian dynamics. Its mean evolution is parameterized in the Laplace domain via learnable complex-conjugate poles, so the latent trajectory can be evaluated directly at arbitrary irregular timestamps in closed form [2605.19805]. In the broader literature, LLapDiff sits at the intersection of nonlocal Laplacian diffusion, spectral Laplacian restoration, and latent diffusion modeling: the self-similar Laplacian provides a physically derived nonlocal diffusion generator [1207.7132], Laplacian-eigenbasis restoration operationalizes diffusion in a Laplace-structured spectral space [2402.08563], and latent diffusion in other domains supplies the compression-first rationale for moving generative modeling away from raw observation space [2605.28427; 2601.13780]. ## 1. Problem setting and conceptual motivation LLapDiff addresses long-horizon forecasting for irregular multivariate time series, where discrete methods and continuous-time methods induce different failure modes. Discrete methods usually re-grid or interpolate irregular data onto a regular timeline, which can distort temporal structure via re-gridding. Continuous-time models preserve timestamps, but usually require step-by-step numerical integration with ODE/CDE/SDE solvers, which is expensive and can accumulate drift over long horizons. Standard diffusion time-series models denoise directly in observation space and treat irregularity mostly through masks and embeddings, so they lack explicit stability and dynamical structure [2605.19805]. The observed history is written as $$ H_{t_i}=\{(t_j,X_{t_j},M_{t_j})\}_{j=i-\ell+1}^{i}, $$ and the target or query window as $$ V_{t_i}=\{(t_q,Y_{t_q},M_{y,t_q})\}_{q=1}^{h}. $$ Queries may be future forecast points or historical missing-value locations. LLapDiff models $$ p(z \mid E_{t_i}), $$ where $z$ is a latent target trajectory over the query window and $E_{t_i}$ is a learned summary of the irregular history [2605.19805]. The model’s central design choice is to generate a low-dimensional latent trajectory rather than raw observations. This places LLapDiff in a broader latent diffusion lineage. In missing-data imputation, latent diffusion was studied through a two-stage framework in which a robust VAE-based imputer first learns compact semantic features from incomplete observations and a diffusion model is then trained in the resulting latent space; on MNIST, that model remained stable up to $50\%$ missingness and achieved consistently better performance than pixel-space diffusion [2605.28427]. In graph generation, latent diffusion was paired with Laplacian autoencoders to obtain a latent representation whose dimensionality scales linearly with the number of nodes, eliminating the quadratic bottleneck of adjacency-space diffusion [2601.13780]. These results do not define LLapDiff, but they support the general premise that latent-space modeling can stabilize generative learning when raw-space diffusion is brittle. ## 2. End-to-end architecture LLapDiff has three main components. First, a latent target encoder/decoder maps the ground-truth query window into a low-dimensional latent sequence, $$ z_0 = \mathrm{VAEenc}(V_{t_i}) \in \mathbb{R}^{h\times d_z}. $$ Second, a history summarizer maps the irregular history into a token sequence, $$ E_{t_i} = S_\theta(H_{t_i}). $$ Third, a latent diffusion model corrupts the latent target and reverses that corruption while predicting stable modal parameters $$ \{(p_k,\omega_k,C_k,b_k)\}_{k=1}^K, $$ which define a closed-form latent trajectory over all query timestamps [2605.19805]. The target is represented as $$ z = \{z(t_q)\}_{q=1}^h \in \mathbb{R}^{h\times d_z}. $$ During training, the model encodes the target window with a pretrained VAE, diffuses the latent trajectory, and learns to reconstruct the clean latent $z_0$. At inference, it samples $z_T \sim \mathcal{N}(0,I)$, reverse diffuses to obtain $\hat z_0$, and decodes $\hat z_0$ to produce forecasts or imputations [2605.19805]. A defining property is that the model is solver-free over physical time: it does not integrate an ODE step-by-step across the forecast horizon. Once modal parameters are predicted, the whole trajectory is synthesized in one parallel pass. This differs sharply from standard continuous-time models, whose evolution typically takes the form $$ x(t) = x(t_0)+\int_{t_0}^{t} f(x(\tau),\tau)\,d\tau. $$ A plausible implication is that LLapDiff is designed to decouple horizon length from numerical rollout cost in physical time, replacing solver dependence with latent denoising plus closed-form modal evaluation [2605.19805]. ## 3. Laplace-domain dynamics and stability bias The model’s stability prior is motivated by stochastic port-Hamiltonian dynamics. LLapDiff introduces an auxiliary state $x_t$ evolving according to $$ dx_t = \Big[(J-R)\nabla_x H(x_t;\nu_t) + G(\nu_t)u_t\Big]dt + E_t\,dW_t, $$ where $J^\top=-J$ is skew-symmetric, $R>0$ is dissipative, $H(x_t;\nu_t)$ is the Hamiltonian, $u_t$ is context or input, and $E_t dW_t$ is stochastic noise. The port-collocated output is $$ y_t = G(\nu_t)\nabla_x H(x_t;\nu_t). $$ Applying Itô’s lemma yields the expected energy balance $$ \frac{d}{dt}\mathbb{E}[H(x_t;\nu_t)] = \mathbb{E}[\nabla H^\top \dot{\nu}_t] - \mathbb{E}[\nabla H^\top R \nabla H] + \mathbb{E}[y_t^\top u_t] + \mathbb{E}[\operatorname{tr}(E_t^\top \nabla^2 H E_t)]. $$ Between context updates, the input term vanishes, so the expected energy is non-increasing in the noise-free unforced case [2605.19805]. LLapDiff uses this as a prior for the clean latent trajectory, $$ x_t \leftrightarrow z_0(t), $$ biasing the learned mean dynamics of the diffusion denoiser toward dissipative, stable behavior. The paper then linearizes the dynamics around a local operating point $(x_{t_0},\nu_{t_0})$, defining $$ \delta x_t = x_t - x_{t_0}, \qquad \delta u_t = u_t - u_{t_0}, $$ with linearized SDE $$ d(\delta x_t)=\big(A\,\delta x_t + B\,\delta u_t\big)\,dt + E_{t_0}dW_t, $$ where $$ A=(J-R)\nabla_x^2 H(x_{t_0};\nu_{t_0}), \qquad B=G(\nu_{t_0}). $$ Taking expectations removes the martingale term: $$ \mathbb{E}[\delta x_t] = e^{A(t-t_0)}\mathbb{E}[\delta x_{t_0}] + \int_{t_0}^{t} e^{A(t-\tau)}B\,\mathbb{E}[\delta u_\tau]\,d\tau. $$ For a linear readout $y_t=Cx_t$, the mean output is $$ \mathbb{E}[y_t] = Ce^{A(t-t_0)}\mathbb{E}[\delta x_{t_0}] + \big(g * \delta u\big)(t), $$ with $$ g(t)=Ce^{At}B. $$ The unilateral Laplace transform gives the transfer function $$ G(s)=C(sI-A)^{-1}B. $$ This is the basis for LLapDiff’s Laplace-domain parameterization of mean dynamics [2605.19805]. Rather than explicitly learning $A$, LLapDiff learns a stable rational modal form: $$ G(s)=\sum_{k=1}^{K} \frac{C_k\,\omega_k\, b_k}{s^2+2p_k s+(p_k^2+\omega_k^2)}, $$ where $p_k>0$ is the decay rate, $\omega_k>0$ is the oscillation frequency, and $b_k, C_k$ are low-rank residue vectors. The corresponding complex-conjugate poles are $$ s_k = -p_k \pm i\omega_k. $$ The positivity constraint $p_k>0$ ensures stability. Each mode can be realized by a real $2\times 2$ block $$ A_k= \begin{bmatrix} -p_k & \omega_k \\ -\omega_k & -p_k \end{bmatrix}, \qquad B_k= \begin{bmatrix} 1\\0 \end{bmatrix} b_k^\top, \qquad C_k = C_k[-1\ \ 0]. $$ Stacking these yields a full stable state-space system with $$ A=\operatorname{blkdiag}(A_1,\dots,A_K). $$ Since each block has eigenvalues $-p_k \pm i\omega_k$, the system is Hurwitz whenever $p_k>0$ [2605.19805]. ## 4. Irregular timestamps, renewal averaging, and the history summarizer Once poles and residues are predicted, the latent trajectory can be evaluated at any relative query time $\tau_q$ using $$ \hat z_0(\tau_q) = \sum_{k=1}^K e^{-p_k \tau_q} \Big( C_k \cos(\omega_k \tau_q) + b_k \sin(\omega_k \tau_q) \Big), $$ optionally plus a small residual correction MLP. This permits parallel closed-form synthesis over all query timestamps without recursive rollout from one query point to the next [2605.19805]. A major theoretical contribution is the renewal-averaging analysis. Let irregular gaps be $$ \Delta_j = t_j - t_{j-1} \ge 0. $$ For a single continuous-time mode $$ \zeta(t)=e^{s_k(t-t_0)}\zeta(t_0),\qquad s_k=-p_k+i\omega_k, $$ the event-time recursion becomes $$ \zeta_{j+1}=e^{s_k\Delta_{j+1}}\zeta_j. $$ Taking expectations yields $$ \mathbb{E}[\zeta_{j+1}] = \mathbb{E}[e^{s_k\Delta}]\,\mathbb{E}[\zeta_j] = \lambda_k \mathbb{E}[\zeta_j], $$ where $$ \lambda_k = \mathbb{E}[e^{s_k\Delta}] $$ is the effective discrete or event-domain multiplier, and the effective event-domain log-pole is $$ \tilde s_k = \log \lambda_k. $$ This shows that irregular sampling maps continuous-time poles into effective discrete dynamics [2605.19805]. Because $p_k>0$, $$ |e^{s_k\Delta}| = e^{-p_k\Delta}\le 1, $$ hence $$ |\lambda_k| = \left|\mathbb{E}[e^{s_k\Delta}]\right| \le \mathbb{E}[|e^{s_k\Delta}|] = \mathbb{E}[e^{-p_k\Delta}] \le 1, $$ which implies $$ \Re(\tilde s_k)=\log|\lambda_k|\le 0. $$ The paper therefore concludes that random sampling preserves mean stability and often increases attenuation. It also gives the second-order approximation $$ \tilde s_k = \log \mathbb{E}[e^{s_k\Delta}] \approx s_k\mathbb{E}[\Delta] +\frac{1}{2}s_k^2\operatorname{Var}(\Delta), $$ showing that the effective observed dynamics depend not only on intrinsic pole parameters but also on the gap distribution [2605.19805]. This directly motivates the gap-aware history summarizer $S_\theta$. LLapDiff uses three token types: port tokens based on observed values, dynamics tokens based on local differences or finite-difference proxies, and temporal tokens based on timestamps, per-step gaps, and masks. These tokens are fused into a history summary token sequence $E_{t_i}$, which conditions the reverse diffusion and modal pole prediction [2605.19805]. The token ablations reported in the paper show that the temporal token is the most important, with dynamics tokens next and port tokens less severely but still adversely affecting performance when removed. This supports the renewal-gap theory [2605.19805]. ## 5. Diffusion formulation and inference LLapDiff uses a standard DDPM forward process on the latent trajectory: $$ q(z_T\mid z_0)=\mathcal{N}(\sqrt{\bar\alpha_T}z_0,(1-\bar\alpha_T)I), $$ equivalently, $$ z_T=\sqrt{\bar\alpha_T}z_0+\sqrt{1-\bar\alpha_T}\,\epsilon,\qquad \epsilon\sim\mathcal{N}(0,I). $$ The denoiser predicts the clean latent trajectory from the noisy latent, diffusion step, and history summary: $$ \hat z_0 = C_\theta(z_T,T,E_{t_i}), $$ where $C_\theta$ is implemented as a modal predictor plus modal synthesizer. The training objective is the standard $x_0$-parameterization with MSE, $$ \mathcal{L}(\theta) = \mathbb{E}\left[\|z_0 - \hat z_0(z_T,T,E_{t_i})\|_2^2\right]. $$ During inference, DDIM-style sampling is used with conditional and unconditional predictions combined through classifier-free guidance: $$ \hat z_0^{\text{guided}} = \hat z_0^{\text{cond}} + w(\hat z_0^{\text{cond}}-\hat z_0^{\text{uncond}}). $$ The modal predictor constrains poles by construction, $$ p_k=\operatorname{Softplus}(\cdot)+p_{\min}, \qquad \omega_k = \omega_{\max}\sigma(\cdot), $$ so $p_k>0$ [2605.19805]. The training and inference pipelines are explicit. During training, a history-target pair $(H_{t_i},V_{t_i})$ is sampled, the target is encoded as $z_0=\mathrm{VAEenc}(V_{t_i})$, the history summary $E_{t_i}=S_\theta(H_{t_i})$ is built, conditioning may be dropped with probability $p_{\text{uncond}}$, a diffusion step and noise are sampled, the latent is corrupted, and modal parameters are predicted to synthesize $\hat z_0$, minimizing $\|z_0-\hat z_0\|_2^2$. During inference, the model builds $E_{t_i}$, samples $z_T\sim\mathcal{N}(0,I)$, reverse diffuses with DDIM-style steps, predicts conditional and unconditional modal parameters at each step, applies guidance, and decodes $\hat z_0$ to obtain forecasts or imputations [2605.19805]. The paper emphasizes that LLapDiff’s generation does not scale with horizon length via sequential physical-time integration. Modal synthesis is parallel over query points, so inference time is nearly flat across forecast horizons, with runtime dominated by the diffusion loop rather than trajectory rollout [2605.19805]. ## 6. Empirical behavior, ablations, and operational scope LLapDiff is evaluated on seven datasets: BMS Air, UCI Air, PhysioNet, NOAA US, NOAA UK, Crypto, and US Equity. The baselines include DLinear, PatchTST, mr-Diff, TimeGrad, mTAN, T-PATCHGNN, ContiFormer, NeuralCDE, and CSDI for imputation comparisons [2605.19805]. The paper reports that LLapDiff achieves the best average rank overall and is especially strong at longer horizons, more irregular datasets, and sparser observation regimes. A representative example is NOAA UK at horizon $h=168$, where LLapDiff reduces MSE from $11.126$ for the strongest baseline in the table to $6.176$ [2605.19805]. Qualitatively, forecasts remain coherent through regions with multiple missing values, where many baselines drift or oversmooth. For imputation, the same model reconstructs masked historical values by querying historical timestamps instead of future ones [2605.19805]. The imputation mechanism follows directly from the continuous-time latent trajectory formulation. To impute missing entries inside the observed window, the model includes the missing historical timestamps in the query set, keeps the same conditioning history $H_{t_i}$, synthesizes the latent at those historical times, and decodes to recover imputed values. The paper characterizes this as causal or filtering-style imputation rather than bidirectional smoothing [2605.19805]. The ablations isolate the role of each major component. Removing learned poles significantly hurts performance. Removing the latent trajectory formulation degrades results, indicating that compact latent generative modeling is important for long-horizon stability. Removing history conditioning causes the largest degradation, confirming that the summary sequence is essential. The learned poles all satisfy $p_k>0$, staying safely away from instability, and conditioning shifts the pole distribution, indicating that the summarizer adapts damping and frequency to the observed gap pattern [2605.19805]. Under artificially increased missingness, performance degrades gracefully, and the model remains relatively stable unless informative channels are severely lost. Controlled regime shift experiments show that the learned modal dynamics generalize under frequency and decay shifts, though extreme shifts degrade performance somewhat [2605.19805]. The stated limitations are twofold: dependence on pretrained latent representations and a locally linear modal approximation that may not fully capture richer nonlinear behavior [2605.19805]. ## 7. Relation to Laplacian diffusion and neighboring research directions The name “Latent Laplace Diffusion” invites comparison with several distinct uses of Laplace or Laplacian structure in generative modeling. The oldest of these in the present set is the construction of a self-similar Laplacian in $n$ dimensions from a quasi-continuous linear chain with self-similar harmonic interparticle springs. There, self-similarity at a point is imposed by $$ A(Nh) = N^{\delta}A(h), $$ and Hamilton’s variational principle yields a non-local integral operator with power-law kernel. In $n$ dimensions the operator is $$ \Delta_{(\delta,h)}u(\mathbf{x}) = \frac{h^{\delta-(n-1)}}{2} \int_{\mathbb{R}^n} \frac{u(\mathbf{x}+\mathbf{r})+u(\mathbf{x}-\mathbf{r})-2u(\mathbf{x})}{r^{\delta+1}}\, d^n r, $$ for $0<\delta-(n-1)<2$, and it coincides, up to a strictly positive prefactor, with the fractional Laplacian, $$ \Delta_{(\delta,h)} = -A_{n,a}\,(-\Delta)^{a/2}, \qquad a=\delta-(n-1),\ 0