Second-Order TPDEGs: Advanced Graph Neural PDEs

• Second-Order TPDEGs are a multidomain graph framework that employs oscillatory dynamics to overcome the smoothing limitations of first-order models.
• The approach leverages separability of cosine kernels on Cartesian product graphs to enable efficient spectral filtering and closed-form analytical solutions.
• The methodology ensures stability under graph perturbations and mitigates over-smoothing while reducing computational complexity via factorized eigen-decompositions.

Second-Order TPDEGs (So-TPDEGs) denote, in the explicit contemporary usage of the term, a second-order, wave-like extension of tensorial partial differential equations on Cartesian product graphs, introduced as a framework for second-order continuous product graph neural networks (Einizade et al., 2 Sep 2025). In this formulation, multidomain data are modeled on multiple interacting graphs, and the passage from first-order to second-order dynamics is motivated by the observation that first-order continuous product-graph models behave like low-pass filters: they smooth signals, suppress high-frequency components, and can be less effective for multi-scale and heterophilic structure. The second-order construction replaces purely diffusion-like propagation by oscillatory propagation, leverages separability of cosine kernels on Cartesian product graphs, and comes with theoretical analyses of stability under graph perturbations and over-smoothing. A broader reading of the literature shows that closely related second-order generalized differential structures also appear in SODE superposition theory, operator-geometric treatments of second-order PDEs, and stochastic target formulations for fully nonlinear parabolic PDEs. This suggests that the acronym is context-dependent rather than universally fixed.

1. Core formulation on Cartesian product graphs

The explicit So-TPDEG framework is built on a Cartesian product graph of $P$ factor graphs with Laplacians $\{\mathbf{L}_p\}_{p=1}^P$. For two graphs, the Cartesian product Laplacian is written as

$$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$

and, more generally,

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$

The model is proposed as a response to limitations of first-order continuous product-graph models such as CITRUS / first-order TPDEGs. Those first-order constructions are described as low-pass filters that damp high-frequency signals, slow information propagation, and fit heterophilic or multi-scale structure less well. The second-order alternative is motivated by oscillatory propagation, with the paper explicitly stating that So-TPDEGs “do not dampen the medium to high-frequency components” in the way first-order TPDEGs do (Einizade et al., 2 Sep 2025).

The governing second-order tensorial PDE is

$\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$

The first term gives each factor graph a second-order spatial interaction through $\mathbf{L}_i^2$, while the cross term encodes interactions between different graph domains. The paper emphasizes two departures from first-order TPDEGs: the use of $\mathbf{L}_i^2$, which extends the model beyond immediate neighborhoods, and the presence of explicit cross-interactions between factor graphs, which are absent in the first-order formulation. In this sense, So-TPDEGs are not merely a higher-order smoothing mechanism; they are a coupled multidomain propagation model.

2. Spectral structure, closed-form solution, and filtering interpretation

A central result is that the second-order dynamics remain spectrally tractable because Cartesian product graphs preserve separability. If each factor Laplacian has spectral decomposition

$$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$

then the product Laplacian satisfies

$$\mathbf{V}_\diamond = \downarrow \otimes_{p=1}^{P} \mathbf{V}_p, \qquad \boldsymbol{\Lambda}_\diamond = \downarrow \oplus_{p=1}^{P} \boldsymbol{\Lambda}_p.$$

This means that eigenvectors of the product graph are Kronecker products of factor-graph eigenvectors, and the full product spectrum is assembled from factor spectra. The construction therefore avoids diagonalizing the full product graph Laplacian directly (Einizade et al., 2 Sep 2025).

Assuming zero initial velocity,

$$\left.\frac{\partial \underline{\mathbf{U}_t}}{\partial t}\right|_{t=0}=0,$$

the solution is given by the separable cosine/sine expansion

$\{\mathbf{L}_p\}_{p=1}^P$0

where

$\{\mathbf{L}_p\}_{p=1}^P$1

The paper interprets this as the continuous product-graph analogue of a wave-type propagator. The spectral mechanism is further summarized by the identity

$\{\mathbf{L}_p\}_{p=1}^P$2

obtained from

$\{\mathbf{L}_p\}_{p=1}^P$3

by taking the real part. This cosine separability is the structural reason that So-TPDEGs admit efficient filtering on multidomain graphs.

At the network level, the layerwise operator is written in compact form as

$\{\mathbf{L}_p\}_{p=1}^P$4

The equivalent tensor expression is

$\{\mathbf{L}_p\}_{p=1}^P$5

This establishes an exact equivalence between the tensorial PDE description and product-graph spectral filtering.

3. Stability, over-smoothing, and computational complexity

The graph-theoretic theory of So-TPDEGs is accompanied by perturbation and over-smoothing results. If each factor graph is perturbed as

$\{\mathbf{L}_p\}_{p=1}^P$6

then the product perturbation satisfies

$\{\mathbf{L}_p\}_{p=1}^P$7

The stability theorem gives

$\{\mathbf{L}_p\}_{p=1}^P$8

so the perturbation error grows at most linearly in time and in the size of the factorwise perturbations (Einizade et al., 2 Sep 2025).

The over-smoothing analysis is phrased through the tensorial Dirichlet energy

$\{\mathbf{L}_p\}_{p=1}^P$9

with normalized product Laplacian

$$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$0

For the layerwise output $$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$1, the paper proves

$$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$2

where

$$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$3

and $$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$4 is the square of the maximum singular value of $$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$5. The spectral factor is

$$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$6

If

$$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$7

then the energy decays exponentially to zero and over-smoothing occurs. The admissible receptive fields satisfy

$$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$8

The paper’s interpretation is that, unlike first-order TPDEGs, So-TPDEGs admit infinitely many valid oscillatory intervals for $$\mathbf{L}_1 \oplus \mathbf{L}_2 = \mathbf{L}_1 \otimes \mathbf{I}_{N_2} + \mathbf{I}_{N_1} \otimes \mathbf{L}_2,$$9, making it possible to tune the receptive field to avoid over-smoothing while preserving propagation.

The computational implications follow directly from separability. Naively diagonalizing the full product Laplacian costs

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$0

whereas using factor-graph decompositions costs

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$1

and keeping only the top $$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$2 eigenpairs per factor costs

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$3

Potential application areas mentioned or implied include spatiotemporal forecasting, sleep staging, recommender systems, brain signal processing, hyperspectral imaging, video analytics, continuum mechanics / wave-like systems, and heterophilic relational data.

4. Analytical antecedents: superposition rules, Lie systems, and characteristic geometry

A broader mathematical precursor to the notion of a second-order generalized differential structure appears in the theory of superposition rules for second-order ordinary differential equations. For a system

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$4

a superposition rule is defined as a map

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$5

such that the general solution can be written as

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$6

for any generic family of particular solutions. “Generic” is specified by an open dense set of initial data in $$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$7. The paper defines a SODE Lie system by requiring that the tangent-lifted first-order system

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$8

be a Lie system, proves that every SODE Lie system admits a superposition rule, and proves a partial converse for free superposition rules. It also extends the construction via quasi-Lie schemes, so that non-Lie second-order systems can still inherit time-dependent superposition rules after suitable time-dependent changes of variables. The worked example

$$\mathbf{L}_\diamond \vcentcolon= \downarrow\oplus_{p=1}^{P}{\mathbf{L}_p}.$$9

is shown to generate an $\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$0-dimensional Lie algebra isomorphic to $\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$1, with an explicit nonlinear superposition formula depending on four particular solutions and their derivatives. The same framework is applied to second-order Riccati equations through a quasi-Lie scheme $\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$2 with $\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$3 (Cariñena et al., 2010).

A related second-order PDE perspective is operator-geometric rather than graph-theoretic. For linear PDEs in two variables,

$\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$4

the principal part

$\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$5

is analyzed through first-order characteristic operators and their invariants. In the hyperbolic case $\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$6, the principal symbol factors using the characteristic slopes

$\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$7

leading to operators

$\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$8

An invariant $\frac{\partial^2\underline{\mathbf{U}_t}}{\partial t^2} = - \sum_{i=1}^{P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i^2} - 2 \sum_{1 \le i < j \le P} \underline{\mathbf{U}_t \times_i \mathbf{L}_i \times_j \mathbf{L}_j.$9 of a first-order operator $\mathbf{L}_i^2$0 satisfies

$\mathbf{L}_i^2$1

so $\mathbf{L}_i^2$2 is constant along characteristic curves. Characteristic coordinates

$\mathbf{L}_i^2$3

reduce the hyperbolic operator to a mixed derivative and the parabolic operator to a pure second derivative, while commutativity of characteristic operators controls whether factorization is residue-free. This literature does not use the term So-TPDEG, but it develops a second-order operator framework in which invariants, factorization, Jacobian identities, and coordinate transport determine canonical form (Pezzi, 2017).

5. Stochastic and control-theoretic second-order formulations

A distinct second-order line of development comes from stochastic target problems and their relation to fully nonlinear parabolic PDEs. On canonical path space, the controlled scalar process is defined by the Stratonovich equation

$\mathbf{L}_i^2$4

with quadratic-variation density

$\mathbf{L}_i^2$5

and covariation density

$\mathbf{L}_i^2$6

The second-order target value is

$\mathbf{L}_i^2$7

The technical innovation is to vary the reference probability so as to allow different volatility scales. This leads to a family of mutually singular measures $\mathbf{L}_i^2$8, indexed by positive definite quadratic-variation densities, and makes the problem genuinely second-order (Soner et al., 2010).

The dual representation is obtained through the convex conjugate

$\mathbf{L}_i^2$9

which produces, under each $\mathbf{L}_i^2$0, the standard BSDE

$\mathbf{L}_i^2$1

The dual value is

$\mathbf{L}_i^2$2

and the main theorem proves, under the stated assumptions, that $\mathbf{L}_i^2$3, with stronger assumptions yielding $\mathbf{L}_i^2$4. In the Markovian case the associated PDE is

$\mathbf{L}_i^2$5

and convexity in the Hessian variable yields a representation as a supremum of semilinear problems indexed by volatility matrices. In the language used in the paper’s details, this is the stochastic side of So-TPDEG-like second-order structures: a second-order control problem is reduced to a family of standard BSDEs, and in the Markovian setting to a stochastic representation of nonlinear scalar second-order parabolic equations (Soner et al., 2010).

6. Terminological scope, adjacent usages, and common confusions

The most explicit and technically developed use of the acronym So-TPDEG is the product-graph framework described above. However, nearby literature employs closely related “second-order” language in other, nonidentical senses. One example concerns total positivity of order two. For a finite-support bivariate distribution represented by a nonnegative matrix $\mathbf{L}_i^2$6, the TP2 constraint is

$\mathbf{L}_i^2$7

This is exactly the order-two total positivity condition: every $\mathbf{L}_i^2$8 minor is nonnegative. The associated maximum-likelihood fitting problem minimizes

$\mathbf{L}_i^2$9

and the paper develops a descent algorithm based on row calibration, column calibration, special-case isotonic regression via PAVA when $$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$0 or $$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$1, and local updates across column cuts. The details explicitly connect this order-two total positivity setting to the language of second-order TPDEGs insofar as the constraint is the bivariate order-2 total positivity constraint (Duembgen et al., 21 Jun 2026).

Another adjacent usage is second-order regular variation in heavy-tailed risk concentration. For identical marginals, the diversification index is

$$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$2

and the second-order framework is formulated through univariate $$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$3 and multivariate $$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$4. If

$$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$5

then tail and quantile expansions can be sharpened beyond first-order regular variation, and the paper derives convergence rates for $$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$6 as $$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$7. The details state that this literature is highly relevant if So-TPDEGs is intended to mean a second-order tail-regularity framework for heavy-tailed dependence and aggregation (Das et al., 2017).

A separate, unrelated use of “second-order” occurs in convex optimization, where $$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$8 means that a closed convex semialgebraic planar set is second-order cone representable. In that setting,

$$\mathbf{L}_p = \mathbf{V}_p \boldsymbol{\Lambda}_p \mathbf{V}_p^\top,$$9

is equivalent to representability by second-order cone constraints, not to tensorial PDEs on graphs (Scheiderer, 2020). A common misconception is therefore to treat all “second-order” acronyms as interchangeable. The literature does not support that identification. Rather, the term So-TPDEG currently has a precise meaning in product-graph continuous learning, while several neighboring theories supply mathematically related but terminologically distinct second-order structures.