TNODEV: Toolbox for Neural ODE Verification

Abstract: Neural ordinary differential equations (neural ODE) have started to appear in safety critical settings such as continuous-time controllers for cyber-physical systems and classifiers integrated into automated decision pipelines, raising the question of whether their behavior can be formally verified. Existing tools dedicated to neural ODE provide only a single reachability call without iterative input set refinement, limiting the precision of their verdicts to whatever one reachability call can deliver. We present TNODEV, the first sound formal verifier for neural ODE that integrates a falsification checker, a fast interval-based reachability backend based on continuous-time mixed monotonicity, a verification and refinement loop with three input-set splitting heuristics, and a parallel scheduler in a single end-to-end pipeline. TNODEV supports safe-set inclusion verification on pure neural ODE, neural ODE in closed loop with a neural network controller and general neural ODE (GNODE), with the safe set specified either as an interval or as the half-space intersection induced by a target classification label. We evaluate TNODEV on a range of benchmarks across safe-set inclusion and classification-robustness properties, including a direct reachability comparison against NNV~2.0 and CORA and a verification comparison against NNV2.0 on MNIST general neural ODE classifiers.

• The paper introduces TNODEV, a toolbox that integrates a falsification module, CTMM-based reachability, and iterative input set refinement for neural ODE verification.
• It employs three splitting heuristics (naive, MSIR, ING) to optimally balance computational cost and approximation tightness, achieving significant speedups in high-dimensional benchmarks.
• Empirical evaluation demonstrates 7x–83x faster verification times compared to contemporary tools, though with trade-offs in over-approximation accuracy for complex classification tasks.

TNODEV: A Toolbox for Neural ODE Verification

Motivation and Context

Neural ODEs (NODEs) parametrize dynamics by neural networks, embedding the model into continuous-time flows and enabling applications in time series modeling, controlled dynamical systems, and continuous-depth deep learning. As NODEs are increasingly integrated into safety-critical applications such as automated control and classification pipelines, the demand for sound, automated formal verification tools has intensified. Existing verification methods for NODEs largely focus on a single, non-iterative reachability analysis, generating over-approximations of the reachable set and relying on the tightness of this single step. This architecture, however, fundamentally limits the certainty and minimality of the verification verdicts, especially for systems with complex, multi-dimensional input spaces where tightening the over-approximation is computationally unfeasible via direct methods alone. TNODEV directly addresses this gap by introducing an integrated, iterative verification framework for NODEs.

TNODEV Architecture

TNODEV provides a sound and modular verification framework for NODEs, general NODE (GNODE), and hybrid closed-loop systems with neural controllers. The pipeline involves four main components: (1) a falsification module, (2) a reachability backend based on continuous-time mixed monotonicity (CTMM), (3) a parallelized, iterative input set refinement loop with configurable heuristics, and (4) a flexible specification interface supporting interval and classification-safe sets.

The end-to-end workflow is as follows: (i) TNODEV initiates with a lightweight falsification phase where the algorithm checks for concrete counter-examples in the initial set by direct simulation; (ii) if falsification fails, it enters the refinement loop, computing over-approximations for subsets in a work-stealing parallel regime until either SAFE, FALSIFIED, or UNKNOWN is returned based on queue exhaustion, discovery of a violating trajectory, or budget/timeout criteria, respectively.

Figure 1: TNODEV verifier architecture, showing the falsification stage, reachability/refinement pipeline, and the cell-splitting decision loop.

Reachability Computation via CTMM

The reachability engine is predicated on CTMM, which embeds the original neural ODE into a monotone extension and propagates interval bounds on solutions efficiently. The procedure involves three main steps: (a) over-approximation of the reachable tube using external reachability tools to bound the Jacobian, (b) construction of a decomposition function using sign-stabilized interval Jacobians, and (c) one-shot simulation of the embedded $2n$-dimensional system to infer interval over-approximations for the reachable set. This method is computationally tractable, producing tight enclosures, particularly suited for iterative refinement.

Input Set Refinement and Heuristics

When direct reachability is inconclusive, TNODEV recursively partitions the initial set, pursuing resolution of local over-approximation effects by focusing the reachability computation on successively smaller subsets. Three splitting heuristics are provided:

• Naive: splits on the widest coordinate.
• MSIR (Most-Sensitive-Input-Radius): splits on dimensions weighted by interval Jacobian magnitudes, leveraging analytic gradient bounds.
• ING (Interval-Norm-Gradient): empirically estimates sensitivity of over-approximation width to each input coordinate by perturbed reachability calls.

Quantitative evaluation demonstrates that MSIR provides an optimal trade-off between computational cost and iteration count, dramatically reducing verification time and number of sub-cells in high-dimensional benchmarks compared to naive and ING heuristics.

Figure 2: TNODEV verification results on the Spiral~2D benchmark. Colored rectangles illustrate the initial set, a single-call over-approximation, the safe set, verified subsets, and trajectory samples.

Empirical Evaluation

Reachability Comparison

Comparative benchmarks against NNV 2.0 and CORA on the Spiral~2D (nonlinear) and FPA~5D tasks show TNODEV’s CTMM reachability backend yields $14\times$ faster (Spiral) and $7\times$ faster (FPA) execution than NNV 2.0, and outpaces CORA by $83\times$ and $39\times$ respectively, at the cost of wider (interval) over-approximations for non-axis-aligned reachable sets. This speed-tightness trade-off is pivotal: TNODEV’s architecture mitigates looseness by iterative refinement rather than reliance on a monolithic reachability call.

Safe-Set Inclusion Verification

On a suite of pure and hybrid NODE control benchmarks (dimensions $2$–$12$), TNODEV achieves verified SAFE verdicts across the majority of test cases, with MSIR heuristic universally outperforming naive and ING in time/iteration cost. Notably, for higher-dimensional systems such as 12D Cartpole, MSIR reduces verification from $2037$ to $63$ iterations and cuts computation time by an order of magnitude.

Figure 3: TNODEV linear spiral~2D verification via naive and MSIR refinement heuristics; verified subsets efficiently tile the input.

Figure 4: TNODEV nonlinear spiral~2D benchmark with naive, MSIR, and ING heuristics, highlighting the effect of heuristic choice on subset decomposition.

(Figure 5, 6, 7)

Figure 5: TNODEV FPA~5D benchmark under naive refinement.

Figure 6: FPA~5D under MSIR refinement.

Figure 7: FPA~5D under ING refinement; MSIR demonstrates more efficient set coverage.

Robustness Verification of Classification GNODEs

Verification of adversarial robustness on MNIST GNODE classifiers reveals the limitations of interval over-approximation for high-dimensional, linear ODE blocks: while TNODEV is $5\times$–$14\times$0 faster per image than NNV 2.0, it only verifies $14\times$1–$14\times$2 of $14\times$3 images at an $14\times$4 radius where NNV 2.0 (which propagates predicate-constrained Star sets) verifies all but one. The performance gap is entirely due to interval hull expansion in the ODE block, consistent with the trade-off observed in reachability analysis—the interval box over-approximation is not tight in the presence of non-axis-aligned affine flows.

Limitations and Future Directions

The principal limitation of TNODEV is rooted in the axis-aligned box representation used in interval reachability. For systems with significant alignment between the reachable set geometry and the axes, this is not problematic; however, in scenarios where the reachable set is polyhedral (e.g., linear ODEs post-convolutional encoder), the wrapping effect of intervals can lose critical precision, decreasing the verification rate for classification tasks as perturbation size grows. TNODEV’s modular architecture, however, is expressly designed to admit richer set representations (zonotopes, polynomial zonotopes, Star sets) as plug-in backends, which would close the speed-tightness gap and potentially increase verification coverage for complex networks and larger perturbation regimes.

Improvement in refinement heuristics is another vector for future research, particularly the development of robust, computationally efficient gradient-based or mixed heuristics that can adaptively select splitting dimensions by context rather than fixed analytic rules. Full support for general GNODE layers, without reliance on third-party reachability tools for discrete layers, would extend the applicability of TNODEV to a broader class of deep learning models.

Conclusion

TNODEV constitutes the first end-to-end sound verifier for neural ODEs to incorporate a falsification module, efficient CTMM-based reachability, iterative input set refinement, and parallelization into a unified, extensible architecture. Empirically, it demonstrates substantial advantages in verification efficiency and scalability, specifically when combined with the MSIR splitting heuristic, and exposes the fundamental speed-tightness trade-offs inherent in interval-based over-approximation. While interval propagation limits tightness compared to polyhedral abstractions, especially in high-dimensional classification settings, TNODEV sets a foundation for future verification toolchains in continuous-time neural systems and suggests a clear path toward integrating richer reachability representations.

---

References:

Sayed et al., "TNODEV: Toolbox for Neural ODE Verification" (2606.16567).