An Asynchronous multi-rate Taylor method for Delay Differential Equations

Abstract: The numerical simulation of high-dimensional, multi-rate Delay Differential Equations (DDEs) is fundamentally bottlenecked by synchronous time-stepping and the dynamic memory allocation required for continuous history tracking. In this paper, we introduce the Asynchronous Adaptive Taylor Solver (AATS), an event-driven integration framework designed to overcome these high-performance computing limitations. By assigning independent local clocks to individual coordinates and advancing them using high-order Taylor polynomials generated via compile-time Automatic Differentiation, AATS restricts computational work to actively evolving sub-graphs. To eliminate the severe memory overhead endemic to traditional DDE solvers, AATS utilizes statically allocated circular buffers to store polynomial segments, achieving interpolation-free continuous dense-output evaluation with a verified zero-allocation runtime memory footprint. Alongside this software architecture, we establish a novel continuous proof of convergence for asynchronous Taylor expansions and formally prove that the framework's algorithmic complexity scales linearly (O(N)). Extensive benchmarks against state-of-the-art synchronous solvers (Julia SciML) validate these theoretical bounds. On large-scale benchmarks (upto $N = 10000$ coordinates) AATS fundamentally minimizes the constant factor of algorithmic work by avoiding redundant evaluations, delivering empirically consistent with O(N) execution scaling and significant wall-clock speedups.

• The paper introduces an asynchronous integration scheme based on high-order Taylor expansions that eliminates global synchronization.
• The method employs statically allocated circular buffers to ensure zero dynamic memory allocation and interpolation-free delay evaluations.
• Empirical and theoretical analyses demonstrate strict O(N) complexity for sparse networks with rigorous error convergence guarantees.

Asynchronous Multi-Rate Taylor Integration for Delay Differential Equations

---

Motivation and Challenges in High-Dimensional DDE Simulation

The numerical integration of Delay Differential Equations (DDEs) dominates computational modeling in systems with inherent lags, including large-scale neural networks, population modeling, and complex control systems. Simulation of high-dimensional or multi-rate DDEs suffers from two major bottlenecks: forced global synchronization and continuous dynamic memory allocation for historical state tracking. State-of-the-art solvers (e.g., Julia SciML, Python SciPy) synchronously evolve all variables on a global time grid bounded by the fastest component, incurring millions of redundant function evaluations and severe computational inefficiency, especially in sparse, heterogeneous networks. Additionally, traditional solvers dynamically manage continuous history via heap allocation and interpolation, resulting in memory fragmentation, cache misses, and latency penalties.

---

The Asynchronous Adaptive Taylor Solver (AATS): Architectural Design

AATS introduces an event-driven, asynchronous integration paradigm which decouples local temporal evolution. Each variable is assigned an independent clock, maintained via high-order Taylor polynomial expansions generated through compile-time Automatic Differentiation (AD). History is represented in statically allocated, fixed-capacity circular buffers, allowing continuous, interpolation-free state access at zero dynamic allocation cost. Mathematical operations, including delayed state evaluation, are performed directly on stored Taylor expansions, bypassing the construction and query of secondary interpolants typical of synchronous solvers.

Key Innovations:

• Temporal Decoupling via Event Scheduling: Each coordinate evolves based only on local error and dynamic history, with latency-minimal event scheduling (Radix Heap, amortized $O(1)$).
• Zero-Allocation Memory Architecture: Static ring buffers store all Taylor segments, ensuring bounded, cache-local memory usage and avoiding heap overhead.
• Interpolation-Free Delay Evaluation: Dense-output is achieved analytically by evaluating high-order Taylor expansions, eliminating both interpolation and associated overhead.
• Rigorous Complexity and Stability Proofs: Algorithmic complexity is proven strictly $O(N)$ for sparse networks. Rigorous continuous convergence and stability bounds guarantee global error scaling as $O(\epsilon^{K/(K+1)})$ (where $\epsilon$ is user-defined local tolerance, $K$ is Taylor degree).

---

Algorithmic Complexity and Mathematical Analysis

AATS operates by processing coordinate-wise asynchronous events. For each update, the solver advances the active coordinate via Taylor expansion, projects delayed states for dependent variables using binomial time-alignment, and reschedules events conditioned on local truncation error. Crucially, all computations are performed at the granularity dictated by localized dynamics.

• Time Complexity: Per event, complexity is $O(E_v K^2)$ (where $E_v$ is local in-degree), strictly independent of global system size $N$.
• Global Work: For sparse networks, total computational cost over the integration interval is $O(N\, \epsilon^{-K/(K+1)})$, breaking the synchronous global worst-case scaling.
• Memory Footprint: Strictly bounded by $O(N M K)$ (with $O(N)$0 as buffer segment capacity), completely independent of integration steps taken, achieving a verified zero-allocation runtime profile.
• Empirical Validation: Benchmarks on large-scale neural and chaotic systems (up to $O(N)$1), including Continuous-Time Recurrent Neural Networks (CTRNN), heat diffusion DPDEs, Ikeda DDEs, and state-dependent Mackey-Glass models, exhibit superior wall-clock speedups and strict empirical $O(N)$2 scaling for sparse networks.

---

Convergence, Stability, and Error Propagation

The paper provides a novel continuous convergence proof for the asynchronous Taylor expansion scheme, leveraging continuous induction and Gronwall-type inequalities. Unlike traditional discrete nodal analyses, AATS establishes stability and convergence by analyzing the piecewise-smooth continuous extension across asynchronous event times.

• Zero-Stability and Error Propagation: Errors introduced during polynomial expansion are shown to decay due to delay-differential interaction, not propagate or amplify across the network.
• Convergence Order: For adaptive local tolerances, global error satisfies $O(N)$3 scaling, reflecting the intrinsic order of the Taylor method and confirming error estimators tightly bound actual error.
• Functional Equivalence: Empirical and theoretical analysis confirms no global phase drift relative to synchronous solvers—precise tracking of state trajectories is maintained even under extreme multi-rate and chaotic regimes.

---

Numerical Results and Quantitative Analysis

Significant quantitative evidence supports all theoretical claims. AATS achieves strict linear runtime scaling and minimizes redundant computation for sparse networks:

• Execution Speedup: Across benchmarks, AATS reduces execution time by orders of magnitude compared to Julia SciML DelayDiffEq, particularly where network sparsity and timescale heterogeneity are pronounced.
• Memory Consumption: Dynamic heap allocation remains flat (near zero) as N increases, validating static allocation claims, while synchronous solvers incur exponential memory growth.
• Dense Network Limitations: For maximally dense systems, queue overhead introduces computational penalties; synchronous solvers regain competitiveness due to optimized contiguous memory layouts and hardware vectorization.

---

Relation to Prior Work

AATS distinguishes itself from multirate, QSS, and synchronous DDE integration frameworks by:

• Eliminating Global Synchronization: Multirate and QSS methods retain macro-step synchronization or quantization-driven event barriers; AATS operates in a fully continuous, asynchronous domain with event generation governed purely by adaptive error control.
• Unified Dense-Output and History via Taylor Expansion: No secondary interpolant construction or root-finding is needed for delay evaluation; polynomial segments inherently provide dense continuous state access.
• Native Support for High-Dimensional, Sparse DDEs: The architecture exploits sparsity, bypassing dense memory and computational bottlenecks in conventional solvers.

---

Implications and Prospects for Future AI and Numerical Methods

AATS fundamentally transforms simulation paradigms for high-dimensional, multi-rate DDEs by removing synchronization and memory allocation barriers. The architectural choices—zero-allocation memory, event-driven updates, polynomial-based dense-output—advance both theoretical and practical numerical integration capabilities.

Implications:

• Enables efficient simulation of massive neural, biological, and control networks where heterogeneous dynamics render global synchronization infeasible.
• Facilitates deployment in resource-constrained environments, yielding predictable, minimal memory utilization without garbage collection overhead.
• Opens new theoretical avenues in continuous-time, asynchronous numerical analysis, including rigorous convergence proofs for event-driven methods.

Future Developments:

• Integration of localized SIMD-vectorized AD pipelines to recapture hardware efficiency for dense networks.
• Extension to stiff and stochastic DDEs, broadening applicability to more complex, real-world systems.
• Parallelization of the asynchronous event queue for concurrent temporal updates, leveraging multicore architectures.

---

Conclusion

The Asynchronous Adaptive Taylor Solver (AATS) presents a comprehensive integration framework for Delay Differential Equations, effectively addressing the key bottlenecks of synchrony and dynamic memory allocation. By leveraging compile-time AD, static memory management, and event-driven architecture, AATS achieves mathematically validated linear complexity, empirical speedups, and strict memory bounds in sparse, large-scale networks. The rigorous continuous convergence analysis ensures high-order accuracy and stability, making AATS a potent tool for the simulation of modern, complex dynamical systems and laying foundations for further innovation in high-performance numerical computation (2606.21044).