Compiling Differentiable Audio Graphs to Real-Time DSP

Abstract: Differentiable audio processors are habitually designed and optimised in machine-learning frameworks, but deploying them as real-time audio effects still often requires non-automatic implementation in a dedicated digital signal processing language. The translation is error-prone, demands an onerous verification process, and detaches research prototypes from usable production tools. That being so, we present ADAC, a compiler that lowers a trained model to a framework-agnostic intermediate representation and emits efficient FAUST code whose impulse response matches the source model to within floating-point arithmetic noise, direct paths included. The optimisation loop is made audible by replacing the model in a running plugin after each gradient step. The exported processor carries a small set of macro-controls that leave its stability intact. A stability certificate computed from the shipped parameters is checked before the plugin is built. At the demonstration, a feedback delay network is trained and exported to a working plugin.

• The paper presents ADAC, a compiler that automates converting differentiable audio graphs into verified, production-grade real-time DSP code with sample-level equivalence.
• It extracts model topology and parameters into a JSON-based intermediate representation, enabling extension to new operator types and simplifying deployment.
• The system supports real-time live training and deployment with sub-10ms latency, robust macro-controls, and formal stability certification to ensure safety.

Compiling Differentiable Audio Graphs to Real-Time DSP: ADAC

Introduction and Motivation

The paper "Compiling Differentiable Audio Graphs to Real-Time DSP" (2606.21277) presents ADAC, an automatic compiler for deploying differentiable audio processor models, developed in ML frameworks, into production-ready real-time DSP code. The core motivation addresses a persistent translational bottleneck: ML-trained audio DSPs must typically be reimplemented manually in dedicated DSP languages (e.g., FAUST), introducing verification challenges and separating research prototypes from usable real-time systems. ADAC eliminates this barrier by providing an automated, framework-agnostic toolchain bridging ML-centric design with DSP-centered deployment.

System Architecture and Intermediate Representation

ADAC processes differentiable audio models (demonstrated with PyTorch/FLAMO), extracting both topology and parameters into a JSON-based intermediate representation. This IR encodes series, parallel, and recursive compositions, precisely capturing LTI topologies, parameter sets, and functional operators (e.g., mixing, gains, delay structures). Parameters that are structurally encoded in training (e.g., skew-symmetric matrices for orthogonality or vector parametrizations for Householder reflections) are unwrapped into their effective forms for DSP emission, yet raw weights are preserved to enable source-level reversibility.

This separation between framework-specific graphs and the IR decouples source traversal from target emission, enabling extensibility to novel operator types and simplifying adaptation to other backends beyond FAUST.

Code Generation and Sample-Level Equivalence

Generation of FAUST code involves direct mapping from structural nodes to FAUST primitives: series to composition (:), parallel to split/merge, recursion to FAUST's recursion operator (~), and leaves to FAUST atomic DSPs (delays, gains, filters). Delay-line treatment accounts for FAUST's implicit per-recursion delay, ensuring exact alignment of the synthesized output with original model semantics. For integer-sample delays, the emitted FAUST implementation reproduces the training model's impulse response up to floating-point noise, with sample-level departures below $7 \times 10^{-5}$ of peak—an error solely attributable to single-precision arithmetic.

Correctness is underpinned by signal flow analysis: all quantities $$(\mathbf{A}, \mathbf{m}, \mathbf{B}, \mathbf{C}, D)$$ defining the FDN or general audio graph are precisely preserved through serialization and code emission, guaranteeing transfer-function equivalence within floating-point tolerance.

Macro-Controls, Stability Certification, and Workflow Integration

Exported DSP plugins expose a reduced, semantically meaningful control surface—macro-controls like reverberation time, dry/wet mix, and pre-delay—rather than raw learned parameters that would risk instability and non-robustness. The reverberation-time parameterization follows Jot's formulation, mapping a single control to homogeneous decay rates through explicit gain calibration per delay line. Empirically, compiled plugins exhibit reverberation times that match the control target to within $0.0001$ seconds, emphasizing numerical robustness.

ADAC integrates formal stability certification based on small-gain criteria, checking the product of per-loop singular values across the frequency band ($$\sup_\omega \prod_k \sigma_{\max}(E_k(e^{j\omega})) < 1$$). Certification operates post-quantization, reflecting the actual parameterization as shipped (after single-precision casting and decimal emission). Marginal instability from floating-point artifacts is detected, and builds are blocked for unproven/unstable parameter sets unless explicitly overridden. The full verification result is exported alongside the DSP code.

Real-Time Live Training and Deployment

A distinguishing feature is the audibilization of the training process in real time. After each gradient step, a callback replaces the FAUST plugin code in a live host—enabling direct, seamless monitoring of model evolution. End-to-end toolchain latency (model to plugin reload) is sub-10 ms on commodity hardware (Apple M2), supporting tight feedback and immediate deployment. Plugins are generated with full automation (code, certification, host project, installation) covering all standard plugin formats (VST3, AU, CLAP), and macro-control addresses remain stable across reloads for uninterrupted automation.

Generality, Limitations, and Future Directions

While the case study focuses on feedback delay networks (FDNs), the abstraction is generic for arbitrary LTI audio graphs composed from serial/parallel/recursive operators and parameterized leaves. The same infrastructure is shown to apply to complex SDNs for room acoustics, and extension to time-varying or nonlinear modules is anticipated, contingent on supportive front-end frameworks (e.g., dasp, differentiable all-pole filters [diffapf]). The primary limitations are (1) expressivity is bounded by the operator and parameter coverage of the IR and emitter, and (2) formal equivalence and stability guarantees presently pertain only to the LTI setting.

Further development entails broadening the repertoire of differentiable audio blocks and stabilizing the IR format to encourage multi-framework interoperability. Extension of stability and equivalence verification to broader nonlinear and time-varying classes remains a primary research direction.

Conclusion

ADAC offers an expressive, automated, and formally verified route from differentiable audio ML models to efficient, production-grade real-time DSP code. It eliminates the ML-to-DSP translation barrier, provides robust macro control and safety guarantees, supports real-time deployment and training, and generalizes across LTI audio processor topologies. These properties position it as a clean bridge from optimization-centric research workflows to practical audio DSP engineering and point toward extensible architectures for differentiable DSP deployment in both research and industry contexts.