Audio Effect Estimation with DNN-Based Prediction and Search Algorithm

Abstract: Audio effects play an essential role in sound design. This research addresses the task of audio effect estimation, which aims to estimate the configuration of applied effects from a wet signal. Existing approaches to this problem can be categorized into predictive approaches, which use models pre-trained in a data-driven manner, and search-based approaches, which are based on wet signal reconstruction. In this study, we propose a novel approach that integrates these approaches: first, DNNs predict the dry signal and effect configuration, and then a search is performed based on wet signal reconstruction using these predictions. By estimating the dry signal in the prediction stage, it becomes possible to complement or improve the predictions using reconstruction similarity as an objective function. The experimental evaluation showed that methods based on the proposed approach outperformed the method solely based on the predictive approach. Furthermore, the findings suggest that the task division of predicting the effect type combination followed by the search-based estimation of order and parameters was the most effective across various metrics.

• The paper introduces a two-stage system that first predicts the dry signal and unordered effect types using a DNN and then refines order and parameters through search optimization.
• It employs a hybrid U-Net-based architecture with cross-domain attention and leverages algorithms like CMA-ES and TPE for effective parameter estimation.
• Experimental results show high SI-SDR, Macro F1, and low MR-STFT values, demonstrating substantial improvements in wet signal reconstruction over baseline methods.

Integrative DNN-Based Prediction and Search Algorithms for Audio Effect Estimation

Overview

The paper "Audio Effect Estimation with DNN-Based Prediction and Search Algorithm" (2604.22276) addresses the complex inverse problem of inferring the configuration of audio effects—both their type and parameters—applied to audio signals (commonly referred to as "wet" signals). The proposed method combines deep neural network (DNN)-based predictive models with optimization-based search strategies that leverage wet signal reconstruction, yielding improved estimation accuracy over existing approaches that rely solely on either predictive or search-based strategies.

Figure 1: A conceptual overview of the two-stage audio effect estimation process, where a DNN provides initial predictions refined by a subsequent search algorithm.

Methodology

Problem Formulation

The central task is, given a wet signal $\bm{x}_N$—an audio signal processed by a chain of $N$ effects—to estimate the original dry signal $\bm{x}_0$, the sequence of effect types $C = (c_1, ..., c_N)$, and their associated parameter vectors $P = (\bm{p}_1, ..., \bm{p}_N)$. The mapping from dry signal through an effect chain is denoted as:

$\bm{x}_N = F_{C, P}(\bm{x}_0)$

Estimating $(\hat{C}, \hat{P}, \hat{\bm{x}}_0)$ from $\bm{x}_N$ is ill-posed due to the high dimensionality and potential non-invertibility of some effects.

Two-Stage Approach: Prediction and Search

The proposed framework is a two-stage pipeline:

1. DNN-Based Prediction: A neural model predicts the dry signal and (all or subset of) the effect configuration.
2. Search-Based Refinement: The predictions are refined by black-box optimization, maximizing the similarity between the original wet signal and a reconstructed wet signal generated using the predicted configuration applied to the predicted dry signal.

This joint approach leverages the generalization and expressiveness of DNNs while addressing their potential misestimations through targeted local search in the effect parameter space.

Figure 2: The prediction model architecture employs an effect remover with a U-Net-based hybrid time-frequency domain backbone and an effect configuration estimator branched at the encoder bottleneck.

Prediction Model Architecture

The DNN architecture follows the SunAFXiNet paradigm. The model comprises:

• A hybrid U-Net-based effect remover operating in both time and frequency domains.
• A cross-domain encoder with five alternating self- and cross-attention layers.
• An effect configuration estimator branching at the encoder's midpoint, performing type conditional estimation and parameter regression.

Effect-type conditioning is executed by broadcasting predicted type embeddings across time steps and merging them with latent activations via channel-wise concatenation and a point-wise convolution.

Figure 3: Conditioning-by-type mechanism within the cross-domain encoder allows effect-specific representation processing.

Task Division Strategies

The study evaluates three settings for partitioning the estimation task between the prediction and search stages:

• Dry-Type-Direct: The DNN predicts the unordered set of types and the dry signal, leaving order and parameters for the search stage.
• Bypass-Type-Iter: The DNN iteratively predicts the type of the last-applied effect and the bypass signal. The predicted types yield an ordered sequence.
• Bypass-Config-Iter: As above but also predicts effect parameters at each step.

Search Algorithms

Given the often black-box nature of practical audio effects, CMA-ES is employed for continuous parameter optimization. For one-dimensional search spaces, TPE is used.

Experimental Design

Dataset

The dataset comprises 2231 10-second dry guitar segments from multiple established guitar datasets. For each segment, all possible single and multi-effect (up to three, non-repeating types) chains formed from Chorus, Distortion, and Reverb were synthesized with parameter sampling across defined ranges, yielding a comprehensive supervised pairing of $(\bm{x}_0, C, P, \bm{x}_N)$ combinations.

Evaluation Metrics

• Macro $F_1$ and Exact Match Accuracy (EMA): For type sequence estimation.
• SI-SDR and MR-STFT: For dry signal estimation and wet signal reconstruction.
• Parameter MAE: For regression quality of effect parameters.

Training and Optimization

DNNs are trained in two stages, first on effect removal with MAE and MR-STFT losses, then freezing the remover while training effect configuration estimation with classification and regression losses.

Results

Effect Type Sequence Estimation

The Dry-Type-Direct prediction followed by search attained a Macro $N$0 of 0.958, Levenshtein Distance (LD) of 0.313, and EMA of 0.774—consistently outperforming iteration-based models. Notably, this strategy separates type set prediction from ordering and parameterization, with the latter resolved via search.

Effect Parameter Estimation

Parameter regression in Bypass-Config-Iter achieved a MAE of 0.0885, but search refinement further improved reconstruction.

Audio Effect Removal and Wet Signal Reconstruction

All search-augmented methods surpassed pure prediction baselines in wet signal reconstruction. Dry-Type-Direct plus search achieved the highest SI-SDR (23.07) and lowest MR-STFT (0.340) when reconstructing wet signals from ground-truth dry inputs, highlighting strong parameter and order estimation efficacy.

Implications and Future Directions

This integrative method provides a scalable framework for inverse estimation of effect chains, positioning it for applications in automatic transcription of sound design processes, audio forensics, and assists in creative reverse engineering of studio productions. The strong results for order and parameter estimation are significant, especially given the combinatorial nature and non-differentiable landscapes of real-world effect chains.

The approach's modularity suggests extensibility to more diverse effect sets, complex chain topologies (such as branched or feedback structures), and possibly differentiable approximation of notorious black-box effects. Addressing optimization difficulties for specific parameters (e.g., Chorus rate) by combining differentiable surrogates or improved similarity measures is anticipated.

Furthermore, the methodology may inspire analogous frameworks for other unconstrained inverse problems in audio (e.g., source separation, room impulse response estimation) and, more generally, for hybrid neural/optimization pipelines in multimodal signal inversion and content analysis in AI.

Conclusion

The study demonstrates that combining DNN-based predictions with search-based refinement for audio effect estimation provides substantial improvements over prediction or search in isolation. The most effective configuration predicts the unordered effect type set and dry signal, then resolves order and parameterization using similarity-based search. While current scope is limited to a restricted effect palette and linear effect chains, the approach is highly generalizable. Future work should address scaling to more complex effect types, non-linear and longer chains, and more challenging variable optimization landscapes for broader applicability.