FT-JNF: Frequency-Time Joint Nonlinear Filter

• FT-JNF is a nonlinear signal processing framework that jointly filters across time and frequency with adaptive and non-separable features.
• It employs warped filter banks, iterative algorithms, and scattering transforms to achieve precise localization and robust performance.
• Its versatile applications in audio processing, wireless communications, quantum measurements, and graph signal processing offer enhanced resolution and reduced complexity.

A Frequency-Time Joint Non-linear Filter (FT-JNF) is a signal processing construct designed to enable joint, often adaptive and/or non-linear, filtering operations along both the time and frequency axes. FT-JNFs are instantiated in various forms to address fundamental challenges in time-frequency analysis, adaptive signal decomposition, perceptually-aligned audio processing, nonlinear system identification, quantum measurement, graph signal processing, and wireless communications. Their unifying principle is the explicit non-separability and nonlinearity of filtering across both frequency and time, as opposed to traditional separable, linear, or one-dimensional approaches.

1. Theoretical Foundations and Warped Filter Bank Constructs

Warped filter banks provide a foundational, frame-theoretic construction for FT-JNF, enabling precise adaptation of frequency resolution according to an arbitrary nonlinear frequency scale (Holighaus et al., 2014). Specifically, the filter bank is formed by translating a prototype filter $\theta$ in the frequency domain after application of a strictly increasing, continuously differentiable warping function $\Phi : D \subset \mathbb{R} \to \mathbb{R}$: $g_m(\xi) = \sqrt{a_m}\; \theta(\Phi(\xi) - m)$ where $a_m$ are decimation factors set relative to the local bandwidth via

$$a_m = \alpha_0 / w(m), \quad w(\tau) = (\Phi^{-1})'(\tau)$$

This construction achieves a nonuniform frequency tiling; for example, logarithmic warping $\Phi(\xi) = \log(\xi)$ produces wavelet-type resolution, and ERB warping yields filters aligned to perceptual auditory scales. The frame property is established when

$$0 < A \leq \sum_m |(T_m\theta)(\Phi(\xi))|^2 \leq B < \infty$$

guaranteeing stable synthesis even after nonlinear manipulation of filter coefficients.

FT-JNF thereby enables localization in both linear time and warped (nonlinear) frequency domains, permitting subsequent nonlinear processing in each channel. Applications include auditory filterbanks (ERBlet frames) and any domain where matching the filterbank structure to signal- or perception-specific frequency scales is critical.

2. Adaptive and Iterative Nonlinear Time-Frequency Decomposition

Adaptive local iterative filtering (ALIF), and more generally the iterative filtering (IF) framework, realize FT-JNF concepts through fully adaptive, nonlinear time-frequency decomposition (Cicone et al., 2014). In the IF approach, the local average is computed by convolution with a compact, smooth, symmetric filter $w$, producing successive fluctuation signals: $$\mathcal{L}(f)(x) = \int_{-l}^{l} f(x+t)w(t)\,dt,\qquad \mathcal{S}(f) = f - \mathcal{L}(f)$$ where for convergence, $\left|1-\hat{w}(\xi)\right|<1$ for all nonzero $\xi$. ALIF extends this by computing a spatially varying mask length $l(x)$, adapting to local signal scales (e.g., local extremum spacing), and thus effecting a nonlinear, time-frequency-localized decomposition even when subcomponents have overlapping instantaneous frequency ranges.

FP (Fokker–Planck) filters are constructed as steady-state solutions to certain parabolic PDEs, ensuring compact support and requisite smoothness for filter stability. Additionally, the paper introduces a local instantaneous frequency definition based solely on local polar coordinates of the signal and its derivative, improving robustness to nonstationary phenomena compared to global Hilbert-based estimators. Empirical results establish robustness to noise and the capacity to separate physically meaningful oscillatory modes in both synthetic and real-world settings.

3. Joint Time-Frequency Scattering and Non-separable Wavelet Networks

The joint time-frequency scattering (JTFS) architecture represents a deep convolutional, non-learned realization of FT-JNF by cascading wavelet transforms and modulus nonlinearities in both time and log-frequency dimensions (Andén et al., 2015, Andén et al., 2018). The first layer computes the scalogram

$x_1(t, \log\lambda_1) = |x * \psi_{\lambda_1}(t)|$

followed by a second layer comprising a 2D convolution with separable wavelets in time and log-frequency: $$S_2 x(t, \log\lambda_1, \log\lambda_2) = \left|x_1 * \Psi_{\lambda_2}(t, \log\lambda_1)\right| * \phi_T(t)$$ filtering across both dimensions in a non-linear and non-separable manner.

JTFS discards the assumption of separable subband processing, rendering it sensitive to complex time-frequency structures such as time-varying filters, frequency modulation, and cross-frequency dependencies. Compared to MFCCs and time-only scattering, JTFS improves discriminability, as verified by reduced phone error rates (TIMIT) and enhanced audio texture synthesis fidelity. The architecture is parameter-free except for the final classifier, enabling robustness and efficiency on large-scale classification and generation tasks.

4. Adaptive Nonlinear Time-Frequency Transforms via Focus Functions

A further generalization introduces non-linear adaptive time-frequency transforms parameterized by focus functions, which locally control the analysis window scale in time or frequency as a signal-dependent function (Warion et al., 2023). The time-focused transform, for example, defines time-localized atoms via

$$h_{t,\omega,\sigma_f^{(\tau)}}(x) = \sqrt{\gamma'(\omega) \sigma_f^{(\tau)}(t)}\, e^{2\pi i \gamma(\omega)x} h\left(\sigma_f^{(\tau)}(t)(x-t)\right)$$

with $\sigma_f^{(\tau)}(t)$ extracted from entropy or norm measures on reference spectrograms. Thus, time-frequency resolution can be adaptively squeezed in transient or complex regions, achieving analysis tuned to local signal features. Frame-theoretic energy bounds are established, but the non-linearity breaks standard invertibility, presenting open challenges for perfect reconstruction—an important consideration for FT-JNF deployments requiring synthesis.

5. Block-Based, Frequency-Domain, and Low-complexity Implementations

In high-dimensional, real-time, or resource-constrained applications, FT-JNF is realized through block-based, frequency-domain expansions combined with time-domain nonlinearities (Yu et al., 2022, Ohlenbusch et al., 6 Sep 2024, Metzger et al., 25 Jul 2025). The Frequency domain Exponential Functional Link Network (FDEFLN) approach decomposes input signals into blocks, expands via nonlinear basis functions (e.g., exponentials), applies FFT to enable efficient convolution and adaptation via overlap-save, and updates weights in the frequency domain, drastically lowering computational complexity.

For hearable devices, low-complexity FT-JNF variants process the real and imaginary parts of multi-microphone STFT coefficients with frequency-direction and time-direction LSTM layers. Output complex masks are combined with the input signals for own-voice reconstruction (Ohlenbusch et al., 6 Sep 2024). Similar FT-JNF neural architectures applied to multi-microphone speech enhancement leverage knowledge distillation methods to compress large teacher models into small, efficient student models without substantial loss in objective quality metrics (e.g., PESQ) (Metzger et al., 25 Jul 2025).

6. FT-JNF in Graph Signal Processing and Wireless Communications

FT-JNF concepts extend to signals residing on graphs that evolve in time, necessitating transformations and filtering in the joint temporal-graph spectral domain (Loukas et al., 2016). The joint Fourier transform is constructed as

$$\text{JFT}\{X; \mathcal{G}\} = \mathcal{G} X T^\top$$

where $\mathcal{G}$ and $T$ are the orthonormal graph and temporal Fourier bases, respectively. FT-JNF in this context supports the design of distributed, joint FIR filters that act on both frequency and time, with applications in distributed interference cancellation and optimal Wiener filtering, realized via spectral polynomial approximations for linear complexity.

In advanced wireless communication systems with doubly-selective (time- and frequency-varying) channels, iterative and joint estimation schemes for channel response and data detection leverage FTN signaling with superimposed pilots (Keykhosravi et al., 21 Mar 2025). Here, joint frequency-time estimation suppresses inter-symbol interference and tracks channel properties nonlinearly in iterative turbo equalization frameworks, demonstrating superior MSE and BER performance especially under rapid channel fading.

7. FT-JNF in Quantum Measurement and Information Protocols

FT-JNF is instantiated in quantum measurement settings via joint time-frequency nonlinear filtering of entangled photon states (Liu et al., 2019). In sum-frequency generation (SFG) processes in nonlinear optical media, joint detection of the frequency sum and time difference of photon pairs realizes non-destructive, joint filtering of quantum correlations. This approach underpins protocols for continuous-variable superdense coding (TFE SDC) and quantum illumination (TFE QI), wherein the joint SFG measurement facilitates simultaneous, high-precision estimation of both temporal and spectral degrees of freedom, enhancing information capacity and metrological sensitivity.

Summary Table: FT-JNF Principles Across Domains

FT-JNF Domain	Mechanism/Framework	Key Properties/Applications
Warped Filter Banks	Nonlinear frequency warping, frame theory	Perceptual filtering, invertible decomposition, channelwise time-domain processing
Adaptive Decomposition	IF/ALIF, local masks, FP filters	Robust, nonlinear, localized decomposition, adaptive resolution, instantaneous freq.
Wavelet Networks	2D wavelet transform, modulus nonlinearity	Non-separable time-frequency analysis, invariance, audio classification
Quantum Measurement	Nonlinear joint measurement via SFG	Joint frequency-time entanglement, superdense coding, quantum illumination
Block/FFT Implementations	FFT-based filtering after time-domain nonlinear expansion	Low-complexity, real-time, speech enhancement, system identification
Graph/Temporal Filtering	Joint graph-temporal Fourier transforms, polynomial filters	Distributed, optimal Wiener/joint filtering
Wireless Channels	Iterative, joint frequency-time estimation/detection	Channel estimation in doubly-selective, high-SE wireless systems

FT-JNF thus constitutes a broad, unifying paradigm wherein non-linear, adaptive, and joint filtering in both the frequency and time domains is the mechanism for achieving high resolution, adaptability, robust performance, and/or low complexity across a spectrum of modern signal processing, quantum, and communications applications.