SliCQ: Streaming Invertible CQT for Audio

• SliCQ is a streaming, invertible time-frequency representation that employs a slice-wise constant-Q NSGT to facilitate real-time audio analysis.
• It dynamically adapts time and frequency resolution, offering fine frequency detail at low frequencies and precise timing at high frequencies, aligning with human auditory perception.
• The framework guarantees perfect reconstruction using dual-window techniques and overlap-add, making it ideal for applications like source separation, pitch shifting, and effect processing.

The Sliced Constant-Q Transform (sliCQ) is an invertible, streaming-capable time-frequency representation designed for real-time audio processing. It achieves variable time–frequency resolution by leveraging a slice-wise implementation of the nonstationary Gabor transform (NSGT) across overlapping windowed segments of the input signal. This approach yields a logarithmic frequency analysis that aligns with human auditory perception and the harmonic structure of music, enabling fine frequency resolution at low frequencies and fine temporal resolution at high frequencies. The sliCQ framework is theoretically underpinned by frame theory and enables perfect reconstruction, making it suitable for musical signal analysis, effect processing, and machine learning applications in source separation (Hanssian, 2021, Holighaus et al., 2012).

1. Theoretical Foundations

The standard Constant-Q Transform (CQT) analyzes a signal using filters with bandwidths proportional to their center frequencies, resulting in a fixed quality factor $Q = f/\Delta f$ across all bins. For each bin $k$, the center frequency is $f_k = f_{\min}2^{k/B}$, where $B$ is the number of bins per octave. The window length for bin $k$ is $N_k = \lceil Q f_s / f_k \rceil$, with $f_s$ as the sampling rate. Each frequency channel applies a windowed, modulated analysis, adapting the tradeoff between time and frequency resolution according to the musical scale (Holighaus et al., 2012).

The nonstationary Gabor transform (NSGT) generalizes the CQT by allowing arbitrary window lengths and hop sizes per channel. Slicing techniques introduced in sliCQ further adapt the NSGT framework to process the signal in blocks, facilitating streaming and real-time processing while preserving invertibility via tight frames.

2. Slice-wise Processing Scheme

Rather than analyzing the entire signal at once, sliCQ divides the input $x[n]$ into overlapping slices of fixed length $L_s$, with hop size $H_s$. Each slice is windowed (typically with a half-cosine or Tukey window) to ensure smooth overlap and coverage, satisfying $k$0. On each slice, a CQ-NSGT is performed, using zero-padded or periodized analysis windows tailored to that slice's length.

For slice $k$1:

$k$2

CQ-NSGT coefficients are computed per-channel $k$3 as:

$k$4

with per-channel hop sizes $k$5 chosen to admit a painless frame and support invertibility (Hanssian, 2021, Holighaus et al., 2012).

3. Invertibility and Computational Properties

Perfect reconstruction is achieved by choosing dual windows $k$6 for analysis/synthesis and dual slice windows $k$7, followed by overlap-add:

$k$8

where $k$9 is synthesized per-slice from the modified coefficients.

Theoretical guarantees ensure that, with proper window and hop parameters, the system forms a tight frame over $f_k = f_{\min}2^{k/B}$0. Numerical experiments demonstrate end-to-end signal-to-noise ratios exceeding 120 dB, up to floating-point precision (Holighaus et al., 2012).

Computational complexity per slice is $f_k = f_{\min}2^{k/B}$1 for slice length $f_k = f_{\min}2^{k/B}$2; overall, the cost scales linearly with the signal length, enabling real-time, low-latency operation for suitably chosen parameters.

4. Time–Frequency Trade-off and Design Parameters

sliCQ exhibits varying time–frequency resolution across the spectrum. Low-frequency bins have large $f_k = f_{\min}2^{k/B}$3, enhancing frequency discrimination at the cost of temporal precision; high-frequency bins have shorter $f_k = f_{\min}2^{k/B}$4, offering finer time resolution and coarser frequency spacing. This log-frequency partitioning mirrors auditory pitch perception and the spectral layout of musical instruments.

Typical design parameters for musical audio (as in Hanssian et al.) include:

• Bins per octave $f_k = f_{\min}2^{k/B}$5: Bark-scale mapping to 262 bins from 32.9 Hz to 22.05 kHz.
• Slice length $f_k = f_{\min}2^{k/B}$6: ≈ 8192 samples (about 185 ms at 44.1 kHz).
• Slice hop $f_k = f_{\min}2^{k/B}$7: $f_k = f_{\min}2^{k/B}$8 (50% overlap).
• Analysis windows: Hann for $f_k = f_{\min}2^{k/B}$9, half-cosine for $B$0.
• Channel hop $B$1: typically $B$2 to $B$3.
• Zero-padding: each filter window is zero-padded to $B$4.
• Parameter optimization: slice parameters selected by random search to maximize source separation performance (Hanssian, 2021).

5. Comparison with Short-Time Fourier Transform

The standard Short-Time Fourier Transform (STFT) uses uniformly spaced frequency bins and a fixed window length, resulting in invariant time–frequency resolution throughout the spectrum. This restricts its ability to resolve low-frequency components temporally and high-frequency components spectrally.

In contrast, sliCQ’s log-frequency binning and variable window lengths allocate denser bins in frequency where musically relevant and permit adaptive time-resolution, potentially yielding improved representation of bass harmonics and percussion transients.

Computationally, STFT runs at $B$5 per frame for fixed $B$6. sliCQ, depending on the distribution of $B$7, can be heavier but benefits from ragged slicing and frequency grouping to partially mitigate overhead (Hanssian, 2021).

6. Applications and Observed Limitations

sliCQ enables invertible, low-latency, and musically adaptive time-frequency representations suitable for music information retrieval, effect processing, pitch shifting, and source separation.

In music demixing, Hanssian et al. implemented an end-to-end system substituting the STFT with sliCQT for the Open-Unmix architecture. An analysis using the “noisy-phase oracle” separation found that sliCQT improved the median SDR from 6.23 dB (STFT) to 7.42 dB. However, the full-trained demixing model xumx-sliCQ achieved lower median SDR (3.6 dB) compared to the baseline UMX (4.64 dB) and X-UMX (5.54 dB).

Conjectured causes for this disparity include the complexity introduced by ragged, per-frequency-group convolutional architectures, difficulty modeling multiple time-resolutions per-slice, and possible phase artifacts introduced across slices—challenges not present in the regular STFT grid. No numerical or invertibility instability was observed; the limitations were concentrated in the downstream learning models (Hanssian, 2021).

7. Implementation Notes and Practical Considerations

• Slicing windows (e.g., Tukey, half-cosine) must form a partition of unity for invertibility.
• Filter length and slice size trade off latency with frequency accuracy; longer slices reduce aliasing and edge effects.
• With adequately chosen parameters, coefficient fidelity compared to full-length CQ-NSGT is 60–80 dB SNR (Holighaus et al., 2012).
• Real-time feasibility is demonstrated: for 1 s audio at 44.1 kHz with $B$8 bins/octave and slice length 16,384, runtime is <$B$9 ms per second of audio on a commodity CPU.

A plausible implication is that the theoretical advantages of adaptive time–frequency resolution do not automatically translate into superior performance in learned source separation, underscoring the need for architectures explicitly tailored to exploit the properties of the sliCQ representation (Hanssian, 2021, Holighaus et al., 2012).