PSELDnet: Pre-Trained SELD Architectures

• PSELDnet is a pre-trained SELD architecture using robust audio encoders like PANNs/CNN14, PaSST, and HTS-AT with unified Multi-ACCDOA outputs.
• It leverages supervised pretraining on large-scale synthetic spatial mixtures and pseudo-FOA reconstruction to adapt effectively to stereo SELD tasks.
• Replacing traditional decoders with bidirectional Mamba and asymmetric convolutions reduces computational load while boosting localization and detection performance.

Searching arXiv for the original PSELDnet paper and related SELD pretraining work.
PSELDnet denotes a family of pre-trained neural backbones for sound event localization and detection (SELD), built on strong audio encoders such as PANNs/CNN14, PaSST, and HTS-AT, and trained on large-scale synthetic SELD datasets with Multi-ACCDOA outputs. Its central premise is supervised pretraining on diverse spatial mixtures so that the learned representation transfers effectively to downstream SELD tasks with limited real data. In the stereo SELD setting studied in 2025, PSELDnet provides the pretrained backbone for CNN14–Conformer and HTS-AT variants that are then fully fine-tuned on DCASE2025 Task 3; one of these variants replaces the Conformer decoder with bidirectional Mamba sequence modeling and asymmetric convolutions [2506.13455].

1. Definition and distinguishing characteristics

PSELDnet is defined as a pre-trained SELD architecture built on top of state-of-the-art audio encoders and SELD-specific heads. The output representation follows the Multi-ACCDOA paradigm, which unifies sound event activity and localization into a single representation rather than separating sound event detection and direction-of-arrival estimation into distinct branches. This distinguishes PSELDnet from traditional SELD baselines such as CRNN systems trained from scratch on limited data and using separate SED and DOA branches [2506.13455].

The pretraining objective is supervised SELD rather than self-supervised or semi-supervised. The stated advantage is exposure to large-scale synthetic spatial scenes and diverse spatial mixtures, which provides robust features for localization and detection. This suggests that the defining contribution of PSELDnet is not a single decoder design, but a transfer-learning regime in which pretrained spatial-acoustic representations are reused across downstream SELD tasks with constrained labeled data [2506.13455].

2. Backbone family and representation learning

The PSELDnet family is organized around interchangeable pretrained encoders and SELD heads. The encoders explicitly named are PANNs/CNN14, PaSST, and HTS-AT. In the stereo Task 3 study, two variants are fine-tuned: a CNN14–Conformer model and an HTS-AT model. In the CNN14-based system, the encoder is retained as CNN14 from PANNs, described as six VGG-style convolutional blocks with (3 \times 3) convolutions, batch normalization, and ReLU, yielding hierarchical time–frequency features [2506.13455].

The SELD output parameterization is tied to ACCDOA-style geometry. The data specify the standard 3D Cartesian mapping
[
x = \cos\varphi\cos\theta,\quad y = \sin\varphi\cos\theta,\quad z = \sin\theta,
]
and note that for the stereo DCASE2025 Task 3 configuration, which excludes elevation, the mapping reduces to
[
x = \cos\varphi,\qquad y = \sin\varphi,\qquad \theta \equiv 0.
]
In the final stage of the stereo system, the authors apply Tanh on DOA-related outputs, mapping them to ([-1,1]), followed by the linear azimuth conversion
[
\varphi = 90^\circ \cdot z,\quad z \in [-1,1],
]
and ReLU on distance outputs to enforce non-negativity. Because Relative Distance Error is reported, a distance head is present in the system [2506.13455].

A notable implementation choice is full fine-tuning of all parameters, with no frozen layers. The stated justification is the feature mismatch between the original pretrained setting and the stereo adaptation, particularly the use of pseudo-FOA features in place of the original input formulation [2506.13455].

3. Stereo adaptation and pseudo-FOA construction

PSELDnet expects FOA-format features consisting of 4-channel log-mel spectrograms and 3-channel intensity vectors. To reuse pretrained weights in the stereo DCASE2025 Task 3 pipeline, the study reconstructs a “pseudo FOA” signal from stereo left/right channels under ACN/SN3D assumptions:
[
\begin{aligned}
W(n) &= \frac{L(n) + R(n)}{2}, \
Y(n) &= \frac{L(n) - R(n)}{2}, \
X(n) &= 0,\quad Z(n) = 0.
\end{aligned}
]
From this, the system computes 4-channel log-mel spectrograms and 3-channel intensity vectors, concatenating them into a 7-channel input tensor [2506.13455].

The feature extraction configuration is fixed at a 24 kHz sampling rate, with STFT window length 40 ms, hop length 20 ms, and 64 mel filters. Each input sequence has 250 frames aligned to 50 label frames. In stereo Task 3, elevation is not evaluated and azimuth is folded to the frontal field of view, with azimuth constrained to ([-90^\circ, 90^\circ]) [2506.13455].

The augmentation strategy is Audio Channel Swapping (ACS), which exchanges left and right channels and adjusts azimuth labels accordingly. The study states that ACS effectively doubles the development data and helps with front–back ambiguity. A plausible implication is that stereo adaptation in PSELDnet depends as much on label-consistent spatial augmentation as on encoder reuse, because pseudo-FOA reconstruction alone does not restore missing (X) and (Z) components [2506.13455].

4. Decoder substitution: BiMamba and asymmetric convolutions

In the 2025 stereo study, the original Conformer decoder in the CNN14–Conformer PSELDnet is replaced by a bidirectional Mamba module, yielding a CNN14–BiMamba system. The BiMamba module runs two Mamba streams, one forward and one backward over time; their hidden representations are concatenated and fused, followed by a linear compression layer that produces the sequence output. Bidirectionality is used to address the causal limitation of the original Mamba formulation and to model both past and future context [2506.13455].

The underlying sequence model is presented through the continuous-time linear time-invariant state-space system
[
\begin{aligned}
\mathbf{h}'(t) &= \mathbf{A}\,\mathbf{h}(t) + \mathbf{B}\,x(t),\
y(t) &= \mathbf{C}^\top \mathbf{h}'(t) + \mathbf{D}\,x(t),
\end{aligned}
]
with Zero-Order Hold discretization
[
\bar{\mathbf{A}} = \exp(\Delta\,\mathbf{A}),\qquad
\bar{\mathbf{B}} = \mathbf{A}^{-1}\big(\exp(\Delta\,\mathbf{A}) - \mathbf{I}\big)\,\mathbf{B},
]
and discrete recursion
[
\mathbf{h}k = \bar{\mathbf{A}}\,\mathbf{h}{k-1} + \bar{\mathbf{B}}\,x_k,\qquad
y_k = \mathbf{C}^\top \mathbf{h}_k.
]
The convolutional-kernel view is written as
[
K(\Delta) = \mathbf{C}\,e^{\mathbf{A}\Delta}\,\mathbf{B}.
]
In selective state-space models such as Mamba, parameters can vary across time to modulate which inputs are emphasized [2506.13455].

The study adds asymmetric convolutions in the decoder to decouple temporal and spectral dependencies more effectively. These convolutions are applied in alternating order across time and frequency axes; kernel sizes are not explicitly stated, but the conceptual description is that elongated kernels such as (k \times 1) or (1 \times k) are used to target temporal or spectral context separately. The paper does not disclose exact hidden sizes, number of layers, or Mamba state dimensions, but it states that BiMamba is inserted where the Conformer had been used and that bidirectional processing gives the model global temporal context [2506.13455].

The theoretical motivation is computational. Conformer-style attention is described as (O(n^2)) in sequence length, whereas BiMamba uses linear-time selective state-space modeling. In the paper’s interpretation, this lowers computational burden while preserving or improving SELD accuracy [2506.13455].

5. Evaluation protocol and empirical profile

The evaluation follows DCASE Task 3 conventions, with location-dependent (F1) using a (20^\circ) tolerance, mean absolute azimuth error as DOAE, and Relative Distance Error as RDE. The metric definitions are given as
[
\mathrm{Precision} = \frac{TP}{TP+FP},\quad
\mathrm{Recall} = \frac{TP}{TP+FN},\quad
F1 = \frac{2\,\mathrm{Precision}\cdot\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}},
]
[
\mathrm{DOAE} = \frac{1}{M}\sum_{i=1}^M \big|\hat{\varphi}i - \varphi_i\big|,
]
and
[
\mathrm{RDE} = \frac{1}{M}\sum{i=1}^M \frac{|\hat{d}_i - d_i|}{d_i}\times 100\%.
]
True positives require class and time agreement together with azimuth error at most (20^\circ) [2506.13455].

Training is performed on the official DCASE2025 Task 3 development set, described as stereo data derived from STARSS23 real recordings via mid-side conversion. The setup uses seed 42, the Adam optimizer, a ReduceLROnPlateau scheduler with mode='max', factor=0.5, patience=5, and 120 epochs. Model-specific hyperparameters are reported for HTS-AT, Conformer, ConBiMamba, BiMamba, and BiMambaAC. Sequence length sensitivity, memory footprint, and latency are not reported [2506.13455].

Variant	Params / MACs	Dev-set results
BiMambaAC (s1)	76M / 4.63G	(F1(20^\circ)=39.6\%), DOAE (=15.8^\circ), RDE (=33\%)
BiMamba (s2)	178M / 7.57G	(F1=36.2\%), DOAE (=16.6^\circ), RDE (=33\%)
Conformer (s3)	210M / 4.69G	(F1=38.2\%), DOAE (=15.9^\circ), RDE (=33\%)
HTS-AT (s4)	28M / 2.88G	(F1=35.1\%), DOAE (=16.5^\circ), RDE (=30\%)
ConBiMamba	338M / 7.98G	(F1=36.2\%), DOAE (=16.9^\circ), RDE (=53\%)
Baseline CRNN	0.7M / 57M	(F1=22.8\%), DOAE (=24.5^\circ), RDE (=41\%)

The best-performing model is BiMambaAC, the CNN14–BiMamba system with asymmetric convolutions. Relative to the original Conformer-based PSELDnet, it improves (F1) by (+1.4) points, slightly improves DOAE by (0.1^\circ), reduces parameters from 210M to 76M, and reduces MACs from 4.69G to 4.63G. All pretrained PSELDnet variants substantially outperform the baseline CRNN, confirming the benefit of large-scale pretraining. The comparison between plain BiMamba and BiMambaAC further indicates that the asymmetric convolutions are a major contributor, since BiMambaAC improves (F1) by (+3.4) points while reducing parameter count by 102M [2506.13455].

6. Limitations, interpretive issues, and nomenclature

Several limitations follow directly from the stereo adaptation procedure. Because pseudo-FOA reconstruction sets (X=0) and (Z=0), the input discards components needed for elevation or general 3D localization. The study also states that generalization to non-stereo arrays, including native FOA or microphone arrays, would require revisiting the input features. Elevation is excluded in the DCASE2025 stereo task, and the remaining front–back ambiguity is mitigated only partially through azimuth folding and ACS; the authors note that confusions may still occur near the (\pm 90^\circ) limits [2506.13455].

The reported error analysis is correspondingly bounded. The study states that selective SSM gating and bidirectional context help discriminate overlaps, but that failures remain for heavily overlapping events in reverberant scenes. Robustness to SNR and source movement is not explicitly analyzed, although the authors suggest that the gains imply better handling of dynamic sequences through bidirectional context. A plausible implication is that the principal demonstrated advantage of PSELDnet in this configuration is not universal spatial completeness, but efficient transfer of pretrained SELD structure into a stereo-constrained benchmark [2506.13455].

A separate terminological issue arises outside SELD. In a 2026 paper on distribution system state estimation, the explanatory note states that the underlying method is not named “PSELDnet” in the paper itself; rather, that label is introduced only within the explanation to denote an end-to-end pseudo-measurement learning framework for DSSE. This is a distinct usage and should not be conflated with the SELD backbone family introduced by Hu et al. [2605.23525].