Region-Specific Audio Tagging

• Region-specific audio tagging is a specialized task that labels sound events in designated spatial regions using multichannel recordings.
• It combines spectral, spatial, and positional features, such as inter-channel phase differences and directional cues, to enhance event localization.
• Applications include fixed-region, location-aware, and weakly supervised tagging, yielding improvements in mAP and more precise event detection.

Region-specific audio tagging is the task of labeling sound events that occur within a specified region of a recording rather than over the recording as a whole. In its most explicit formulation, the input is multichannel spatial audio recorded by a microphone array together with a region specification, and the output is the set of sound event labels present in that region; the region can be specified as an angular space or a distance from the microphone (Zhao et al., 11 Sep 2025). Within a broader research lineage, the topic also intersects with weakly supervised temporal region selection, graph-based modeling of spectrogram regions, and conditioning on spatial or geospatial context beyond the waveform itself (Kong et al., 2016, Singh et al., 2023, Hou et al., 11 Mar 2026).

1. Formal problem definition and task scope

The direct formulation of region-specific audio tagging uses multichannel audio $\mathbf{x} \in \mathbb{R}^{4 \times L}$ from a tetrahedral microphone array and a region specification $\mathbf{p}$, with the goal of predicting the set of sound event labels present in region $\mathbf{p}$ (Zhao et al., 11 Sep 2025). The region may be given as an angular range or as a specific distance from the array. This formulation differs from standard audio tagging, which labels all present sound events without spatial restriction; region-specific tagging labels only those events within the specified spatial region (Zhao et al., 11 Sep 2025).

Three system types are explicitly distinguished. Omnidirectional (OD) tagging is standard tagging without region. Fixed-region (FR) tagging predicts tags for pre-defined angular regions. Location-aware (LA) tagging uses regions dynamically aligned to actual source positions and is described as an upper bound (Zhao et al., 11 Sep 2025). These distinctions matter because they separate unconditional clip-level recognition from recognition conditioned on a query region.

Generic audio tagging baselines are not inherently region-specific. In the DCASE 2019 baseline systems, generic cross-task systems based on convolutional neural networks were proposed for several audio recognition tasks, and the authors reported that the optimal architecture is task-dependent, with the 9-layer CNN with average pooling after convolutional layers being a good model for a majority of the DCASE 2019 tasks (Kong et al., 2019). This provides a useful contrast: conventional baselines are designed to transfer across tasks without exploiting task-specific spatial characteristics, whereas region-specific tagging explicitly conditions prediction on spatial scope.

2. Region encoding and feature representations

The direct spatial formulation combines three feature families: spectral, spatial, and positional (Zhao et al., 11 Sep 2025). The spectral feature is the logarithm power spectrum,

$\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$

used for sound event identification. Spatial cues include inter-channel phase difference,

$$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$

and generalized cross-correlation phase transform,

$$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$

These features provide localization cues, while positional features encode the region of interest itself (Zhao et al., 11 Sep 2025).

For angular queries, the paper defines Directional Features and a Field of View feature. The directional feature for each angle $\theta$ is

$$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$

with

$$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$

For a candidate region $\Theta$, the feature construction uses

$\mathbf{p}$0

and then

$\mathbf{p}$1

For distance queries, a neural network is trained to embed the given distance $\mathbf{p}$2 as a positional feature (Zhao et al., 11 Sep 2025).

Earlier work on stereo environmental audio tagging had already shown that explicit spatial features can improve tagging. A Convolutional Gated Recurrent Neural Network incorporated the Interaural Magnitude Difference,

$\mathbf{p}$3

through an auxiliary CNN whose activations were concatenated with those from the main feature CNN before a GRU-RNN stage (Xu et al., 2017). On DCASE 2016 task 4, the proposed structure reduced the equal error rate from $\mathbf{p}$4 to $\mathbf{p}$5 on the development set, and the spatial features further reduced the EER to $\mathbf{p}$6; on the evaluation set, the best existing system had $\mathbf{p}$7 EER, while the proposed MFB+IMD system achieved $\mathbf{p}$8 EER (Xu et al., 2017). This earlier result established that spatial cues can improve tag prediction even before the explicit introduction of region-conditioned querying.

3. Weak supervision and informative temporal regions

A substantial part of the literature operationalizes “region” as an informative temporal segment within a weakly labeled clip. The Joint Detection-Classification model addresses the fact that most audio tagging datasets are weakly labelled and that bag-of-blocks models unrealistically assume all tags occur throughout the clip (Kong et al., 2016). For each tag $\mathbf{p}$9 and block $\mathbf{p}$0, a detector produces an informativeness weight $\mathbf{p}$1 and a classifier produces a block-level probability $\mathbf{p}$2. The detector outputs are normalized as

$\mathbf{p}$3

and the clip-level probability is

$\mathbf{p}$4

Training uses only clip-level labels through binary cross-entropy (Kong et al., 2016).

On the CHiME Home dataset, the MFCC + GMM baseline yielded $\mathbf{p}$5 EER on the evaluation set, the bag-of-blocks model yielded $\mathbf{p}$6, and the Joint Detection-Classification model yielded $\mathbf{p}$7 (Kong et al., 2016). The paper further reports that detector visualizations align with true event locations even without event-level supervision. This indicates that region-specific tagging can be approached as implicit event localization when only weak labels are available.

The Global-local Attention Framework extends this line by introducing a two-stream structure in which a global stream analyzes the whole clip and a local stream processes class-wise selected sub-clips (Wang et al., 2021). The global stream computes per-frame, per-class activations; for each of the top $\mathbf{p}$8 predicted classes, the system selects the frame $\mathbf{p}$9 and extracts a sub-clip centered at $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$0 with duration $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$1. The total loss is

$\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$2

On AudioSet, the framework improved mAP from $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$3 to $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$4 for TALNet, from $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$5 to $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$6 for CNN10, from $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$7 to $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$8 for ResNet38, and from $\mathbf{L} = \log\left(|\mathbf{X}|^2\right),$9 to $$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$0 for AT-SCA (Wang et al., 2021). This suggests that, even absent explicit spatial microphones or external metadata, conditioning on selected local regions is a practical route to finer-grained tagging.

4. Spectrogram-region modeling with graphs

Another research direction treats the spectrogram itself as a set of regions with non-grid interactions. ATGNN models the spectrogram as a graph rather than as a fixed 2D grid or a sequence of patches (Singh et al., 2023). Each element of the feature map is treated as a node, edges are constructed dynamically using k-nearest neighbors, and graph convolution uses a max-relative update. The architecture contains a Patch GNN for correlations within spectrogram patches, a Patch-Label GNN for relationships between spectrogram regions and learnable class embeddings, and a Label-Label GNN for semantic correlations between labels (Singh et al., 2023).

The Patch-Label GNN is explicitly region-label oriented. Each class label has a learnable embedding, each label node connects to its k-nearest spectrogram patch neighbors, and label embeddings are updated by message passing from those patch nodes. Final prediction combines patch-based and label-based logits through

$$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$1

ATGNN achieved $$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$2 mAP on FSD50K and $$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$3 mAP on AudioSet-balanced, with comparable results to Transformer based models and significantly lower number of learnable parameters (Singh et al., 2023). The reported ablations show that removing the explicit label-patch and label-label modeling reduces mAP on both datasets.

LHGNN also emphasizes local and higher-order relations among spectrogram regions (Singh et al., 7 Jan 2025). The model flattens the feature map $$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$4 into nodes, builds local neighborhoods with k-NN, and derives higher-order relations from Fuzzy C-Means clusters. The core update is

$$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$5

On AudioSet, LHGNN achieved $$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$6 mAP with ImageNet pretraining and $$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$7 without pretraining, compared with AST at $$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$8 and $$\mathbf{I}^n = \mathrm{angle}(\mathbf{X}^{n_1}) - \mathrm{angle}(\mathbf{X}^{n_2}),$$9 respectively; on FSD50K, LHGNN achieved $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$0 with pretraining and $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$1 without pretraining, compared with AST at $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$2 (Singh et al., 7 Jan 2025). In the ablation on FSD50K without pretraining, local-only yielded $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$3 mAP, higher-order-only yielded $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$4, and the combined model yielded $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$5 (Singh et al., 7 Jan 2025). A plausible implication is that “region-specific” modeling in audio tagging encompasses not only explicit physical regions, but also learned local and higher-order regions in the time-frequency representation.

5. Spatial and geospatial context as region priors

Region-specificity can also be encoded through metadata associated with where and when audio was recorded. SONYC-UST-V2 is an urban sound tagging dataset with spatiotemporal information consisting of $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$6 audio recordings from the SONYC acoustic sensor network, including timestamp of audio acquisition and location of the sensor (Cartwright et al., 2020). Its metadata include NYC block identifier, latitude and longitude of the block centroid, borough, hour, day of week, and week of year. The baseline model concatenates 11×512-dimensional OpenL3 embeddings with spatial and temporal features, yielding 597-dimensional inputs per frame, and uses a multi-label MLP with a single hidden layer of 128 nodes and AutoPool (Cartwright et al., 2020).

The reported baseline comparison shows little difference between using and omitting spatiotemporal context. At the fine level on the test set, the baseline with STC achieved Macro-AUPRC $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$7, Micro-AUPRC $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$8, and LWLRAP $$\mathbf{G}^n(\tau) = \int_{-\infty}^{+\infty} \frac{\mathbf{X}^{n_1}(f)\, \mathbf{X}^{n_2*}(f)}{|\,\mathbf{X}^{n_1}(f)\mathbf{X}^{n_2*}(f)|} e^{i2\pi f\tau} df.$$9, while the baseline without STC achieved $\theta$0, $\theta$1, and $\theta$2 (Cartwright et al., 2020). The paper states that no significant improvement is observed from the inclusion of STC features in this naive MLP baseline. This is an important corrective to the common assumption that adding region metadata is automatically beneficial.

Geo-ATBench formulates geospatial audio tagging as conditioning multi-label sound event tagging on geospatial semantic context derived from geographic information system data such as points of interest (Hou et al., 11 Mar 2026). The benchmark contains $\theta$3 hours of audio across 28 event categories, and each clip is paired with a GSC representation from 11 semantic context categories (Hou et al., 11 Mar 2026). GeoFusion-AT evaluates feature-level, representation-level, and decision-level fusion on PANNs, AST, and CLAP, alongside audio-only and GSC-only baselines. For late fusion, the fused logits are

$\theta$4

The best reported results include GeoFusion-Early with AST at $\theta$5 mAP versus audio-only AST at $\theta$6, GeoFusion-Inter with CLAP at $\theta$7 versus audio-only CLAP at $\theta$8, and GeoFusion-Late with PANNs at $\theta$9 versus audio-only PANNs at $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$0 (Hou et al., 11 Mar 2026). A GSC-only baseline achieves $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$1 mAP. The paper additionally reports that 17 out of 28 classes benefit with more than $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$2 AP increase, and that a crowdsourced listening study with 10 participants on 579 samples found no statistically significant difference in performance between models on benchmark labels and aggregated human labels (Hou et al., 11 Mar 2026).

A related precursor is audio geotagging from content alone. “Audio Content based Geotagging in Multimedia” infers city-level location from the composition of sound classes in a recording, using semi-NMF to obtain semantic content and kernel fusion over event-specific histograms (Kumar et al., 2016). On a dataset of 1079 Flickr videos from 18 cities, the best proposed method achieved $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$3 MAP, compared with $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$4 for Bag of Audio Words and $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$5 for GMM supervectors (Kumar et al., 2016). Although this is location inference rather than event tagging conditioned on region, it demonstrates that sound-event composition carries geographically relevant information.

6. Empirical findings, limitations, and research directions

The direct evidence for the explicit spatial task comes from “Region-Specific Audio Tagging for Spatial Sound,” which studies both simulated and real datasets and extends PANNs, AST, and PSLA to accept concatenated spectral, spatial, and positional features (Zhao et al., 11 Sep 2025). On feature combinations, LPS + IPD + FOV yielded the best mAP of $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$6, LPS + IPD + DF yielded $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$7, LPS + IPD + learned yielded $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$8, LPS + GCCPHAT + DF was comparable to IPD, and SALSA features yielded $$\mathbf{D}(\theta) = \sum_{n} \cos(\mathbf{I}^n - \mathbf{P}^n(\theta)),$$9 (Zhao et al., 11 Sep 2025). On model comparison, PANNs achieved $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$0 mAP, PSLA $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$1, and AST $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$2 (Zhao et al., 11 Sep 2025).

The same study reports a marked difference between omnidirectional and region-conditioned systems. OD tagging yielded mAP approximately $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$3, fixed-region tagging yielded $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$4, and location-aware tagging yielded $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$5; the corresponding EERs were $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$6, $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$7, and $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$8 (Zhao et al., 11 Sep 2025). Region size is also consequential: as the angular region grows, performance drops from $$\mathbf{P}^n(\theta) = 2\pi f \phi^n \cos(\theta) f_s / c.$$9 mAP at $\Theta$0 to $\Theta$1 at $\Theta$2 and $\Theta$3 at $\Theta$4, approaching omnidirectional tagging (Zhao et al., 11 Sep 2025). On dataset difficulty, PANNs achieved $\Theta$5 mAP on STARSS23 and $\Theta$6 on SRSAT, the latter being described as having higher overlap and up to 10 events (Zhao et al., 11 Sep 2025). For distance-query tagging, PANNs achieved $\Theta$7 mAP, PSLA $\Theta$8, and AST $\Theta$9 (Zhao et al., 11 Sep 2025).

Two limitations recur across the literature. First, contextual information is not uniformly useful: SONYC-UST-V2 found that naive inclusion of latitude, longitude, hour, day of week, and week of year produced no significant improvement in the baseline MLP (Cartwright et al., 2020), whereas Geo-ATBench found consistent gains when GSC was integrated through explicit fusion mechanisms (Hou et al., 11 Mar 2026). Second, directional features help when the task is region-conditioned but do not help omnidirectional tagging (Zhao et al., 11 Sep 2025). A plausible implication is that region-specific audio tagging requires the region specification to be treated as a first-class conditioning signal, rather than as an undifferentiated auxiliary feature.

Taken together, the field now contains at least three technically distinct meanings of “region”: physical regions in space specified by angle or distance, informative temporal regions selected under weak supervision, and discriminative spectrogram regions represented as graph nodes or context-conditioned embeddings (Zhao et al., 11 Sep 2025, Wang et al., 2021, Singh et al., 2023). The explicit spatial formulation is the most direct definition of the term, but the broader literature shows that successful tagging increasingly depends on conditioning mechanisms that restrict inference to the relevant subset of the audio scene rather than collapsing all evidence into a single clip-level summary.