---
title: TACos for Few-Shot Keyword Spotting
url: https://www.emergentmind.com/topics/tacos
type: topic
---

# TACos for Few-Shot Keyword Spotting

TACos is a few-shot keyword spotting (KWS) method that replaces fixed-size sliding-window detection with temporally structured embeddings used as templates for dynamic time warping (DTW). It was introduced to address a central mismatch in few-shot KWS: keywords and their spoken instances vary in duration, so a fixed window must be long enough to contain the keyword while avoiding irrelevant context such as multiple words or noise. TACos addresses this by learning two-dimensional embeddings that retain temporal properties of the underlying speech signal, and by coupling those embeddings with subsequence DTW for detection and boundary localization [2305.10816].

## 1. Problem setting and motivation

Few-shot KWS systems often segment audio with a sliding window of fixed size. In this setting, window selection is intrinsically difficult: a window should be long enough to contain all necessary information needed to recognize a keyword, but a longer window may contain irrelevant information such as multiple words or noise and thus makes it difficult to reliably detect on- and offsets of keywords. TACos is designed specifically for this regime of limited labeled examples and variable keyword duration.

The method is formulated around template matching rather than closed-vocabulary classification over fixed windows. This is significant because few-shot KWS requires reliable matching from very small support sets, while also preserving the internal progression of a spoken keyword. Standard embedding objectives can separate keyword classes, but they tend to produce representations that are comparatively static over time. TACos instead makes temporal progression part of the representation objective.

A plausible implication is that TACos is best understood not as a generic embedding model with a DTW backend added afterward, but as a representation-learning scheme whose primary purpose is to make DTW meaningful at the segment sequence level.

## 2. Temporally structured embedding architecture

TACos produces two-dimensional embeddings that preserve temporal order across the signal. The preprocessing pipeline is: waveform, then log-Mel spectrogram, then a sequence of overlapping segments. A shared ResNet backbone encodes each segment, with global max-pooling over frequency only, so temporal resolution is preserved through padding.

The resulting embedding for a segment is \(e \in \mathbb{R}^{T \times D_{\text{emb}}}\), where \(T\) is the temporal dimension and \(D_{\text{emb}}\) is the embedding dimension. In the reported configuration, \(D_{\text{emb}} = 128\). Rather than flattening time, the model yields a sequence of embedding vectors, one per time step, so the representation remains suitable for sequence alignment.

This design contrasts with sliding-window baselines that pool on both time and frequency and therefore flatten temporal structure. In TACos, each short segment can be embedded as a vector while maintaining ordering across the audio. The summary describes this as a 2D sequence of embeddings whose variation over time reflects both keyword identity and internal structure. This temporal organization is the basis for later DTW matching.

## 3. Joint keyword-position supervision

The defining component of TACos is a joint loss that combines supervised keyword discrimination with self-supervised positional prediction [2305.10816]. The two components are:

- **Supervised keyword loss**: an AdaCos angular margin loss, \(\mathcal{L}_{\text{kw}}\), classifying the segment as belonging to a keyword.
- **Self-supervised positional loss**: a sub-cluster AdaCos loss, \(\mathcal{L}_{\text{pos}}\), predicting the relative position of the segment within the keyword.

Relative rather than absolute position is used because keywords vary in length. Positions are discretized with \(N_{\text{pos}}\) classes, defined as the maximum number of segments across keywords. If a keyword is shorter, the positional label is assigned softly or uniformly.

The similarity between an embedding and a keyword-position pair is defined through trainable cluster centers:
$$
\theta(e, i_{\text{kw}}, i_{\text{pos}}) =
\frac{1}{T}\sum_{t=1}^{T}
\max_{i_{\text{cluster}}}
\frac{\langle e(t), c(i_{\text{cluster}}, i_{\text{kw}}, i_{\text{pos}})\rangle}
{\|e(t)\|_2 \, \|c(i_{\text{cluster}}, i_{\text{kw}}, i_{\text{pos}})\|_2},
$$
where \(c(\cdot)\) are trainable cluster centers and \(N_{\text{cluster}}\) is a hyperparameter. In the reported training setup, \(N_{\text{cluster}} = 16\).

The total TACos loss is
$$
\mathcal{L}_{\text{TAC}} =
-\frac{1}{K}\sum_{k=1}^{K}
\frac{1}{N_{\text{seg}(k)}}
\sum_{i_{\text{seg}}=1}^{N_{\text{seg}(k)}}
\left(
\mathcal{L}_{\text{kw}}(e_{k,i_{\text{seg}}}) +
\mathcal{L}_{\text{pos}}(e_{k,i_{\text{seg}}})
\right).
$$

This objective encourages the model to learn not only which keyword a segment comes from, but also where in that keyword the segment lies. The central contribution of TACos is therefore not merely angular-margin supervision, but angular-margin supervision with explicit temporal structure.

## 4. DTW-based detection and template matching

The TACos backend uses subsequence DTW for keyword detection in audio streams. For each training keyword example, the system stores the full 2D embedding sequence as a template. For a test utterance, the audio is segmented with overlap, encoded into an embedding sequence, and compared against the templates using a pairwise cosine-distance cost matrix.

DTW is then applied with allowed steps \((2,1)\), \((1,1)\), and \((1,2)\). Accumulated costs are normalized by path length, and negative normalized accumulated costs are used as matching scores. Detections above threshold are taken as keyword matches, with corresponding onsets and offsets. If matches overlap, the one with the highest score is kept per time.

An important implementation choice is that TACos uses templates from individual training examples rather than a hard center average. The reported experiments state that Fréchet means hurt performance. This indicates that preserving instance-level temporal variation is preferable to collapsing few-shot supports into a single averaged prototype.

The role of DTW in TACos is therefore broader than alignment tolerance alone. Because the learned embeddings are temporally structured, DTW can exploit internal keyword progression rather than only endpoint similarity.

## 5. Training protocol, augmentation, and KWS-DailyTalk

Experiments are conducted on KWS-DailyTalk, a few-shot KWS dataset presented with the method. The dataset contains 15 keywords, with 5 isolated training samples per keyword and total training audio of approximately 39 seconds. Validation and test data consist of real sentences of approximately 10 minutes each, with diverse background conditions and multiple, zero, or overlapping keyword occurrences. Onsets and offsets are manually annotated. Training examples for a keyword are separated from evaluation examples at the conversation level, so no training sample of a keyword comes from the same conversation as any evaluation sample.

The reported metrics are micro-averaged event-based F-score, precision, and recall using the `sed_eval` toolbox. The encoder is a ResNet-style model with 713k parameters and a 2D structure. Training uses Mixup and SpecAugment data augmentation together with random oversampling for class balance.

A further augmentation is specific to TACos: time-reversed keyword segments are included during training. For each segment other than no-speech segments, a time-reversed copy is added with a unique auxiliary label. The model is trained to recognize forward and reverse versions as distinct keywords, with positions encoded using uniform soft labels for no-speech and reverse. The stated purpose is to encourage the model to capture the correct sequence order and avoid trivial matching.

This suggests that the augmentation is not a generic regularizer. It is targeted at sequence order sensitivity, which is especially relevant when DTW is the downstream matcher.

## 6. Empirical performance and interpretation

The experiments compare TACos against hand-crafted features, a sliding-window neural baseline, and prior embeddings without positional loss. Test-set F-scores reported for the main systems are as follows [2305.10816].

| System | Test F-score |
|---|---:|
| HFCC (best) | 57.7% |
| Sliding window (best) | 50.4% |
| Prior embedding (no pos loss, no reverse) | 53.0% |
| Prior embedding (no pos loss, reverse) | 61.3% |
| TACos (positional loss, no reverse) | 63.3% |
| TACos (positional loss, reverse, global threshold) | 70.5% |
| TACos (positional loss, reverse, per-kw threshold) | 69.2% |

The strongest result is TACos with positional loss and time-reversed segments, using a global threshold, at 70.5% F-score. This outperforms HFCC + DTW, the sliding-window neural baseline, and the prior embedding-based approach, despite using only approximately 39 seconds of training audio.

The ablations separate two effects. First, adding reversed segments improves all approaches, including the prior embedding system. Second, the positional loss further improves performance beyond reverse augmentation alone. The combination yields the best result. A common misconception is that TACos’s gains can be reduced to DTW replacing a sliding window; the reported comparison indicates instead that the decisive factor is the interaction between temporally structured embeddings, positional supervision, and DTW-based matching.

Another reported observation is that per-keyword threshold tuning helped sliding-window baselines a bit, but did not further improve TACos, which worked well with a global threshold. This has a practical consequence for deployment because threshold management is simplified without sacrificing the best reported F-score.

Source: https://www.emergentmind.com/topics/tacos