---
title: Device-Directed Speech Detection
url: https://www.emergentmind.com/topics/device-directed-speech-detection-ddsd
type: topic
---

# Device-Directed Speech Detection

Device-Directed Speech Detection (DDSD) is the binary inference problem of deciding whether a spoken utterance is addressed to a voice assistant or device, as opposed to side conversation, background speech, media audio, or other non-target speech. In the literature summarized here, DDSD is motivated by two recurring product conditions: rejection of false wake-ups or unintended activations, and support for wake-word-free or triggerless follow-up interaction after an initial command. The task has evolved from utterance-level classifiers that fuse acoustics, ASR decoder uncertainty, and lexical hypotheses into broader multimodal, streaming, and large-foundation-model formulations, while retaining the same operational objective: accept device-directed speech and suppress non-device-directed speech under realistic deployment constraints [1808.02504], [2310.15261], [2406.09617], [2604.08412].

## 1. Canonical task definition and relation to adjacent problems

In its canonical form, DDSD is a binary classification task with labels such as **device-directed** versus **nondevice-directed**, or equivalently **intended** versus **unintended** speech. The practical setting is usually a voice assistant that has already been activated, either by a wakeword, touch, or a preceding conversational turn, after which the system must determine whether subsequent speech should be routed to downstream ASR, NLU, or response generation modules. This framing is explicit in early follow-up-query work and remains central in later studies of multimodal and LLM-based systems [1808.02504], [2010.01949], [2411.00023].

A persistent distinction in the literature is between DDSD and wake-word spotting. Wake-word systems detect a specific lexical trigger; DDSD infers user intent in the absence of such a fixed lexical anchor. For touch-based invocation, this distinction becomes especially stark, because the model must determine whether arbitrary payload speech is directed to the device without keyword evidence. Several papers therefore describe DDSD as a form of false-trigger mitigation that is harder than conventional keyword detection because the utterance content is unconstrained and the relevant cues span acoustics, ASR confidence, lexical structure, and conversational context [2203.15975], [2110.04656].

The literature also identifies nearby but non-identical tasks. A mobile-array front/back speech detector can be interpreted as a spatial proxy for likely device-directed speech, but it does not model addressee intent, wake words, or lexical semantics; the “front” class is only an approximation to device-directedness [2106.14787]. An accessibility-oriented detector of intentional nonverbal mouth sounds similarly solves a DDSD-like gating problem—distinguishing deliberate user input from speech and background audio—but its target events are closed-set nonverbal commands rather than open-vocabulary directed speech [2202.07750]. These related tasks are best understood as precursors or specialized variants rather than substitutes for canonical DDSD.

A further conceptual reframing appears in work on **Sequential Device-Addressed Routing (SDAR)**, which argues that utterance-local classification is insufficient in multi-speaker, pre-ASR edge settings. In that formulation, DDSD becomes a sequential, cost-sensitive routing problem over current evidence \(X_t\), bounded interaction history \(\mathcal{H}_t\), and deployment thresholds, with actions \(a_t \in \{forward,\; suppress,\; abstain\}\). This suggests that some ambiguity in device addressability is fundamentally temporal rather than utterance-local [2604.08412].

## 2. Early architectures: acoustics, ASR hypotheses, and decoder uncertainty

The 2018 baseline formulation established a three-source architecture that remains historically central. It used short-term acoustics, ASR decoder features, and the ASR 1-best hypothesis as complementary signals. The acoustic branch consumed **64-dimensional log filter-bank energies (LFBEs)** computed with a **25 ms window** and **10 ms shift**, trained an LSTM with repeated utterance labels at frame level, and used the **pre-softmax output of the last frame** as an utterance representation called the acoustic embedding \(\mathbf a\). The lexical branch encoded the 1-best character sequence with **pre-trained GloVe embeddings**, trained an LSTM over character positions, and used the last character output as the char embedding \(\mathbf c\). Decoder-side features \(\mathbf d\) were engineered from the ASR trellis and confusion network, including trellis entropy, Viterbi costs, ASR confidence, and average number of arcs per node. The fused utterance vector was written as
\[
\mathbf f = [\mathbf a,\mathbf c,\mathbf d].
\]
The paper reports this as a **22-dimensional utterance vector** passed to a feed-forward DNN classifier [1808.02504].

That system also established a recurring empirical pattern: decoder uncertainty features are individually strong, lexical hypotheses alone are weaker, and multimodal combination is substantially better than any single stream. On real recordings of natural far-field interactions, the reported EERs were **9.3%** for decoder features, **10.9%** for the acoustic embedding, and **20.1%** for the char embedding; pairwise combinations improved further, and the full \([\mathbf a,\mathbf c,\mathbf d]\) system reached **5.2% EER**, described as a **44% relative improvement** over the best single feature set [1808.02504].

Subsequent lexical-semantic work broadened the textual side of DDSD. A 2020 study treated device-directedness classification as a mixed lexical and lightweight acoustic problem using current-turn ASR text, optional previous-turn text, and token-level ASR confidence. Word embeddings were **300-dimensional fastText wiki news embeddings**, the lightweight acoustic feature was the token-level ASR confidence, and the best system was an **LSTM + Attention** model. On a follow-up dataset of roughly **240K utterances**, the best configuration using current text, previous text, and token confidences achieved **9.1% EER**, compared with **10.6%** for the state-of-the-art acoustic-only baseline it referenced. The paper reports this as a **14% relative EER reduction**. Its ablation study found removal of token confidences especially damaging, with EER rising to **14.8%**; removal of current text raised EER to **13.7%**, whereas removal of previous-turn text raised it to **9.5%** [2010.01949].

Two points are notable in these early architectures. First, ASR uncertainty is not merely an auxiliary confidence score; it is repeatedly treated as a structured cue to mismatch between speech and a device-directed distribution. Second, lexical information becomes more valuable when previous-turn context is available, particularly in wake-word-free follow-up scenarios where short commands such as “play,” “stop,” or “next” may appear in both directed and non-directed conditions [1808.02504], [2010.01949].

## 3. Multimodal fusion, prosody, and missing-modality robustness

Later DDSD work generalized the classic acoustic-text-decoder triad into broader multimodal systems with explicit robustness mechanisms. One line of work added **non-verbal/prosodic** cues to otherwise verbal DDSD pipelines. In that formulation, the verbal side comprised an acoustic false trigger mitigator on **40-dimensional filter-bank features**, a text DDSD model built as a DNN head on top of a fine-tuned **BERT-like** model, and an ASR/lattice-based model using a modified **LatticeRNN**. The prosodic side extracted **pitch, voicing, jitter, shimmer, and VAD scores** at **100 Hz**, with VAD implemented as a CNN-based speech/silence classifier followed by a 2-state HMM. The prosody model was a **single-layer GRU** with **128 hidden units**, trained with **Adam**, learning rate **0.001**, **50 epochs**, **batch size 150**, weighted binary cross-entropy, and about **50K parameters** [2310.15261].

That work compared late fusion of scalar scores with intermediate fusion of penultimate-layer embeddings. The scores were denoted
\[
S_\text{a},\; S_\text{t},\; S_\text{asr},\; S_\text{p},
\]
and the embeddings
\[
E_\text{a},\; E_\text{t},\; E_\text{asr},\; E_\text{p},
\]
with dimensions **256**, **128**, **16**, and **128** respectively. Its main finding was that **intermediate fusion outperforms late fusion**, and that prosody provides complementary information when fused nonlinearly at the embedding level. On a dataset of **over 19,000 audio recordings**, about **530 hours**, and roughly **245k segments**, verbal-only intermediate fusion reached **7.1% FA@10% FR** and **9.2% EER**; adding prosody improved this to **6.5% FA@10% FR** and **8.9% EER**. With **modality dropout**, the prosody-augmented intermediate-fusion model reached **5.7% FA@10% FR** and **8.9% EER** on the full test set, and under a missing-modality test condition with **30%** of modality inputs randomly dropped, FA@10% FR improved from **11.46%** to **10.61%** when modality dropout was added [2310.15261].

A related but distinct robustness problem arises when text and audio are present but misaligned because ASR transcripts are erroneous. The **M\(^3\)V** formulation addresses this by treating DDSD as a **multi-view learning** problem with four views: audio-only evidence, text-only evidence, multimodal fused evidence, and an explicit **text-audio alignment view** learned with an InfoNCE-style contrastive loss. The model uses **Wav2vec2** as the audio encoder and **GPT2** as the text encoder, with separate FFN heads for the audio view \(V_a\), text view \(V_t\), and multimodal view \(V_m\), plus an alignment branch \(V_{\text{align}}\). At inference, a policy module either arbitrates among views using thresholds on the alignment score or fuses the four view scores with an SVM. On real in-vehicle assistant data from **NOMI**, the multi-modal baseline reached **95.27% accuracy** and **6.59% EER** on the normal test set, while the paper reports **96.41% accuracy** for the full framework. On an **ASR error test set** with **55.60% character error rate (CER)**, the paper reports **95.71% accuracy** and states that this surpasses human judgment performance on that test set [2409.09284].

These multimodal studies jointly suggest that DDSD performance depends not only on having multiple information streams, but also on how modality reliability is represented. Prosody captures “how” an utterance is produced; alignment branches estimate whether lexical and acoustic evidence should be jointly trusted; modality dropout trains fusion models not to fail when some streams disappear [2310.15261], [2409.09284].

## 4. Streaming and on-device DDSD

A major branch of DDSD research targets streaming inference and strict on-device resource budgets. One purely acoustic line developed a **streaming ResLSTM** architecture with a CNN front-end, residual connections, stacked unidirectional LSTMs, and a **causal mean aggregation** layer. The CNN contained **13 convolutional layers total**, followed by **3 unidirectional LSTM layers** with **64 units** each and **2 fully connected layers** of size **64**. Its key streaming mechanism computed a running utterance representation
\[
s_t = \frac{1}{t}\sum_{i=1}^{t} h_i
\]
with the recursive form
\[
s_t = \frac{t-1}{t}\cdot s_{t-1} + \frac{1}{t}\cdot h_t.
\]
The model used **256-dimensional log energy of STFT** features, **30 ms** frames, and utterances truncated to **300 frames**. On **4,000 hours** of training audio and **6 million utterances**, the paper reports that **ResLSTM** achieved a **41.1% relative EER reduction** over its baseline LSTM-S, and that causal mean aggregation matched attention at end-of-utterance evaluation while performing better for earlier decisions [2007.09245].

A closely related deployment problem is unified streaming DDSD across **voice-triggered (VT)** and **touch-based (TB)** invocation. An on-device acoustic false-trigger-mitigation framework addressed both with a common streaming architecture and a new **s-TCN** decision layer derived from temporal convolutional networks. Audio was processed in causal overlapping blocks of size **\(B=64\) frames** with shift **\(S=32\)**, corresponding to about **1.92 s** of context per block and a new decision about every **1 second** after the first block. The model used **40-D Mel filterbank features**, augmented to **280-D** with neighboring frames, and **6 streaming attention layers**. At the selected operating points, VT FAR values were **3.2%** for the non-streaming A2A baseline, **4.2%** for s-LSTM, **6.4%** for s-TCN, and **8.6%** for s-AVE; on the harder TB task, FAR values were **24.3%** for A2A, **25.1%** for s-TCN, and **28.7%** for both s-LSTM and s-AVE. The paper further reports **33.3%** runtime peak-memory reduction for s-TCN relative to the non-streaming counterpart, compared with **7.3%** for s-LSTM [2110.04656].

Playback interference constitutes another streaming failure mode. Instead of explicit echo cancellation followed by detection, an **implicit acoustic echo cancellation (iAEC)** framework conditions the detector directly on a playback reference signal. The mixture model is written as
\[
\mathbf{y} = \Gamma(\mathbf{r}) + \mathbf{u},
\]
or under a linear approximation
\[
\mathbf{y} = \mathbf{r} * \mathbf{h} + \mathbf{u}.
\]
The most effective variant, **iAEC-M**, uses shared encoders for mixture and reference features, derives a sigmoid mask
\[
\mathbf{M} = \sigma(\mathcal{P}([\mathbf{Z}_y, \mathbf{Z}_r])),
\]
and gates the mixture representation via
\[
\mathbf{Z}_\gamma = \mathbf{M} \odot \mathbf{Z}_y.
\]
On a real Alexa follow-up-mode dataset with **538k utterances**, the baseline TCN had non-playback FRR **0.189** and playback FRR **0.348**, whereas iAEC-M at the D2 fusion point achieved non-playback FRR **0.181** and playback FRR **0.150**, which the paper describes as a **56% reduction in false-reject rate for the DDD task during device playback conditions** [2111.10639].

The strongest deployment-oriented reframing appears in **SAS**, which treats pre-ASR device-addressed speech detection as SDAR. SAS is a three-stage cascade: acoustic geometry or beamforming, utterance-level classification with a lightweight **1D convolutional classifier** over **64-dim log-mel filterbank features** using **25 ms** frames and **10 ms** hop, and a **small causal Transformer** over an **8-second** history window of previous Stage-2 outputs, VAD state, and inter-utterance time deltas. Stage 3 produces a multiplicative prior
\[
c'_t = \alpha_t \cdot c_t^{(\text{S2})}, \qquad \alpha_t \in [0,1].
\]
On a held-out **60-hour** multi-speaker English test set, the primary audio-only configuration achieved **F1=0.86** with **precision=0.89** and **recall=0.83**; with optional camera-based fusion, audio+video reached **F1=0.95**, **precision=0.97**, and **recall=0.93**. Removing Stage 3 reduced audio+video F1 from **0.95** to **0.57 ± 0.03**, described as the largest observed ablation effect. The full system runs on **ARM Cortex-A class hardware** with **<150 ms latency** and **<20 MB footprint** [2604.08412].

## 5. Weak supervision, distillation, and compact deployment models

DDSD research has repeatedly confronted the problem that labeled intended-versus-unintended data are expensive or ambiguous. One solution is weak supervision regularized by distillation. In a touch-based false-trigger-mitigation setting, weak labels were generated deterministically using SNR and ASR-text intent rules: samples with \(\text{SNR} \le 5\text{ dB}\) and background-speech intent were labeled unintended; samples with \(\text{SNR} \ge 15\text{ dB}\) and not-background-speech intent were labeled intended; all others were discarded. The paper reports **8% misclassification** for the low-SNR rule and **2% misclassification** for the high-SNR rule on the covered subset, motivating regularization. An acoustics-only **AFTM** model trained on **40-D mel filterbanks at 100 fps** and a stronger ASR-based **LatticeRNN** teacher were combined via
\[
loss = loss_{BCE} + \alpha \cdot KL(\text{LRNN}, \text{AFTM}),
\]
with \(\alpha = 10\). The distilled acoustic model, **AFTM-D**, improved from **18.4% EER** to **6.2% EER**, which the paper describes as a **66% gain in accuracy** over the base AFTM, while fusion of LRNN and AFTM-D achieved **4.0% EER** versus **4.8%** for LRNN alone [2203.15975].

A more recent knowledge-distillation line adopts a large pretrained ASR acoustic encoder as teacher and a small DDSD student. In **Adaptive Knowledge Distillation for Device-Directed Speech Detection**, the teacher is a frozen **12-layer Conformer** ASR encoder with a task-specific DDSD head, while the student is a compact **Transformer** or **Conformer** of about **5M parameters**. The distillation objective combines supervised cross-entropy with embedding distillation,
\[
\mathcal{L}_{\text{ED}} = \text{MSE}(E^\text{T}, E^\text{S}),
\]
pseudo-labeling,
\[
\mathcal{L}_{\text{PL}} = \text{CE}(\hat{y}, p^\text{S}),
\]
and attention regularization,
\[
\mathcal{L}_{\text{AR}} = \sum_t (\alpha_t^\text{T} - \alpha_t^\text{S})^2.
\]
The paper reports **+26% relative improvement** in EER for keyword-based invocations and **+19%** for keyword-free follow-up invocations, and states **22% average EER reduction across invocation types** [2508.02801].

A different compact-deployment strategy appears in **SELMA**, a speech-enabled language model that handles DDSD jointly with voice trigger detection and ASR. DDSD is formulated generatively: after a task token such as `<|DD|>`, the model reads the probability of the next token being `yes` versus `no`,
\[
p_\theta(Y=\text{yes} \mid c).
\]
SELMA is built on **Qwen-Audio-Chat**, combining a Whisper-large-v2-style audio encoder of about **650M parameters** with a decoder-only **Qwen** backbone of about **7.4B parameters**, but trains only **5.5M** parameters, about **0.84%** of the full model, using LoRA. A key architectural contribution is mean pooling of the audio sequence to a global summary \(\mathbf R\) and concatenation with the original sequence. On a DDSD dataset of about **237k queries**, the main configuration **SELMA 1** achieved **7.78% EER**. The paper reports this as a **22% relative EER improvement** over a strong baseline at **10.00% EER**, and notes that the gating variant **SELMA 4** reached **7.63% EER** on DDSD, though it was not the preferred overall multitask configuration [2501.19377].

These studies converge on a recurring theme: small deployable DDSD models can be improved either by transferring structure from stronger teachers or by exploiting multitask parameter-efficient adaptation, without abandoning the binary directedness objective [2203.15975], [2508.02801], [2501.19377].

## 6. LLM and foundation-model formulations

A major recent shift is the use of LLMs not merely as text encoders but as multimodal decision engines for DDSD. One GPT2-based approach treats DDSD as autoregressive generation conditioned on acoustic prefixes, ASR 1-best hypotheses, and decoder signals. Whisper provides the acoustic sequence, mean-pooled and projected by a mapping network \(M_1\); decoder features are projected by \(M_2\); GPT2 then receives the acoustic prefix, decoder-signal prefix, the 1-best hypothesis tokens, and a prompt. The system is trained with the autoregressive cross-entropy objective
\[
\mathcal{L}=-\sum_{i=1}^L \sum_{j=1}^l \log p_\theta\left(t_j^i \mid a^i, b^i, t_1^i, \ldots, t_{j-1}^i\right).
\]
On combined evaluation data, the reported EERs were **13.00%** for Whisper plus a classification head, **11.40%** for the text-only GPT2 baseline, **8.57%** for the audio-only proposed model, **6.81%** for text+audio, **6.45%** for text+audio+decoder signals, and **6.34%** when decoder signals were combined with text, audio, and an additional **500k** text-only corpus. The paper reports relative EER improvements of **38.9% over text-only** and **61% over audio-only** for multimodal configurations, and notes further gains of **up to 18%** from larger LLMs and LoRA [2403.14438].

A closely related decoder-only LLM framework emphasizes data and resource efficiency. It combines ASR 1-best text, decoder features such as graph cost, acoustic cost, word posterior confidence, and number of alternatives per word, and audio representations from either **Whisper-medium** or a specialized **UAD** backbone. The audio encoder output is mean-pooled and mapped to an LLM prefix:
\[
a^i = M_1 \left(\frac{1}{T} \sum_{t=1} ^T h_t \right),
\]
while decoder signals are mapped to \(b^i\). The system uses **Falcon 7B** or **RedPajama 7B** with **LoRA** and prefix tuning. On a balanced dataset of **80,630** training examples and **37,354** evaluation examples, the best multimodal configuration, **Multi 4**, achieved **8.23% EER** with **Falcon + UAD**, compared with **9.31%** for the best audio-only unimodal baseline using the same backbone. The paper also reports that **10k examples** with the multimodal UAD configuration can outperform the **80k** audio-only baseline, and argues that specialized **256-dimensional** UAD representations are more effective than **1024-dimensional** Whisper representations in low-data regimes [2312.03632].

**FLoRA** extends parameter-efficient adaptation from unimodal text LLMs to multimodal DDSD. It projects audio embeddings \(E^A\), video embeddings \(E^V\), and text embeddings \(E^T\) into the text-embedding space, concatenates them, and inserts two adapter layers per encoder/decoder layer. The adapted representation is written as
\[
E' = (E \times W) + E'_V + E'_A + E'_T.
\]
To handle missing modalities, **adapter dropout** skips modality-specific adapters when that modality is absent. On **B2B-1**, text-only full fine-tuning achieved **EER 8.8** and **FA@10 7.0**, while **FLoRA (A+V+T)** achieved **EER 6.8** and **FA@10 2.6**, which the paper reports as a **22% relative EER reduction over the text-only approach** and near-parity with full multimodal fine-tuning at only about **5%** trainable parameters. On **B2B-3**, FLoRA reached **EER 8.4** and **FA@10 6.9**, compared with **14.0** and **20.2** for text-only full fine-tuning [2406.09617].

A separate follow-up-conversation study uses **Vicuna-7B-v1.3** and models DDSD from the first query plus the follow-up ASR text, optionally with an **8-best** hypothesis list and ASR costs. It explores prompt-only inference, LoRA-finetuned prompting, a linear classification head on the LLM, and a LoRA-based classifier head. The paper reports that jointly modeling previous-query context and ASR uncertainty yields **20–40% reduction in false alarms at 10% fixed false rejects** compared with modeling follow-ups alone. For the main fixed-FRR metric, the best reported **ClassHead** result is **EER = 7.9%** and **FAR @ 10% FRR = 4.8%**, while the best **ClassLoRAHead** result is **EER = 8.5%** and **FAR @ 10% FRR = 6.1%** [2411.00023].

Taken together, these LLM-based studies do not eliminate classic DDSD ingredients; rather, they recast them. Acoustic encoders still provide prosodic and interactional cues, ASR hypotheses still encode lexical structure, and decoder signals still represent uncertainty. What changes is the fusion mechanism: instead of hand-designed binary classifiers over fixed embeddings, the decision is delegated to multimodal LLMs adapted by LoRA, prefix tuning, or modality-specific adapters [2403.14438], [2312.03632], [2406.09617], [2411.00023].

## 7. Evaluation practice, recurring findings, and limitations

The dominant DDSD evaluation metric is **Equal Error Rate (EER)**, defined at the operating point on the **Detection Error Tradeoff (DET)** curve where false positive and false negative rates are equal. Many papers also report operating-point metrics such as **FA@10% FR**, **FA@4%FR**, **FAR at 5% FRR**, or **FAR at 10% FRR**, reflecting deployment needs where false accepts and false rejects are not equally costly [1808.02504], [2310.15261], [2203.15975], [2411.00023]. More deployment-centered work reports class-specific **F1**, precision, and recall for the device-addressed class, especially in pre-ASR routing scenarios [2604.08412].

Several findings recur across otherwise different architectures.

**Complementarity of modalities**: acoustics, lexical hypotheses, decoder uncertainty, and prosody repeatedly provide non-redundant evidence. Early LSTM fusion, intermediate embedding fusion, GPT2-based multimodal prefixing, and FLoRA-style adapters all report improvements over unimodal baselines [1808.02504], [2310.15261], [2403.14438], [2406.09617].

**Importance of uncertainty**: decoder features, ASR n-best lists, and alignment views are consistently useful because 1-best transcripts can be ambiguous or incorrect. This is especially explicit in systems using confusion-network confidence, LatticeRNN features, prompt designs that expose n-best costs, and alignment-aware multi-view models [1808.02504], [2312.03632], [2411.00023], [2409.09284].

**Value of context and history**: previous-turn text modestly improves lexical DDSD; first-query context substantially improves follow-up classification; short-horizon causal interaction history can dominate performance in multi-speaker edge settings. This suggests that a nontrivial portion of DDSD difficulty arises from temporally ambiguous utterances rather than purely local acoustics or lexical form [2010.01949], [2411.00023], [2604.08412].

**Deployment tradeoffs**: streaming and on-device systems often sacrifice some representational richness to meet latency, memory, or pre-ASR constraints. Yet several papers show that causal architectures, TCN-based summarizers, reference-conditioned detectors, and compact student models can remain competitive, particularly when distillation or history-aware routing is added [2007.09245], [2110.04656], [2111.10639], [2508.02801].

A concise comparison of representative milestones is given below.

| System | Main signals | Reported headline result |
|---|---|---|
| LSTM fusion DDSD [1808.02504] | acoustics + 1-best + decoder features | **5.2% EER** |
| Lexical + token confidence [2010.01949] | current text + previous text + token confidences | **9.1% EER** |
| Prosody + verbal intermediate fusion [2310.15261] | acoustic + text + ASR + prosody | **5.7% FA@10% FR**, **8.9% EER** |
| Streaming ResLSTM [2007.09245] | acoustics only | **41.1% relative EER reduction** over baseline |
| iAEC-M [2111.10639] | mixture audio + playback reference | playback FRR **0.150** vs **0.348** baseline |
| GPT2 multimodal LLM [2403.14438] | audio + text + decoder signals | **6.34% EER** |
| FLoRA multimodal LLM [2406.09617] | audio + video + text | **6.8 EER**, **2.6 FA@10** on B2B-1 |
| SELMA [2501.19377] | audio + text in multitask LLM | **7.78% EER** |
| SAS / SDAR [2604.08412] | audio-only or audio+video + causal history | **F1=0.86** audio-only, **0.95** audio+video |

The literature also makes clear what DDSD is not. It is not equivalent to wake-word spotting, because many settings of interest are keyword-free. It is not solved by spatial localization alone, because a speech source in front of the device is only a proxy for device addressability. It is not reducible to text-only classification, because ASR ambiguity, overlap, playback, and missing modalities can materially alter the decision boundary. A plausible implication of the collected work is that future DDSD systems will continue to combine multimodal evidence, uncertainty modeling, and causal interaction context, while adapting the exact architecture to whether the deployment point is pre-ASR edge routing, post-ASR follow-up disambiguation, or a multitask assistant stack [2106.14787], [2411.00023], [2604.08412].

Source: https://www.emergentmind.com/topics/device-directed-speech-detection-ddsd