Demenba: Dementia Speech Classification

• Demenba is an automatic dementia classification framework that analyzes hour-long neuropsychological assessments using state space models and a Mamba/VMamba backbone.
• It combines audio feature extraction with ASR-derived text modeling via selective segment voting and late multimodal fusion to capture sparse diagnostic cues.
• Empirical evaluations on the Framingham Heart Study indicate notable AUC improvements in both binary and detailed normal/MCI/dementia classification compared to baseline models.

Demenba is an automatic dementia classification framework for long-form neuropsychological test recordings that uses state space models (SSMs), specifically a Mamba/VMamba-style backbone, to recognize cognitive impairment directly from speech. It was developed for the Framingham Heart Study (FHS) neuropsychological audio collection and trained on over 1,000 hours of assessments. Its central premise is that dementia-related evidence is sparse, recordings are often around an hour long, and conventional short-window or transformer-style approaches are poorly matched to that regime; Demenba therefore combines long-sequence audio modeling, ASR-derived text modeling, selective segment voting, and late multimodal fusion, with its strongest reported gains appearing on fine-grained normal/MCI/dementia classification (Wang et al., 14 Jul 2025).

1. Clinical and methodological scope

Demenba belongs to the line of work on automatic dementia classification from speech, but it addresses a different operating regime from much of the earlier literature. Earlier systems often focused on short, highly structured tasks such as Cookie Theft picture description, using transcript models, multimodal spontaneous-speech classifiers, or prompt-based LLM reasoning over relatively short utterances (Palo et al., 2019, Ilias et al., 2023, Park et al., 26 Jun 2026). Demenba instead targets clinician-administered neuropsychological interviews that are long, multi-speaker, noisy, and diagnostically unevenly informative (Wang et al., 14 Jul 2025).

The paper’s clinical motivation is conventional but technically consequential. Early dementia detection can support earlier intervention, better disease management, and improved quality of life, yet neuropsychological assessments remain labor-intensive and require expert scoring. Demenba treats speech from those assessments as a source of ADC signals such as hesitations, pauses, disfluencies, semantic breakdowns, and interactional patterns. The paper’s more specific claim is architectural: because useful cues are intermittent and can be separated by long stretches of low-information dialogue, linear-time SSMs are a better fit than quadratic-attention transformers or short-context CNN pipelines for this setting (Wang et al., 14 Jul 2025).

This framing makes Demenba less a generic speech classifier than a long-context clinical sequence model. The design goal is not merely improved binary screening, but preservation of discourse-scale and turn-scale context over minute-scale segments that are then aggregated at the recording level.

2. Data basis and task formulation

The data source is the FHS neuropsychological audio collection. The paper states that FHS contains approximately 11,000 hour-long neuropsychological test recordings, of which 2,058 had been reviewed for dementia. After adjudication, each reviewed recording is labeled as normal, mild cognitive impairment (MCI), or dementia. To balance training material, the authors randomly sample roughly 400 recordings per class, yielding a 1,200-recording balanced subset for the 3-class task; they also define a 2-class task by merging MCI and dementia into an impaired class and sampling 400 recordings from the combined impaired pool (Wang et al., 14 Jul 2025).

The reported split statistics are unusually detailed. For the balanced 3-class setup, training has 586 participants, 74 interviewers, class counts 407/399/399 for normal/MCI/dementia, and 1,447 hours total. The test set has 10 recordings per class and 36 hours. An eval set is also reported with 92 participants, 20 interviewers, 68/10/14 normal/MCI/dementia recordings, and 77 hours. For the balanced 2-class setup, the training split contains 378 participants, 61 interviewers, class counts 400/234/173 over normal/MCI/dementia, and 943 hours total; the test set has 10/5/5 recordings over normal/MCI/dementia and 25 hours. The paper further states that participants do not overlap between training and test splits, although the exact construction of the held-out validation split is underdescribed (Wang et al., 14 Jul 2025).

The assessments are long, clinician-administered interviews probing memory, attention, perception, verbal fluency, and reasoning. Labels come from adjudicated review rather than a single test response, which is clinically important because Demenba is not predicting a proxy score from isolated utterances. The paper also revisits a historically used 92-recording FHS subset with manual transcripts and manual diarization, enabling cleaner comparison with prior work; on that subset, Whisper ASR has a character error rate of about 60%, underscoring the difficulty of the speech signal (Wang et al., 14 Jul 2025).

3. System architecture

Demenba has four operational components: a speech segmenter, an ASR system, an audio classifier, and a text classifier. A frozen speech segmenter first divides each recording using either voice activity detection (VAD) or speaker diarization (SD). Consecutive chunks are then merged up to a maximum segment length of 360 seconds, preserving substantial context while fitting a fixed memory budget. For the audio branch, each merged segment is converted into 128-bin filterbank features with 10 ms frame shift, 25 ms frame length, and a Hanning window (Wang et al., 14 Jul 2025).

The audio classifier is SSM-based and uses a VMamba backbone. The backbone contains four groups of Mamba layers, each followed by downsampling by a factor of two, and the final sequence representation is mean-pooled to produce class logits. Demenba is reported in two sizes: Demenba-small with 29M trainable parameters and Demenba-medium with 48M. The principal baseline is EfficientNet-b6 with 40M trainable parameters, matched as closely as possible in front-end features, segment durations, and optimization settings (Wang et al., 14 Jul 2025).

The text branch runs in parallel. Speaker-diarized segments are passed to Whisper-Large v2 ASR, each utterance is prefixed with its role such as “Participant” or “Interviewer,” and the full transcript is then classified by either BERT or larger LLMs including LLaMA, Qwen, and phi-4. The paper explores both zero-shot prompting and fine-tuning; for zero-shot LLMs, the prompt includes a task description, transcript, valid label set, and the instruction “Answer:”. Because context windows remain limited, the same selective voting mechanism used for audio can also be applied over transcript segments (Wang et al., 14 Jul 2025).

Audio and text are combined by late fusion:

$$p_{\text{audio+text}} = (1-\lambda)p_{\text{audio}} + \lambda p_{\text{text}},$$

with $\lambda \in [0,1]$ tuned to balance acoustic and linguistic evidence. This is the paper’s main explicit fusion definition. In the 2-class setting it compares BCE and CE, reporting that CE works best for Demenba while BCE works better for the EfficientNet baseline; in the 3-class setting it uses weighted CE with class weights $(1,3,3)$ for normal, MCI, and dementia (Wang et al., 14 Jul 2025).

4. Training, inference, and selective aggregation

Training for the audio model runs for 40 epochs with Adam, learning rate $10^{-5}$, $\beta_1=0.95$, $\beta_2=0.999$, weight decay $5\times10^{-7}$, batch size 1, 1,000 warmup steps, and exponential decay by 0.5 per epoch beginning at epoch 10. All training is performed on a single NVIDIA A6000 48GB GPU. The VMamba backbone is initialized from ImageNet-pretrained weights, which the authors state performed better than AudioSet pretraining in preliminary tests (Wang et al., 14 Jul 2025).

Inference is selective rather than uniform. Recording-level prediction is not obtained by averaging all segment outputs, because the model assumes many segments are uninformative. Instead, if a recording has segment-level probabilities, Demenba ranks segments by their highest predicted class probability, keeps the top $k$ segments, with $k$ tuned on validation, and performs a soft majority vote over only those top-$k$ segments. The same logic can be applied to transcript segments for text models (Wang et al., 14 Jul 2025).

This inference design is clinically motivated. It treats hour-long assessments as sparse evidence streams in which only some moments are strongly diagnostic. A plausible implication is that Demenba is as much an evidence-selection framework as a backbone choice: the SSM handles long segments, and selective voting suppresses dilution by low-information regions.

The paper also explores multiple text back ends. In 2-class classification, fine-tuned BERT reaches 0.91 AUC, LLaMA with LoRA reaches 0.83, and Qwen with LoRA reaches 0.85. Parameter counts reported for text systems include BERT at 109M trainable parameters, LLaMA LoRA at 21M, and Qwen LoRA at 20M; fused systems include Demenba-medium+BERT at 157M and Demenba-medium+LLaMA at 69M (Wang et al., 14 Jul 2025).

5. Empirical performance and scaling behavior

The main reported metric is AUC. On audio alone, Demenba consistently outperforms EfficientNet, with larger gains on the harder 3-class task. In 2-class classification, EfficientNet-b6 reaches 0.76 AUC with VAD and 0.82 with SD, while Demenba-medium reaches 0.82 with VAD and 0.87 with SD. In 3-class classification, EfficientNet-b6 reaches 0.62 with VAD and 0.69 with SD, while Demenba-medium reaches 0.75 with VAD and 0.83 with SD (Wang et al., 14 Jul 2025).

Setting	Best reported system	AUC
2-class audio on FHS	Demenba-medium + SD	0.87
3-class audio on FHS	Demenba-medium + SD	0.83
2-class audio+text on main setup	Demenba-medium + BERT	0.95
92-recording FHS subset, audio+text	Demenba-medium + BERT	0.92

The paper’s headline claim of “up to 21% improvement” refers to relative AUC improvement in fine-grained dementia classification. The most direct reading given in the paper is the comparison between Demenba-medium with VAD at 0.75 and EfficientNet with VAD at 0.62, i.e.

$\lambda \in [0,1]$0

That formulation is important because the strongest absolute 3-class result is actually 0.83 AUC with SD, while the “21%” figure is a relative comparison under a specific segmentation condition (Wang et al., 14 Jul 2025).

Text-only and multimodal experiments show that transcripts remain highly informative despite noisy ASR. The best overall system is Demenba-medium+BERT at 0.95 AUC in the main 2-class setup, exceeding both the best audio-only and best text-only systems. On the 92-recording subset with manual transcripts and diarization, Demenba-medium audio-only reaches 0.81 AUC and Demenba-medium+BERT reaches 0.92, compared with 0.83 for the prior multimodal EfficientNet+LM baseline (Wang et al., 14 Jul 2025).

The scaling studies are among the paper’s clearest contributions. Sample-size scaling shows steady improvement from 50 to 400 hours per class, with no plateau for Demenba-medium. Sequence-length scaling shows consistent gains from 30 to 180 seconds, with 8–17 absolute AUC points in the 2-class task and 4–5 points in the 3-class task; moving from 180 to 360 seconds adds only about 1–2 points. Including silence yields a 10–15% AUC boost in the 2-class setting and only 0–2% in the 3-class setting. This suggests that long-context modeling and pause structure matter strongly for normal-versus-impaired discrimination, while finer MCI-versus-dementia separation depends on more than silence alone (Wang et al., 14 Jul 2025).

6. Limitations, interpretability, and position in the literature

The paper is explicit that Demenba is not a fully transparent clinical system. It argues that fusion with text-based models may move toward greater inspectability because transcripts and language-model outputs are easier to inspect than raw audio embeddings, and selective voting can surface highly indicative segments rather than averaging all content. But it does not present saliency maps, rationale extraction, or clinician-facing evidence summaries (Wang et al., 14 Jul 2025).

Several limitations are central. Evaluation is confined to a single study cohort, FHS, so broader generalization remains open. The exact validation protocol is underdescribed. ASR is noisy, with about 60% CER on the manually transcribed subset. Most notably, interviewer speech alone is highly predictive in the role-specific ablation, confirming interviewer bias or interviewer adaptation to patient impairment. For Demenba-medium in 2-class evaluation on the eval set, using both speakers reaches 0.81 AUC, roughly 4 points higher than participant-only and 5 points higher than interviewer-only. The paper explicitly raises ethical concerns here: a model may exploit examiner behavior rather than patient pathology (Wang et al., 14 Jul 2025).

Within the broader literature, Demenba occupies a specific niche. Transcript-based and multimodal systems on short Cookie Theft narratives have shown strong performance on curated benchmarks (Palo et al., 2019, Ilias et al., 2023), while later LLM-based systems on ADReSSo frame dementia detection as structured multi-view reasoning over transcripts, pauses, discourse, fluency, and phonology (Park et al., 26 Jun 2026). Demenba differs by centering long clinical interviews and linear-time sequence modeling rather than prompt-serialized short utterances. Longitudinal multimodal work has also argued that repeated naturalistic communication may be more informative than one-off case-control snapshots, especially for monitoring change over time (Gkoumas et al., 2021). This suggests that Demenba is best understood not as a replacement for those paradigms, but as a specialized framework for hour-scale assessment recordings where context retention, selective evidence aggregation, and audio-text complementarity are decisive.

Overall, Demenba is a long-context SSM-based ADC system that treats dementia detection from neuropsychological audio as a sparse-evidence sequence problem. Its main technical significance lies in showing that VMamba-style audio modeling, selective top-$\lambda \in [0,1]$1 segment voting, and late fusion with text can improve both binary impairment detection and finer normal/MCI/dementia discrimination on FHS, while using fewer parameters than the prior EfficientNet audio baseline (Wang et al., 14 Jul 2025).