---
title: 'CoughViT: Self-Supervised Cough Audio Transformer'
url: https://www.emergentmind.com/topics/coughvit
type: topic
---

# CoughViT: Self-Supervised Cough Audio Transformer

to=arxiv_search ￣第四色ាជូន  สำนักเลขานุการองค์กร  大发快三的 ￣奇米影视json
{"query":"2508.03764 CoughViT: A Self-Supervised Vision Transformer for Cough Audio Representation Learning", "max_results": 5}
to=arxiv_search  大发快三怎么json
{"query":"2103.12063 QUCoughScope: An Artificially Intelligent Mobile Application to Detect Asymptomatic COVID-19 Patients using Cough and Breathing Sounds", "max_results": 5}
CoughViT is a self-supervised Vision Transformer pre-training framework for learning general-purpose cough sound representations from unlabelled audio, formulated as masked spectrogram reconstruction on log-mel cough inputs [2508.03764]. It was introduced to address two recurrent constraints in cough audio machine learning: scarcity of labeled data beyond COVID-19 and noisy, self-reported labels in crowd-sourced corpora. By adapting masked autoencoding to cough spectrograms, the method aims to enhance diagnostic performance in downstream tasks with limited data while reducing dependence on imperfect disease labels [2508.03764].

## 1. Clinical and methodological setting

Respiratory sound analysis is clinically meaningful because physicians use auscultation to hear wheezes, crackles, and cough characteristics to diagnose conditions like asthma, COPD, pneumonia, and COVID-19 [2508.03764]. Mobile, cough-based diagnostics are attractive for telehealth because cough travels through airways, with less attenuation than chest-wall conduction, and can be recorded with smartphones. This makes cough audio a practical substrate for AI-based screening and triage systems, especially when timely and accessible diagnosis is required.

The principal bottlenecks identified for cough audio ML are label scarcity and label noise. Beyond COVID-19, labeled cough corpora are limited, and crowd-sourced datasets often rely on self-reported health status. These constraints hinder generalization, differential diagnosis, and robust evaluation. CoughViT is positioned as a response to that setting: it uses self-supervised learning with masked data modelling to learn domain-general acoustic structure from large unlabelled cough corpora by reconstructing masked spectrogram regions, thereby avoiding direct dependence on imperfect labels and reducing overfitting to a specific disease [2508.03764].

This framing also places CoughViT in continuity with earlier spectrogram-based respiratory screening systems. QUCoughScope, for example, demonstrated a mobile-first workflow in which users record cough and breathing sounds via an Android app and upload them anonymously for server-side spectrogram-based inference [2103.12063]. That earlier system did not use transformer architectures; it employed ensembles of deep CNN models operating on spectrogram images. In that sense, CoughViT extends an established spectrogram-as-image paradigm into a ViT-based self-supervised regime rather than introducing cough spectrogram modeling itself [2103.12063].

## 2. Input representation and architecture

CoughViT operates on log-mel spectrograms derived from audio resampled to 16 kHz, a choice used to standardize scale across datasets and facilitate transfer learning [2508.03764]. The spectrogram front-end uses frame length $L=25$ ms, hop $H=10$ ms, and 128 mel bins. The number of frames $w$ is defined as

$$
w = (1/L)(t - H) + 1,
$$

where $t$ is the signal duration [2508.03764]. During training, spectrograms are truncated or zero-padded to a fixed number of frames per run to enable batching, although the model is described as naturally accommodating variable lengths via ViT positional encodings. No silence trimming is reported. No spectrogram normalization is applied during SSL pretraining, whereas fine-tuning uses the mean and standard deviation computed over the pretraining dataset, COVID-19 Sounds [2508.03764].

The spectrogram is treated as a two-dimensional image and partitioned into non-overlapping $16 \times 16$ patches in time and frequency, with stride equal to patch size. For a spectrogram of dimensions $w \times h$, the number of patches is

$$
|P| = \lfloor (w - s + \sigma)/\sigma \rfloor \times \lfloor (h - s + \sigma)/\sigma \rfloor,
$$

where patch size is $s$ and stride is $\sigma$ [2508.03764]. The use of non-overlapping patches is explicitly motivated as a way to reduce token count and quadratic self-attention cost without hurting accuracy.

Each patch is linearly projected to a 768-dimensional embedding. A learnable [CLS] token is prepended, and fixed sinusoidal positional encodings are added. The encoder is a standard ViT-Base encoder with 12 Transformer blocks and hidden size $d_{\text{model}} = 768$ [2508.03764]. The paper states that it follows the standard ViT-B configuration, but number of heads, MLP expansion, dropout, and norm placements are not detailed; it notes that typical ViT-B uses 12 heads, MLP size 3072, pre-norm LayerNorm, and residual connections.

The attention mechanism is given in the standard form. For input $X \in \mathbb{R}^{N \times d}$,

$$
Q = XW_Q,\quad K = XW_K,\quad V = XW_V,
$$

with $W_Q, W_K, W_V \in \mathbb{R}^{d \times d_k}$, and single-head attention is

$$
A = \text{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.
$$

Multi-head attention is

$$
\text{MHA}(X) = \text{Concat}(\text{head}_1,\ldots,\text{head}_h)W_O.
$$

These definitions are used in the model description rather than as a novel attention variant [2508.03764].

On the decoder side, CoughViT uses a masked autoencoder-style pipeline. A fraction $r$ of patch tokens is randomly sampled and dropped before encoding; only unmasked tokens and the [CLS] token are passed to the encoder. After encoding, learned 768-dimensional mask tokens are reinserted to restore original ordering and patch count, fixed sinusoidal positional encodings are added again, all tokens are mapped by a linear layer to 524 dimensions, and the sequence is processed by a Transformer decoder with 16 self-attention layers. A final linear layer outputs 256 dimensions per patch, corresponding to the $16 \times 16$ patch reconstruction [2508.03764].

A local windowed self-attention variant was also evaluated in the decoder, using $4 \times 4$ non-overlapping windows with shifted windows in the style of Swin Transformers. In ablations, this was less effective than global attention for cough spectrogram reconstruction [2508.03764].

## 3. Self-supervised learning objective and pretraining data

The self-supervised training objective is masked spectrogram reconstruction. Random masking is applied over two-dimensional patch tokens, with mask ratios $r \in \{0.50, 0.75\}$ studied; the best configuration uses $r=0.75$ [2508.03764]. The decoder predicts raw spectrogram patches represented at the patch level as 256 scalars per $16 \times 16$ patch.

The reconstruction loss is a patch-normalized mean squared error computed only over masked patches:

$$
L_{\text{recon}} = \sum_{i \in \mathcal{M}} \| \tilde{x}_i - \hat{x}_i \|_2^2,
$$

where $x_i$ is the ground-truth patch, $\tilde{x}_i$ is the target patch normalized per patch by its mean and variance, $\hat{x}_i$ is the decoder prediction, and $\mathcal{M}$ is the mask set [2508.03764]. No auxiliary contrastive or BYOL losses are used. No explicit regularizers beyond what is implicit in training are reported; weight decay, warm-up, and gradient clipping are also not reported.

Pretraining uses the crowd-sourced COVID-19 Sounds corpus, comprising 53,449 samples, each with cough, speech, and breath, with only cough recordings used for CoughViT [2508.03764]. Self-reported metadata includes COVID-19 status, age, and comorbidities, and the dataset contains 1,572 positive cases within 14 days. The whole corpus is used for SSL pretraining, with no label reliance, and no de-duplication or additional quality filtering is reported.

Pretraining duration is typically 100 epochs, with an extended 200-epoch run evaluated in ablations and producing only marginal changes [2508.03764]. Optimizer, learning-rate schedule, batch size, training time, hardware, seeds, and code URLs are not provided, which limits strict reproducibility. A practical implication is that architectural reproducibility is stronger than training-procedure reproducibility.

## 4. Downstream tasks and empirical performance

CoughViT is evaluated on three downstream cough classification tasks: COVID-19 detection, wet-or-dry cough classification, and cough detection [2508.03764]. The COVID-19 task uses the Second DiCOVA Challenge dataset, a curated Coswara subset with 965 subjects and 172 positives, and applies 5-fold cross-validation with official train-validation splits; AUROC is the primary metric. Wet-or-dry cough classification uses COUGHVID with expert physician annotations, restricting labels to expert 4 to mitigate inter-rater variability; both cross-validation AUROC and blind test AUROC are reported. Cough detection uses the Edge-AI Cough Counting dataset, with 0.4 s intervals labeled cough versus non-cough and only the outward-facing microphone modality; evaluation includes 5-fold CV AUROC and blind test event-based and sample-based F1 using a sliding-window adaptation [2508.03764].

For downstream adaptation, the pretrained encoder is paired with a simple classifier head. Sequence representation is taken either from mean pooling over encoded patch embeddings or from the [CLS] token, followed by a fully connected layer to class logits and softmax for probability [2508.03764]. Both representations were evaluated and showed minor differences.

The principal baselines are ViT-B without pretraining, ViT-B with supervised pretraining on COVID-19 Sounds, AST without pretraining, AST with supervised pretraining on COVID-19 Sounds, and AST with supervised Audioset pretraining [2508.03764]. AST differs architecturally by using overlapping patches and learned positional embeddings, increasing token count and compute.

The reported cross-validation AUROC values are as follows:

| Task | CoughViT | AST-Audioset |
|---|---:|---:|
| COVID-19 detection | 73.21 | 70.63 |
| Cough detection | 98.25 | 98.73 |
| Wet/dry classification | 74.95 | 78.95 |

Across the full baseline set, the paper reports these AUROC values: ViT with no pretraining scores 56.19 on COVID-19, 97.24 on cough detection, and 60.06 on wet/dry; ViT with supervised COVID-19 Sounds pretraining scores 59.22, 95.89, and 58.95; AST with no pretraining scores 59.79, 97.26, and 58.23; AST with supervised COVID-19 Sounds pretraining scores 60.42, 94.37, and 55.54; AST with Audioset pretraining scores 70.63, 98.73, and 78.95; and CoughViT scores 73.21, 98.25, and 74.95 [2508.03764].

The paper highlights several comparative patterns. Relative to ViT without pretraining, CoughViT improves AUROC by +17.02 points on COVID-19, +1.01 points on cough detection, and +14.89 points on wet/dry [2508.03764]. Relative to AST-Audioset, CoughViT is +2.58 points on COVID-19, −0.48 on cough detection, and −4.00 on wet/dry. This suggests that in-domain self-supervised pretraining on cough audio can exceed large-scale supervised audio pretraining for at least some cough-centric tasks, but not uniformly across all tasks.

Blind test results sharpen that picture. On COUGHVID wet/dry classification, CoughViT attains AUROC 0.71, versus 0.56 for AST-Audioset and 0.59 for a previously reported logistic regression baseline [2508.03764]. On Edge-AI cough segmentation, AST-Audioset slightly outperforms CoughViT, with event-based and sample-based F1 of 0.71 / 0.59 versus 0.69 / 0.57 for CoughViT [2508.03764]. The results therefore do not support a blanket conclusion that CoughViT dominates all supervised baselines; rather, they show task-dependent competitiveness with clear strength in COVID-19 detection and wet/dry discrimination.

The study also reports that supervised pretraining on COVID-19 Sounds provided minimal or negative gains outside COVID-19 detection for both ViT and AST, consistent with overfitting to noisy, imbalanced labels [2508.03764]. This is an important corrective to the assumption that in-domain labels are automatically beneficial.

## 5. Ablations, representation choices, and relation to prior systems

Ablation studies identify mask ratio as a central design variable. A mask ratio of $r=0.75$ consistently outperformed $r=0.50$ across tasks; for example, COVID-19 AUROC reached up to 75.82 versus 71.95 under similar settings [2508.03764]. This is consistent with the masked autoencoder intuition that higher masking can force more informative latent representations, although the paper does not formalize that interpretation beyond the empirical observation.

Global decoder attention outperformed the local windowed alternative. The windowed self-attention variant with $4 \times 4$ windows and shifts underperformed global attention in cough spectrogram reconstruction, contrary to Audio-MAE’s finding for general audio [2508.03764]. A plausible implication is that diagnostically relevant cough structure depends on broader time-frequency dependencies rather than strictly local neighborhoods.

Sequence representation choice had little effect. Both [CLS]-based representations and mean pooling over encoded patches were viable, with only minor differences, and mean pooling slightly simplified the architecture while achieving comparable accuracy [2508.03764]. Extending pretraining from 100 to 200 epochs produced only small changes, indicating diminishing returns beyond 100 epochs.

The architecture-computation trade-off is also explicit. Non-overlapping patches in CoughViT matched or exceeded overlapping-patch AST performance while being more computationally efficient [2508.03764]. Since AST uses overlapping patches and learned positional embeddings, its token count and compute are higher. CoughViT’s design therefore combines a domain-specific pretraining signal with a lower-complexity patching scheme.

Placed in historical context, this differs from QUCoughScope in both model family and task framing. QUCoughScope used three deep CNN models ensembled in two cascaded symptomatic and asymptomatic pipelines, with breathing spectrograms more discriminative for symptomatic users and cough spectrograms used for asymptomatic users [2103.12063]. It reported asymptomatic COVID-19 sensitivity 91.49%, healthy asymptomatic specificity 97.80%, overall accuracy 95.86%, and F1 93.14% in the asymptomatic ensemble, while all signal processing and inference occurred server-side and results were returned within a couple of seconds [2103.12063]. CoughViT, by contrast, is not a deployment paper and does not report a mobile inference system; its contribution is the pretraining framework and its transfer behavior across multiple cough tasks [2508.03764].

## 6. Limitations, ethics, and open directions

Several limitations are explicit. Evaluation still relies partly on crowd-sourced, self-reported labels for some tasks, including DiCOVA and COUGHVID, so dataset bias and label noise remain relevant even when SSL is used [2508.03764]. Domain and device variability are not fully characterized: crowd-sourced recordings vary across microphones, environments, and demographics, and cross-device generalization and clinical validation remain open. SSL mitigates the observed risk of overfitting under supervised pretraining with imbalanced and noisy labels, but it does not eliminate domain shift issues [2508.03764].

Interpretability is also limited. The paper does not report attention maps or time-frequency saliency visualizations [2508.03764]. It notes that cough acoustics often contain transient bursts, broad-band energy, and possible low-mid frequency structures associated with wet coughs, and that the superiority of the global attention decoder suggests wider time-frequency context may be relevant. This suggests, rather than proves, that clinically meaningful cough cues may not be strictly localized.

Ethical considerations center on the fact that cough recordings are health-related data. Responsible data collection, consent, secure storage, and compliant deployment are described as essential [2508.03764]. Diagnostic tools require clinical validation, calibration for demographics, and careful communication to avoid harm. QUCoughScope similarly positioned its app as a pre-screening tool rather than a medical diagnosis system and emphasized anonymized uploads, while not detailing encryption or at-rest security controls [2103.12063]. Taken together, these works support a cautious interpretation: cough-audio models may provide screening or representation-learning capability, but they do not by themselves establish clinically validated diagnosis.

Future directions identified for CoughViT include broader evaluation across diseases such as asthma, COPD, TB, and pneumonia; clinically validated datasets including the UK COVID-19 Vocal Audio Dataset; domain adaptation across devices and environments; interpretability; and ensemble-based differential diagnosis using CoughViT features [2508.03764]. A practical note is that ViT’s ability to handle variable-length inputs via positional encodings simplifies transfer across cough datasets, although truncation and padding are still used for batching. This suggests that the framework is particularly suited to cough-centric tasks where labeled data is scarce or noisy and where dataset heterogeneity would otherwise increase engineering overhead [2508.03764].

Source: https://www.emergentmind.com/topics/coughvit