PROCESS Challenge: Cognitive Decline Detection

• PROCESS Challenge is a benchmark for early detection of cognitive decline and dementia, using spontaneous speech and neurologist-designed prompts.
• It features a specially designed corpus with semantic, phonemic, and narrative tasks to probe varied cognitive functions.
• Submissions utilize both acoustic and linguistic analyses through multimodal fusion and ensemble techniques to improve diagnostic accuracy.

Searching arXiv for papers explicitly using the term “PROCESS Challenge” to ground the article. {"query":"\"PROCESS Challenge\" arXiv", "max_results": 10, "sort_by": "submittedDate"} The PROCESS Challenge most explicitly denotes the Prediction and Recognition of Cognitive Decline through Spontaneous Speech Signal Processing Grand Challenge, a benchmark centered on automatic detection of early-stage cognitive decline and dementia from spontaneous speech (Tao et al., 2024). In the recent arXiv literature, it is characterized by a newly released speech corpus, three neurologist-designed elicitation prompts, and two predictive tasks tied to diagnosis and cognitive status estimation. The term is not fully univocal across domains, however: the same phrase also appears in procedural text comprehension and in process-modeling literature, where it refers to distinct problem settings rather than the speech benchmark itself (Mishra et al., 2018).

1. Definition and task formulation

The speech-oriented PROCESS Challenge was launched to stimulate research into the automatic detection of early-stage cognitive decline and dementia from spontaneous speech (Tao et al., 2024). Its core benchmark couples spontaneous speech elicitation with supervised prediction tasks and challenge-style evaluation. The organizer description specifies two complementary tasks: a binary classification task, in which systems distinguish between healthy controls and individuals showing signs of mild cognitive impairment (MCI) or dementia, and a regression task, in which systems predict each speaker’s Mini Mental State Examination (MMSE) score (Tao et al., 2024). By contrast, later challenge analyses and submissions describe the classification task as a three-way problem over Healthy Control, Mild Cognitive Impairment, and Alzheimer’s Dementia (Botelho et al., 2024, Devahi et al., 2 Oct 2025). This discrepancy is part of the published record and should be kept in mind when comparing reported scores.

Ranking criteria were standardized. Participants were free to submit up to three models per task and were ranked according to macro-averaged F1-score for classification and root-mean-square error (RMSE) for regression (Tao et al., 2024). A later challenge analysis restates the classification metric set as macro-averaged $F_1$, precision, and recall, with RMSE for regression, and reports the challenge’s top test-set scores as $0.696$ for classification and $2.459$ for MMSE prediction (Devahi et al., 2 Oct 2025).

The challenge is situated within a broader clinical objective: speech is treated as a non-invasive biomarker for cognitive decline. The benchmark design therefore emphasizes spontaneous, clinically motivated speech production rather than tightly scripted utterances (Tao et al., 2024).

2. Corpus design and elicitation protocol

The benchmark corpus was designed around three prompts selected by neurologists to tap distinct cognitive and linguistic functions (Tao et al., 2024). All three prompts are one-minute spontaneous speech tasks.

Prompt	Wording	Targeted functions
Semantic fluency	“Please name as many animals as you can in a minute.”	Semantic memory, lexical access, language organization
Phonemic fluency	“Please say as many words beginning with the letter ‘P’ as you can. Any word beginning with ‘P’ except for names of people such as Peter or countries such as Portugal.”	Phonemic fluency, executive control, working memory
Cookie Theft description	Cookie Theft picture description	Spontaneous narrative production, syntactic complexity, discourse coherence, memory recall

The organizer description states that speech was collected remotely via the Cognospeak platform, with training and development splits including manual transcripts and ground-truth labels for diagnosis and MMSE, while the independent test set contained audio only (Tao et al., 2024). It also states that recordings were in English and captured using standard consumer-grade microphones under unsupervised home conditions. A later challenge analysis reports a total of 157 participants, comprising 59 MCI, 82 HC, and 16 AD, with a training set of 117 and a development set of 40, and describes the recordings as single-speaker WAV files from a quiet room with sampling at least 16 kHz (Devahi et al., 2 Oct 2025). These descriptions are not identical; they represent different published characterizations of the same benchmark.

The prompt design reflects clinically differentiated cognitive probes. Semantic fluency is described as sensitive to early-stage Alzheimer’s pathology and semantic network integrity; letter-based fluency loads on frontal executive systems; Cookie Theft picture description engages multiple linguistic and cognitive subsystems and is described as a gold standard in dementia research (Tao et al., 2024). This division of labor across prompts is central to later multimodal fusion systems, which exploit complementary signal from fluency and narrative production rather than treating the tasks as interchangeable (Botelho et al., 2024).

3. Baseline representations, models, and metrics

The organizer baselines combine acoustic and linguistic pipelines (Tao et al., 2024). For acoustic modeling, OpenSMILE’s eGeMAPS feature set converts each recording into a sequence

$X = \{x_1, x_2, \ldots, x_T\}$

of feature vectors, followed by utterance-level aggregation through statistics such as mean, variance, and percentiles. The cited low-level descriptors include $F0$, spectral flux, MFCCs, formant frequencies, jitter, shimmer, and loudness (Tao et al., 2024).

For linguistic modeling, audio is transcribed with Whisper ASR and then encoded with a pretrained RoBERTa transformer. The final hidden state corresponding to the [CLS] token is used as a 768-dimensional summary embedding (Tao et al., 2024). Baseline acoustic classifiers include a linear-kernel Support Vector Classifier with $C=1.0$ and a Random Forest Classifier with 100 trees and Gini impurity; acoustic regressors include SVR with $\epsilon=0.1$ and $C=1.0$ and a Random Forest Regressor with 100 trees. Text baselines use RoBERTa with linear prediction heads, fine-tuned for three epochs in classification and with AdamW for regression (Tao et al., 2024).

The benchmark metrics are explicitly defined. For regression,

$$RMSE = \sqrt{\frac{1}{N} \sum_i (y_i - \hat{y}_i)^2},$$

where $N$ is the number of test samples, $0.696$0 the true MMSE, and $0.696$1 the predicted score (Tao et al., 2024). For classification, macro-$0.696$2 is the unweighted mean of per-class $0.696$3 scores, with

$0.696$4

The baseline results are notable because they do not uniformly favor the largest neural text model. The best reported classification baseline is $0.696$5 F1 using SVC on combined semantic and phonemic fluency prompts; Cookie Theft alone also yields $0.696$6 with SVC and $0.696$7 with RFC, whereas the RoBERTa-based classifier plateaus at approximately $0.696$8 F1 (Tao et al., 2024). For regression, the best baseline RMSE is $0.696$9 with RoBERTa on both VF and VF+Cookie Theft, while acoustic RF regression on VF+Cookie Theft reaches $2.459$0 (Tao et al., 2024). The papers further report that semantic and phonemic fluency tasks produced the most discriminative acoustic signatures for early cognitive decline, while picture description contributed more to MMSE regression when combined with fluency (Tao et al., 2024).

4. Submission systems and feature engineering

Challenge submissions substantially broadened the representational space beyond the organizer baselines. One submission by Botelho et al. adopts a holistic approach spanning knowledge-based acoustic and text-based feature sets, LLM-based macrolinguistic descriptors, pause-based acoustic biomarkers, and neural embeddings from LongFormer, ECAPA-TDNN, and TRILLsson (Botelho et al., 2024). The submission uses nearly one hour of spontaneous speech per subject across Cookie Theft Description, Phonemic Fluency Task, and Semantic Fluency Task, with an initial energy-based de-noising step replacing high-energy non-speech events with low-level Gaussian noise (Botelho et al., 2024).

That system is technically heterogeneous. It extracts eGeMAPS, ComParE 2013, and Praat-derived features; defines pause statistics through quantities such as average pause duration $2.459$1, standard deviation $2.459$2, and pause-to-speech ratio

$2.459$3

and augments them with percentile markers, speech rate, and articulation rate (Botelho et al., 2024). On the text side, it uses 11-dimensional linguistic measures, 8-dimensional fluency-specific features, and a four-dimensional LLM “macrodescriptor” vector

$2.459$4

for global coherence, local coherence, informativeness, and tangentiality, scored by Llama-3.1-70B-Instruct (Botelho et al., 2024). The submission then constructs late-fusion ensembles over qualified single systems, using sparse multinomial logistic regression on stacked soft outputs (Botelho et al., 2024).

A second challenge analysis emphasizes a different bimodal design. It extracts Whisper embeddings from Cookie Theft audio by mean-pooling penultimate encoder states

$2.459$5

with no further normalization beyond the model’s internal layer norm (Devahi et al., 2 Oct 2025). In parallel, it defines 14 linguistic features per task, concatenated to 42 dimensions across Semantic Fluency, Phonemic Fluency, and Cookie Theft. These include pronoun ratio, definite and indefinite NP proportions, total NP rate, filler-word rate, total word-count rate, active interaction ratio, adverbial adjunct ratios, total clause rates, adjunct-clause ratios, and utterance duration (Devahi et al., 2 Oct 2025).

These submissions illustrate a recurring methodological pattern in the challenge literature: simple and interpretable descriptors are not discarded in favor of purely end-to-end systems, but are instead combined with high-capacity embeddings and fusion layers (Botelho et al., 2024, Devahi et al., 2 Oct 2025).

5. Empirical outcomes and methodological lessons

The strongest reported challenge-style results come from ensembles and multimodal systems rather than from any single representation. Botelho et al. evaluate approximately 10,000 ensemble combinations and select two six-system ensembles; on blind test data, the two systems reach $2.459$6 and $2.459$7 macro-$2.459$8, respectively (Botelho et al., 2024). Their development-set performance had been higher, with Ensemble #1 at $2.459$9 UAF1 and Ensemble #2 at $X = \{x_1, x_2, \ldots, x_T\}$0, both with nonzero dementia-class performance (Botelho et al., 2024). A different submission reports its best classification test result as Macro-$X = \{x_1, x_2, \ldots, x_T\}$1 and its best regression test result as $X = \{x_1, x_2, \ldots, x_T\}$2, ranking 34/106 for classification and 15/80 for regression (Devahi et al., 2 Oct 2025).

Several technical lessons recur across papers. First, simple interpretable knowledge-based features remain highly informative, especially pause statistics combined with LLM-derived macrodescriptors (Botelho et al., 2024). Second, neural acoustic embeddings such as ECAPA-TDNN and TRILLsson capture complementary paralinguistic cues across tasks (Botelho et al., 2024). Third, linguistic features appear especially effective for categorical discrimination among HC, MCI, and AD, while audio embeddings can be more competitive for continuous MMSE prediction (Devahi et al., 2 Oct 2025).

The literature also records important limitations. Reported concerns include unknown participant counts in the organizer description, limited language diversity, ASR errors for older speakers with atypical prosody or articulation, the possibility that conventional eGeMAPS misses fine-grained acoustic markers such as dysfluencies and pause distributions, and the tendency of simple linear or tree-based models to underutilize temporal dynamics in spontaneous speech (Tao et al., 2024). Later analyses add that file-level features lose temporal structure, aggressive VAD may discard diagnostically relevant pause statistics, CTD-only audio is restrictive, and cohort size and MMSE availability limit generalizability (Devahi et al., 2 Oct 2025). This suggests that future progress is likely to depend on better temporal modeling, richer multimodal fusion, and broader validation settings rather than on a single dominant feature family.

6. Other uses of the term in arXiv literature

The term “PROCESS Challenge” is not unique to the dementia-detection benchmark. In NLP, “Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension” introduces ProPara, a dataset of 488 paragraphs describing 183 unique processes with 81,345 annotated cells for entity existence and location tracking across paragraph steps (Mishra et al., 2018). That work defines input as a paragraph of ordered sentences and a set of participant entities, and asks models to predict whether each entity is in state “not exist,” “unknown,” or a location span at each step. It evaluates systems through category-specific accuracy and set-$X = \{x_1, x_2, \ldots, x_T\}$3, and reports PROGLOBAL at $X = \{x_1, x_2, \ldots, x_T\}$4 macro-average and $X = \{x_1, x_2, \ldots, x_T\}$5 micro-average, versus a human upper bound of $X = \{x_1, x_2, \ldots, x_T\}$6 and $X = \{x_1, x_2, \ldots, x_T\}$7 (Mishra et al., 2018). In that context, the “challenge” concerns procedural text comprehension and implicit state-change reasoning, not clinical speech assessment.

The phrase also appears in process-modeling literature, but there it functions differently. “Process Query Language: Design, Implementation, and Evaluation” presents PQL as a behavioral query language over process model repositories and states that it directly tackles requirements for decidability, behavioral expressiveness, and scalability associated with a “PROCESS Challenge” framing (Polyvyanyy et al., 2019). “Change Patterns in Use: A Critical Evaluation” uses the phrase in a concluding architectural sketch proposing pre/post-condition-grounded pattern application, preview and look-ahead support, and pattern–primitive integration for correctness-by-construction process modeling (Weber et al., 2015). These are distinct from the speech benchmark and from ProPara.

Because of this terminological overlap, literature searches for “PROCESS Challenge” require domain disambiguation. In current arXiv usage, the phrase may refer to at least three different research objects: a spontaneous-speech benchmark for dementia-related prediction, a procedural-text comprehension dataset and model suite, and challenge-framed requirements in process and business-process-management research (Tao et al., 2024, Mishra et al., 2018, Polyvyanyy et al., 2019).