--- title: 'CPI+DMC: Modular Early Risk Detection' url: https://www.emergentmind.com/topics/cpi-dmc-approach type: topic --- # CPI+DMC: Modular Early Risk Detection The CPI+DMC approach is a modular two-stage framework for sequential decision-making over temporally unfolding data, originally motivated by Early Risk Detection (ERD) problems such as predicting the onset of mental health conditions from users’ social media streams. It partitions the detection process into a Classification with Partial Information (CPI) stage, which generates an updated probability of a target class as evidence accrues, and a Decision-Making Component (DMC), which determines the optimal stopping time to issue an alert. The paradigm enables principled trade-offs between prediction accuracy and detection earliness and allows for independent optimization, deployment, and analysis of the classification and decision rules. This separation of concerns has been empirically validated on recent ERD benchmarks, and has led to state-of-the-art results in shared tasks including eRisk 2024 and MentalRiskES 2025 [2410.17963][2511.23325]. ## 1. Conceptual Framework and Motivation CPI+DMC decomposes the ERD pipeline into two distinct components: 1. **Classification with Partial Information (CPI):** At each time step $d$ (after observing $d$ user posts), CPI outputs a score $p_u(d) = \Pr(y_u=1\,|\,t_1,\ldots,t_d)$ for user $u$. CPI is trained to maximize discriminative accuracy under incomplete, evolving input. 2. **Decision-Making Component (DMC):** Receiving the stream of CPI scores, DMC applies an online policy—a (typically non-decreasing) function of $p_u(\cdot)$ or other derived statistics—to declare at what $d$ to make an irreversible prediction (issue an alert), explicitly balancing the trade-off between accuracy and earliness. The approach is motivated by the structurally antagonistic objectives inherent in ERD: achieving a high true positive rate (CPI) and minimizing the delay before a correct alert is issued (DMC). The modular design enables independent design and tuning of each process to explore and navigate the Pareto frontier in the accuracy-time plane [2511.23325]. ## 2. Mathematical Formulation of CPI+DMC Let $u$ index users and $t_1,\ldots,t_{T_u}$ their chronologically ordered posts. **CPI:** For each delay $d \leq T_u$, CPI computes\[ p_u(d) = \Pr(y_u = 1\,|\,t_1,\ldots,t_d) \]where $y_u \in \{0, 1\}$ indicates high risk. **DMC:** DMC implements a stopping rule to decide the earliest $d=d_u$ at which to make a prediction $\hat{y}_u \in \{0, 1\}$. Common policy families include: - **Threshold-count policy** (BERT/SBERT): Declare $\hat{y}_u=1$ at the first $d$ where\[ \sum_{i=1}^{d} \mathbf{1}\{ p_u(i) \geq \tau \} \geq T \] with fixed threshold $\tau$ and integer counter $T$. - **Adaptive global policy** (SS3): Compute per-user global values $cv^+_u(d)$ and $cv^-_u(d)$, transform to softmax score $s_u(d)$, and compare to a distributional threshold based on the batch’s median and median absolute deviation (MAD) [2511.23325]. ## 3. Implementation: Model Variants and Decision Policies ### CPI Instantiations | CPI Model | Feature Set | Training Objective | |----------------------------|------------------------------------------------------|----------------------| | SS3 | Character trigram frequencies, global-value function | Static F$_1$ | | BERT (with domain tokens) | Sliding post windows, extended WordPiece vocabulary | Static F$_1$ | | SBERT-SetFit | Sentence-pair contrastive embeddings | Macro F$_1$ | ### DMC Policies | Policy Name | Mechanism | Tuning Parameter(s) | |-----------------------|------------------------------------|-----------------------| | Global (SS3) | Adaptive threshold via MAD | $\gamma$ | | History-based | Repeated high-confidence count | $\tau, T$ | DMC policies are selected for their responsiveness to different operational imperatives. The global policy exploits inter-user score distributions for cohort-relative detection; the history-based rule enforces temporal smoothing and consistency in evidence [2511.23325]. ## 4. Metrics and Training Regimes **CPI** is evaluated by static or dynamic F$_1$, accuracy, or macro/micro-averaged scores. **DMC** is evaluated using time-aware metrics penalizing late detection. The ERDE$_\theta$ family is canonical: \[ \mathrm{ERDE}_\theta(d_u) = \begin{cases} 0 & \text{if } y_u=1, d_u\leq\theta \\ 1 - \theta/d_u & \text{if } y_u=1, d_u>\theta \\ 1 & \text{for FP or FN} \end{cases} \] and related quantities such as $F_\mathrm{latency}$ and mean latency of true positives [2410.17963]. Some models (e.g., the time-aware BERT variant in eRisk 2024) directly optimize ERDE$_{50}$ (with a weighting term alongside cross-entropy) via backpropagation, using explicit temporal tokens (“[TIME]”) in the input to encode progression through the stream [2410.17963]. ## 5. Empirical Results and Trade-offs CPI+DMC was the foundation for the UNSL group’s submissions in eRisk 2024 (anorexia detection) and MentalRiskES 2025 (gambling disorder) shared tasks. - In eRisk 2024, two-stage CPI+DMC architectures and a time-aware variant achieved second place in the official ERDE$_{50}$ metric, with ERDE$_{50}=0.03$ versus a system-wide mean of 0.07, and consistently placed at or near the top in ranking-based metrics (P@10, NDCG@10/100) [2410.17963]. - In MentalRiskES 2025, two of three CPI+DMC instantiations (SS3+Global, SBERT+History) occupied the top positions for Macro F$_1$ (0.567 and 0.563 respectively) out of 38 submissions, displaying complementary strengths: SS3 favored earliness at a cost of higher false positives, while SBERT achieved the highest overall F$_1$ at slightly later alert times [2511.23325]. These results support the practical value of optimizing CPI and DMC independently, enabling flexible adaptation to mission requirements (earliness vs. certainty). ## 6. Strengths, Limitations, and Extensions **Strengths:** - Explicit modularity enables independent optimization, plug-and-play combinations, and systematic tuning of the accuracy-timeliness trade-off [2511.23325]. - Both simple (SS3) and parametric (transformer) classifiers can be paired with decision policies tailored for responsiveness or conservatism. - Empirical evidence demonstrates upper-bound state-of-the-art performance on several ERD tasks. **Limitations:** - The strict separation may forego optimal solutions in cases where joint end-to-end training can yield better performance; current variants do not jointly train CPI and DMC except for specialized time-aware models which inject earliness into the loss [2410.17963]. - Discriminative power is limited by the lexical and semantic overlap between class labels in some corpora, leading to ambiguous or noisy predictions [2511.23325]. - Current metrics may inadequately capture “borderline” cases; adaptive and per-user policies are signaled as an important future direction. Future research is indicated in adaptive metrics/policies (e.g., per-user ERDE$_{\theta_u}$), rich expert-grounded reasoning for model interpretability, and joint optimization of CPI and DMC objectives [2511.23325]. ## 7. Applications and Outlook While originally targeted at sequential health risk detection in natural language user streams, the CPI+DMC approach is generalizable to any setting requiring online decision-making with streaming, incomplete data. The paradigm’s modularity, principled metrics, and success in competitive evaluations establish it as a standard protocol for early detection tasks in computational social science and clinical informatics. A plausible implication is that further extensions—incorporating more sophisticated policies, adaptive user-specific thresholds, or chain-of-thought reasoning—will incrementally advance both the interpretability and reliability of such early warning systems [2410.17963][2511.23325]. Source: https://www.emergentmind.com/topics/cpi-dmc-approach