Early Harm Detection in Safety Systems

• Early harm detection is the process of identifying emerging harmful trajectories from partial data to trigger intervention before damage fully materializes.
• The methodology employs diverse techniques such as runtime monitoring, embedding-based classification, and threshold-triggered alerts tailored to specific domains.
• Real-world applications span malware monitoring, ICU patient care, and AI moderation, emphasizing a balance between detection timeliness, precision, and interpretability.

Early harm detection denotes the identification of harmful trajectories from partial, pre-terminal, or still-evolving evidence, with the aim of intervening before an exploit, unsafe shift, clinical event, misleading recommendation, or other damaging outcome fully materializes. In the surveyed literature, this idea appears in pre-exploitation malware monitoring, early disease diagnosis, ICU deterioration prediction, self-harm and depression risk detection, misinformation screening during an infodemic, early identification of fake social-network accounts, runtime monitoring of human-centered systems, and anticipatory safety monitoring for LLMs and reasoning traces (Yaffe et al., 2021, Prashanth, 2024, Ting et al., 2 May 2025, Li et al., 11 Jun 2025, Maity et al., 2024, Kakkar et al., 21 Apr 2026).

1. Scope and conceptual boundaries

Early harm detection is defined operationally by the temporal gap between observation and consequence. In ICU monitoring, the formulation is explicit: an Observation Window of $60$ minutes is followed by a Warning Window of $60$ minutes and then a Target Window of $30$ minutes, so the classifier must act before the Critical Health Episode occurs (Hotegni et al., 2022). In malware detection, the target is in-memory malicious activity “prior to exploitation,” using run-time logs before malware “does its dirty work” (Yaffe et al., 2021). In streaming moderation for LLMs, the goal is to stop harmful output mid-generation rather than judge only a completed response, because full detection introduces high service latency and can expose harmful content before intervention (Li et al., 11 Jun 2025). In human-centered safety-critical systems, the objective is to detect unknown errors “before unsafe shifts in operational data distribution occur,” and in reasoning-trace safety the concern is that harm unfolds through step-wise behaviors before the final answer is produced (Maity et al., 2024, Kakkar et al., 21 Apr 2026).

The harms themselves vary by domain. Some are direct and physical, such as patient deterioration, Acute Hypotensive Episodes, Tachycardia Episodes, aircraft control faults, autonomous braking failures, or lithium-ion battery degradation (Ting et al., 2 May 2025, Hotegni et al., 2022, Maity et al., 2024, Heermann et al., 17 May 2026). Others are behavioral or sociotechnical, including self-harm signals on social media, gambling addiction risk, fake-account abuse, misinformation during disease outbreaks, bias-driven downstream harms in hiring and diagnosis, and harmful or implicitly harmful LLM generations (Bucur et al., 2021, Alhassan et al., 2021, Breuer et al., 2020, Mao et al., 2024, Tantalaki et al., 27 Nov 2025, Park et al., 13 May 2026). This suggests that “early” does not name a single timescale; it names a position in a causal or operational chain.

A common misconception is that early harm detection is merely ordinary classification performed sooner. The surveyed work does not support that reduction. Several papers distinguish partial detection from full detection, or coefficient-level monitoring from output-level monitoring, or step-wise reasoning behavior from final-answer moderation, thereby treating timeliness as a first-class design variable rather than a secondary reporting metric (Li et al., 11 Jun 2025, Maity et al., 2024, Kakkar et al., 21 Apr 2026).

2. Observational substrates and representation learning

The observed substrate determines what can be detected early. In Windows malware monitoring, the substrate is the protected process environment: metadata, runtime state, and PE-specific fields. The detector samples roughly forty fields, including process ID, thread ID, OS version, integrity level, image hash, command-line arguments, register values, 128-byte memory snapshots pointed to by each register, illegal-access addresses and corresponding opcodes, lists of loaded modules/resources/handles, DEP status, injectors’ IDs, section counts, import/export hashes, entropy, timestamps, digital signatures, and PDB path. After cleaning, numeric fields are min-max normalized, textual or categorical fields are embedded with a 32-dimensional word2vec model, grouped blocks are reduced by coordinate-wise means, and the final input is a $192$-dimensional vector $\mathbf{x}\in\mathbb{R}^{192}$ (Yaffe et al., 2021). EarlyMalDetect uses a different runtime substrate: sequences of API calls. Its Phase I employs a fine-tuned GPT-2 transformer, WinApiCallSeq-FineTuneGPT2, to predict the next $m$ API calls from the first $k$ calls, and its Phase II classifies the concatenated observed-plus-predicted sequence using DistilBERT embeddings, a bidirectional GRU, attention, and a fully connected network (Maniriho et al., 2024).

Clinical and physiological systems expose another family of early substrates. For Parkinson’s disease, Prashanth et al. use preprocessed $^{123}$I-FP-CIT SPECT volumes registered to MNI space, then reduce them to either an “Average slice” over axial slices $\#35\ldots48$ or a “Peak slice” at axial slice $\#41$, with intensities scaled to $60$0, and classify with a compact two-layer CNN found by Bayesian hyperparameter optimization (Prashanth, 2024). TARL works on wearable heart-rate time series, extracting shapelets $60$1, constructing a shapelet-transition knowledge graph $60$2, learning transition-aware embeddings, and then classifying the resulting time-series representation $60$3 with XGBoost (Ting et al., 2 May 2025). CausalHealth instead uses per-cycle voltage, current, temperature, and resistance time series, applies PCMCI and KSG-based transfer entropy, and organizes twelve anomaly scores into Magnitude-shift, Predictive-residual, and Complexity-entropy bundles, with Isolation Forest reported separately (Heermann et al., 17 May 2026). In human-centered systems, the representation is even more structured: the DiH-RNN constrains hidden-to-hidden and input-to-hidden connections to the known sparsity pattern of a linear time-invariant physics model, so the learned RNN remains physics-guided at the architectural level (Maity et al., 2024).

Text, dialogue, and internal model states supply a third substrate family. The BLUE team’s eRisk systems tokenize Reddit posts with BERT WordPiece, truncate to $60$4 tokens, and apply BERT-base with task-specific heads for pathological gambling, self-harm, and depression severity (Bucur et al., 2021). “Just a Scratch” introduces CESM-100, a curated set of $60$5 emojis with contextual self-harm interpretations, and SHINES, a $60$6-post dataset annotated for binary self-harm labels and CM/SI spans; its multi-task framework enriches token representations with CESM-100 embeddings and jointly optimizes self-harm classification and span extraction (Ghosh et al., 5 Jun 2025). For streaming LLM moderation, SCM reads each new token together with the prompt and previously generated tokens, produces token-level harmfulness scores $60$7, and uses an auxiliary holistic scorer during training (Li et al., 11 Jun 2025). AERIC moves earlier still by reading hidden states from layer $60$8 of the base model, projecting them to $60$9 dimensions, and combining three linear heads with only $30$0 trainable parameters (Park et al., 13 May 2026). HarmThoughts extends the text substrate to reasoning traces themselves by annotating $30$1 sentences from $30$2 traces with a $30$3-label taxonomy of harmful, harmless, and neutral reasoning behaviors (Kakkar et al., 21 Apr 2026).

3. Objectives, decision rules, and timeliness metrics

Across the literature, the dominant supervised objective is binary cross-entropy. It is used for the LightGBM malware classifier, the Parkinson’s CNN, EarlyMalDetect’s sigmoid output, and the BERT-based early-risk classifiers (Yaffe et al., 2021, Prashanth, 2024, Maniriho et al., 2024, Bucur et al., 2021). Domain adaptation for misinformation augments source-domain cross-entropy with adversarial covariate alignment and a concept-alignment objective $30$4, while SCM combines token-level loss, response-level loss, and a logic consistency loss into

$30$5

The logic term encodes the rule that a harmful response must contain at least one harmful token, whereas a benign response must contain none (Mao et al., 2024, Li et al., 11 Jun 2025).

The decision rules are more varied than the losses. In the malware detector, inference returns $30$6 and declares maliciousness when $30$7 with $30$8 (Yaffe et al., 2021). SCM introduces naive partial detection and a Delay-$30$9 rule,

$192$0

so the alarm fires only after at least $192$1 tokens exceed threshold (Li et al., 11 Jun 2025). DiH-RNN monitoring uses conformal inference on learned coefficients: if the nonconformity score $192$2 exceeds the calibration threshold $192$3, the system signals an unknown error; the optional conformal $192$4-value triggers when $192$5 (Maity et al., 2024). AERIC combines hazard, support, and residual scores into $192$6, smooths with an EMA $192$7, and triggers when $192$8; for prompt-trigger benchmarks, $192$9 is calibrated by a source-side safe-budget rule constraining SafeTrigger to at most $\mathbf{x}\in\mathbb{R}^{192}$0 (Park et al., 13 May 2026).

The metric layer is likewise domain-specific because correctness and timeliness must be balanced rather than collapsed into a single static notion of accuracy. The eRisk tasks use ERDE and latency-weighted $\mathbf{x}\in\mathbb{R}^{192}$1, explicitly penalizing delay in post index $\mathbf{x}\in\mathbb{R}^{192}$2 (Bucur et al., 2021). ICU Critical Health Episode prediction reports Event Recall, Reduced Precision, average Anticipation Time, average False Alarms, and Event $\mathbf{x}\in\mathbb{R}^{192}$3-score, and states that a method is highly efficient only if aveAT and EF1-score are large while aveFA is low (Hotegni et al., 2022). TARL adds an earliness measure over an $\mathbf{x}\in\mathbb{R}^{192}$4 h horizon and an EE-Score defined as mean$\mathbf{x}\in\mathbb{R}^{192}$5 (Ting et al., 2 May 2025). SCM reports macro-$\mathbf{x}\in\mathbb{R}^{192}$6 together with stop-position statistics, and AERIC separates threshold-free AUROC/AUPRC from prompt-level trigger benchmarks under an explicit safe-budget constraint (Li et al., 11 Jun 2025, Park et al., 13 May 2026). This suggests that early harm detection is intrinsically multi-objective: false positives, false negatives, computational overhead, and lead time are jointly optimized rather than sequentially considered.

4. Empirical performance across major application areas

In computer-security settings, the reported results are often strongest when the observed process remains close to the substrate of the eventual attack. The in-memory malware detector trains on $\mathbf{x}\in\mathbb{R}^{192}$7 M total logs and, on a balanced $\mathbf{x}\in\mathbb{R}^{192}$8-point hold-out set at $\mathbf{x}\in\mathbb{R}^{192}$9, reports $m$0, $m$1, $m$2, $m$3, with $m$4, $m$5, $m$6, $m$7, $m$8, and $m$9 (Yaffe et al., 2021). EarlyMalDetect reports that with $k$0 initial API calls and $k$1 predicted calls, for a total length of $k$2, it achieves $k$3 accuracy and $k$4 AUC-ROC on Dataset 1; with total length $k$5, accuracy reaches $k$6 and AUC-ROC $k$7 (Maniriho et al., 2024). SybilEdge addresses a sparser graph setting and reaches $k$8 once new users issue approximately $k$9 friend-requests, outperforming graph-based baselines on Facebook (Breuer et al., 2020).

For generative AI safety, the empirical emphasis is on how much harmful output can be withheld before exposure. SCM reports macro-$^{123}$0 while seeing on average only the first $^{123}$1 of tokens, with $^{123}$2 of harmful responses stopped within the first $^{123}$3 of tokens and more than $^{123}$4 stopped within the first $^{123}$5 (Li et al., 11 Jun 2025). AERIC improves AUROC over Qwen3Guard-Stream-4B from $^{123}$6 to $^{123}$7 on DiaSafety and from $^{123}$8 to $^{123}$9 on Harmful Advice. Under the safe-budget rule, trigger@64 reaches $\#35\ldots48$0 and $\#35\ldots48$1 on HarmBench DirectRequest and SocialHarmBench for Qwen, and $\#35\ldots48$2 and $\#35\ldots48$3 for Gemma, while withholding between $\#35\ldots48$4 and $\#35\ldots48$5 answer tokens on average (Park et al., 13 May 2026). HarmThoughts, by contrast, shows that fine-grained process-level detection remains difficult: binary Harmful/Safe exceeds $\#35\ldots48$6 $\#35\ldots48$7, but full $\#35\ldots48$8-way Macro-$\#35\ldots48$9 falls to approximately $\#41$0–$\#41$1, especially for subtle emergence labels such as PS and IR and concealment labels such as CE and OB (Kakkar et al., 21 Apr 2026).

In health and safety-critical monitoring, high performance is also reported, but the notion of success is more heterogeneous. The SPECT-based Parkinson’s detector reaches $\#41$2 accuracy and $\#41$3 AUC for PD vs. normal with the single-slice CNN, and identifies $\#41$4 SWEDD cases as non-PD, corresponding to $\#41$5 accuracy and a $\#41$6 false-positive rate for SWEDD (Prashanth, 2024). DiH-RNN plus coefficient-conformal inference detects $\#41$7 of AID blockage scenarios exactly at the moment of error onset, $\#41$8 MCAS Angle-of-Attack faults at error onset, and $\#41$9 autonomous-braking controller-fault runs before collision, with $60$00 false alarms at $60$01 over $60$02 held-out error-free runs (Maity et al., 2024). TARL reports best single numbers of accuracy $60$03, recall $60$04, $60$05, average earliness $60$06, and an EE-Score of $60$07, corresponding to approximately $60$08 min lead time on an $60$09 h window (Ting et al., 2 May 2025). MIG-LightGBM reports, for Tachycardia Episode prediction, $60$10, $60$11, aveFA $60$12, aveAT $60$13 min, and $60$14, outperforming XGBoost, SVC, Naive Bayes, and Layered Learning under the reported protocol (Hotegni et al., 2022).

Social and sociotechnical problems show a wider trade-off surface. In eRisk, the BLUE team’s pathological-gambling run attains recall $60$15 with precision $60$16, $60$17, ERDE$60$18, ERDE$60$19, and delay$60$20 posts; the self-harm run reaches precision $60$21, recall $60$22, $60$23, and delay$60$24 posts (Bucur et al., 2021). “Just a Scratch” reports, for Llama-3, an improvement from $60$25 and AUC $60$26 in fine-tuned SHC-only mode to $60$27 and AUC $60$28 in the multi-task setting with CESM-100, with CMSE/SISE span extraction around $60$29 $60$30 (Ghosh et al., 5 Jun 2025). For early infodemic management, DACA achieves recall $60$31 and $60$32 in the English full-data scenario, exceeding misinformation-specific and general-purpose domain-adaptation baselines, and preserves its leading rank even when target comments are unavailable (Mao et al., 2024).

5. Deployment architectures, interpretability, and intervention logic

A notable property of early harm detection systems is that deployment architecture is often as important as model choice. The Windows malware detector uses a two-component design: an on-premises Agent that snapshots a protected process’s environment at user-configurable intervals and a cloud Detector that stores the model. The endpoint Agent never uses more than $60$33 of one logical core and under $60$34 MB of RAM, streams JSON logs typically smaller than $60$35 KB, allows offline caching, and leaves model updates server-side (Yaffe et al., 2021). SCM is described as a lightweight, plug-and-play module running in parallel with any autoregressive LLM, and because its feature extractor is incremental, it adds only a few ms per token instead of the hundreds of ms associated with full re-encoding (Li et al., 11 Jun 2025). AERIC similarly treats deployment efficiency as a design constraint: the monitor artifact is approximately $60$36 MiB for Qwen and $60$37 MiB for Gemma, and fixed-generation latency rises by only $60$38, compared with $60$39 for Qwen3Guard-Stream-4B on the reported benchmark (Park et al., 13 May 2026).

Interpretability is pursued in different ways. TARL explains alerts through recent shapelet transitions; in the case study, the transition $60$40 is associated with rising deterioration risk, and the subsequent appearance of $60$41 reinforces the alarm, enabling the model to state that the pattern $60$42 in under $60$43 min is a known early sign of deterioration (Ting et al., 2 May 2025). CausalHealth grounds its anomaly scores in causal-graph and transfer-entropy quantities, and validates $60$44 against electrochemical impedance spectroscopy via pooled correlation $60$45 with $60$46 and a temperature-controlled partial correlation $60$47, supporting a physically interpretable relation to charge-transfer resistance (Heermann et al., 17 May 2026). In language safety, “Just a Scratch” generates rationales conditioned on extracted CM/SI spans and emoji interpretations, while HarmThoughts supplies a behavioral taxonomy that localizes harmful drift into Refusal Suppression, Constraint Reframing, Task Decomposition, Domain Knowledge Elicitation, Obfuscation, and related categories (Ghosh et al., 5 Jun 2025, Kakkar et al., 21 Apr 2026).

Intervention policies also vary widely. SCM triggers immediate stopping of the LLM once $60$48 (Li et al., 11 Jun 2025). EarlyMalDetect is framed as a preventive measure that can halt or quarantine a process before its malicious payload executes (Maniriho et al., 2024). DiH-RNN monitoring supports immediate pilot alert and reversion to manual control in the MCAS scenario, and collision prevention in autonomous braking (Maity et al., 2024). Social-media systems are more cautious: the eRisk paper explicitly states that low precision makes real-world deployment with human-in-the-loop verification necessary (Bucur et al., 2021). At the earliest stage of the AI lifecycle, ECHO shifts intervention upstream by mapping bias types to harms through stakeholder identification, vignette construction, dual human–LLM annotation, and descriptive and inferential ethical matrices, thereby supporting design and governance decisions before system deployment (Tantalaki et al., 27 Nov 2025).

6. Limitations, misconceptions, and open research directions

A recurrent limitation is that high recall does not by itself constitute effective early warning. The pathological-gambling system reports recall $60$49 but precision $60$50, and the paper states that human-in-the-loop verification is necessary (Bucur et al., 2021). In ICU prediction, Naive Bayes attains very high Event Recall for Acute Hypotensive Episodes, $60$51, but its Reduced Precision is $60$52 and aveFA is $60$53, which the paper treats as clinically impractical (Hotegni et al., 2022). This directly counters the misconception that earlier alarms are always preferable; early alarms must still respect alarm burden and false-positive constraints.

Another persistent issue is shift, evasion, and incomplete specification. The malware paper anticipates that future sophisticated malware may try to bypass detection and proposes periodic randomization of feature-extraction intervals, ensembling with an unsupervised anomaly detector, encrypted attestation channels, tamper-proof logging, and incremental online re-training (Yaffe et al., 2021). DACA argues theoretically and empirically that covariate shift alone is insufficient, and that concept shift between source and target domains must also be mitigated (Mao et al., 2024). AERIC notes that current results cover only two model families and one layer, and that calibration is sensitive to the safe-budget choice (Park et al., 13 May 2026). CausalHealth states explicitly that statistical causality does not prove mechanistic causation and that wider EIS replication is needed (Heermann et al., 17 May 2026). This suggests that early harm detection is often limited less by raw pattern recognition than by the fragility of assumptions linking present signals to future harms.

Reporting gaps and domain specificity also remain substantial. The Parkinson’s CNN paper does not report optimizer settings, learning-rate schedules, epoch counts, or class-weighting strategies (Prashanth, 2024). The Twitter self-harm study by Alhassan et al. does not include a supervised classifier, probability thresholds, or precision/recall/$60$54, even though it supplies topic, sentiment, and engagement analyses that could support future early-warning systems (Alhassan et al., 2021). “Just a Scratch” notes platform bias, cultural variability of emoji meaning, and retained pretraining biases in LLMs (Ghosh et al., 5 Jun 2025). ECHO argues that technical fixes applied after development offer limited preventive value and instead advocates anticipatory governance through stakeholder-centered harm mapping (Tantalaki et al., 27 Nov 2025).

The surveyed future directions therefore converge on three fronts. First, richer supervision is needed for partial and process-level settings, as shown by FineHarm, SHINES, and HarmThoughts (Li et al., 11 Jun 2025, Ghosh et al., 5 Jun 2025, Kakkar et al., 21 Apr 2026). Second, monitoring is moving inward, from outputs to coefficients and hidden states, as in DiH-RNN and AERIC (Maity et al., 2024, Park et al., 13 May 2026). Third, explanation and governance are becoming integral rather than auxiliary, whether through shapelet-transition narratives, transfer-entropy grounding, rationale generation, or ethical matrices (Ting et al., 2 May 2025, Heermann et al., 17 May 2026, Tantalaki et al., 27 Nov 2025). A plausible implication is that the mature form of early harm detection will not be a single detector family, but a layered stack combining anticipatory representations, calibrated stopping or alerting rules, and domain-appropriate human or automated intervention.