Safety-Score in Intelligent Systems

• Safety-score is a scalar metric that quantifies the safety performance of autonomous systems by aggregating risk-sensitive parameters from event-level interactions.
• It integrates domain-specific variables such as collision risk, detection accuracy, and system latency to provide a holistic safety assessment.
• It supports model benchmarking, procurement validation, and regulatory reporting by offering standardized safety quantification across diverse applications.

A safety-score is a scalar metric—often continuous and frequently quantitative—designed to capture the degree of safety or harm-avoidance exhibited by an autonomous or intelligent system within a specific domain. As an evaluation construct, the safety-score is operationalized via formal definitions, ground-truth referencing, or risk-weighted aggregation and is widely used for model selection, benchmarking, procurement evidence, and regulatory reporting.

1. Formal Definitions and Domain Variants

The safety-score has domain-specific instantiations, but universally seeks to encode safety-critical behaviors not captured by conventional performance metrics (e.g., accuracy, F1, BLEU). Its mathematical expression typically aggregates event-level, interaction-level, or model-output properties weighted by their safety relevance.

Canonical Definition Forms:

• Perception/Autonomous Driving: Scores aggregate detection/tracking quality, real-world relevance, and response latency, weighted by potential collision damage or injury (Volk et al., 16 Dec 2025, Gamerdinger et al., 17 Dec 2025).
• LLM Response Evaluation: Scalar or ordinal ratings (human or model-judged) of factual accuracy, harm avoidance, and refusal to comply with “risky” instructions (Tan et al., 2024, Pan et al., 10 Aug 2025, Zhang et al., 2024).
• Computing System Latency: Probability that critical system deadlines are met, given empirical or modeled latency distributions for AV perception/control (Zhao et al., 2019).
• Driver Risk and Behavior: Aggregated feature-based scores derived from trajectory, event, or crash-probability models, often normalized to [0,1] or [0,100] for comparability (Wang et al., 2018, Roy et al., 16 Mar 2026).

Illustrative Table: Safety-score Definitions

Domain	Safety-score Range	Measurement Basis
Perception (EPSM, S)	[0,1]	Weighted accuracy × safety factors
LLM Medical QA (S.C.O.R.E.)	1,5	Human expert qualitative rating
Omni-modal LLM (Omni-SafetyBench)	[0,1]	C-ASR/C-RR joint metric
VLM Red-teaming (RTVLM)	[0,10]	GPT-4-graded response evaluations
AV System Latency (Zhao et al.)	[0,1]	P(latency ≤ safety deadline)
Driver Behavior (SafeDriver-IQ)	[0,100]	Crash classifier inverse probability

2. Principal Mathematical Formulations

Safety-score metrics are grounded in formal aggregation or probability-based models:

• EPSM: Incorporates criticality and severity for detected/missed objects, applies heavy penalties for “critical misses”, fuses with lane evaluation, outputs a normalized score partitioned into qualitative categories (“insufficient” through “very good”) (Gamerdinger et al., 17 Dec 2025).
• Omni-SafetyBench: Defines

$$\text{Safety-score} = \frac{(1 - \text{C-ASR})\,(1 + \lambda\,\text{C-RR})}{1 + \lambda}$$

with $\text{C-ASR}$ (conditional attack success rate) and $\text{C-RR}$ (conditional refusal rate) measured on “understood” prompts only ($\lambda = 0.5$) (Pan et al., 10 Aug 2025).

• S.C.O.R.E. (LLM Healthcare): Expert Likert ordinal ratings, with no automated formula or sub-dimensions formally enumerated (Tan et al., 2024).
• Real-time driving (SafeDriver-IQ):

$$S_{\rm raw}(\mathbf{x}) = p_{\rm safe}(\mathbf{x}) \times 100$$

with calibrated adjustment using multiplicative literature-based penalties (e.g., weather, VRU) (Roy et al., 16 Mar 2026).

• AV System Latency Score:

$S = P[\tau \leq T_h] = F_\tau(T_h)$

where $F_\tau$ is the latency CDF and $T_h$ is the deadline (Zhao et al., 2019).

• Code Generation Safety-Utility Duality (SUDS): Piecewise function combining code utility, safety adherence (no harmful tokens), and warning awareness; weighted by domain constraints (Tan et al., 13 Apr 2026).

3. Methodologies for Scoring and Evaluation

Human Judgment Protocols:

• S.C.O.R.E. applies domain expert Likert scales (1–5), with consensus reached via adjudication for mismatches. Typical application includes two open-ended prompt sets (e.g., ophthalmology, medication) evaluated independently by a specialist and a pharmacist (Tan et al., 2024).

Automated Metrics:

• Omni-SafetyBench employs an LLM judge to assign binary labels (understand, harmful, refuse), with final Safety-score computed via conditional rates (Pan et al., 10 Aug 2025).
• DataShield computes safety-degradation scores for training samples by extracting compliance vectors at the optimal “compliance-aware” layer and measuring projection drift; correlation with GPT-4-based compliance evaluations exceeds r=0.9 (Zhang et al., 29 May 2026).
• SafeDriver-IQ runs a random forest classifier on a 64-dimensional feature vector, producing crash/no-crash probabilities per frame, then applies explicit penalty factors for high-risk conditions (Roy et al., 16 Mar 2026).

Composite and Duality Metrics:

• EPSM fuses object- and lane-level scores, adjusting for their interdependence with explicit bonus/penalty factors, and assigns the final score to risk levels (e.g., insufficient, bad, good, very good) (Gamerdinger et al., 17 Dec 2025).
• SUDS for code LLMs encodes 12 distinct (utility, safety, warning) scenarios, using constraint-driven parameterization to guarantee safety-utility tradeoff fidelity (Tan et al., 13 Apr 2026).

4. Use Cases, Empirical Benchmarks, and Applications

Healthcare LLMs: S.C.O.R.E. framework for medical chatbots uses safety ratings as one pillar, yielding average scores (ophthalmology, 5.0; medication, 4.8); “acceptable”