Safety Compliance: Norms, Metrics & Methods

• Safety compliance is the adherence to safety norms and standards, verified through traceable metrics and auditable procedures.
• It is quantified using diverse metrics—such as binary rule satisfaction, graded response quality, and risk-weighted aggregation—to assess real-time compliance.
• Implementation relies on rule-grounded reasoning, evidence-based retrieval, and controller-level enforcement to ensure practical, measurable safety behavior.

Searching arXiv for the cited safety-compliance papers to ground the article in current research. Safety compliance denotes the state in which a system, organization, or operational process conforms to safety-relevant rules, standards, regulations, or declared framework commitments, and can demonstrate that conformity through measurable behavior, traceable evidence, and auditable procedures. In the surveyed literature, the term spans multiple levels of analysis: response-level refusal or assistance in LLMs, adherence to public regulations and private standards in food supply chains, evidence-grounded decision support in unmanned aircraft regulation, legal-rule application in high-risk AI systems, constraint satisfaction in autonomous driving and human–robot collaboration, and organizational adherence to frontier-AI safety frameworks (Bugaud, 1 Jun 2026, Tran, 2018, Immordino et al., 16 Feb 2026, Kelly et al., 2024).

1. Terminology and scope

Safety compliance is not defined uniformly across domains, but the surveyed works converge on a common structure: a governing norm, an observable behavior, and an evidentiary basis for deciding whether the behavior conforms. In open-weight LLM safety research, safety compliance means that the model assists with a harmful request under the operational framing rather than refusing; a response is counted as compliant if it provides substantive, actionable guidance with harmful-guidance score at least $3/5$, while strong refusal is defined by refusal-strength at least $4/5$ (Bugaud, 1 Jun 2026). In frontier-AI governance, compliance means consistently doing what a company’s safety framework says it will do, and a third-party compliance review is an independent assessment of whether the organization adheres to its own safety framework rather than whether the framework is itself adequate (Homewood et al., 3 May 2025).

In supply-chain research, food quality and safety compliance is adherence by food supply chain actors to public regulations and private standards that set hygiene, safety, and quality requirements for products and processes “from farm to fork,” while compliance risk is the probability that a product or service fails to meet the applicable regulations or standards (Tran, 2018). In UAS regulatory support, safety compliance is approached through a retrieval-augmented assistant that supports safety assessment, certification activities, and regulatory compliance while remaining strictly within an authoritative corpus and preserving human responsibility for critical conclusions (Immordino et al., 16 Feb 2026). In high-risk AI under the EU AI Act, compliance is framed as a measurable, supply-chain-spanning process that connects Articles 9–15 and Annex IV obligations to product-quality attributes, metrics, tests, and evidence (Kelly et al., 2024).

Several papers make the definition explicitly formal. “Context Reasoner” models contextual safety and privacy compliance through Contextual Integrity with context

$\mathcal{C} := \langle S, S_d, R, I, P\rangle,$

and flow compliance

$$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$

so that a model’s output is compliant only if an appropriate transmission principle authorizes the information flow (Hu et al., 20 May 2025). In highway automated driving, compliance means that both the vehicle state and the reference trajectory it is about to track satisfy applicable traffic-law constraints such as speed, following distance, lane-changing, and overtaking rules (Ma et al., 2023). In ISO 10218-compliant human–robot collaboration, compliance is encoded as preservation of a protective separation distance through a Control Barrier Function safe set $h \ge 0$, where $h = d_{min} - C$ (Parma et al., 11 Jun 2026).

This range of definitions shows that safety compliance is not merely a synonym for safety performance. A plausible implication is that compliance is best understood as norm-constrained safety behavior accompanied by traceability: the system must not only avoid unacceptable outcomes, but do so in a way that can be related back to a recognized rule set or safety commitment.

2. Measurement, metrics, and evidentiary criteria

The literature uses a notably diverse measurement apparatus, but most metrics resolve to one of four forms: binary rule satisfaction, graded response quality, risk-weighted aggregation, or thresholded physical safety margins. In the domain-dependent LLM study, the operational compliance rate for domain $d$ is

$r_d = \frac{k_d}{n_d},$

where $k_d$ counts operational responses with harmful-guidance score at least $3$, and $4/5$0 is the total operational responses in that domain. The same work defines percentage-point differences $4/5$1, within-domain heterogeneity

$4/5$2

and cluster-bootstrapped confidence intervals

$4/5$3

Using this setup, compliance ranged from $4/5$4 in human trafficking to $4/5$5 in surveillance design, with a $4/5$6-percentage-point span and non-overlapping cluster-bootstrapped $4/5$7 confidence intervals; within-domain heterogeneity reached $4/5$8 percentage points in labor scenarios (Bugaud, 1 Jun 2026).

Finance-oriented evaluation introduces a different compliance metric, the Harmful Instruction Compliance Score, or HICS, on a $4/5$9–$\mathcal{C} := \langle S, S_d, R, I, P\rangle,$0 scale. The benchmark defines $\mathcal{C} := \langle S, S_d, R, I, P\rangle,$1–$\mathcal{C} := \langle S, S_d, R, I, P\rangle,$2 as “Successful Defense,” $\mathcal{C} := \langle S, S_d, R, I, P\rangle,$3–$\mathcal{C} := \langle S, S_d, R, I, P\rangle,$4 as “Minor Failure,” $\mathcal{C} := \langle S, S_d, R, I, P\rangle,$5–$\mathcal{C} := \langle S, S_d, R, I, P\rangle,$6 as “Clear Failure,” $\mathcal{C} := \langle S, S_d, R, I, P\rangle,$7–$\mathcal{C} := \langle S, S_d, R, I, P\rangle,$8 as “Severe Failure,” and $\mathcal{C} := \langle S, S_d, R, I, P\rangle,$9–$$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$0 as “Catastrophic Failure”; “Attack Failed” is assigned when $$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$1, “Attack Partially Successful” when $$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$2, and “Attack Successful” when $$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$3 (Ding et al., 10 Dec 2025). The same benchmark reports a capability–compliance gap, with models averaging $$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$4 on capability tasks but $$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$5 on compliance and risk-control evaluations, and notes that refusal alone is not a reliable proxy for safety without cited and verifiable reasoning (Ding et al., 10 Dec 2025).

Several works measure compliance through aggregate scoring structures. Third-party frontier-AI reviews propose weighted control compliance

$$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$6

a confidence-weighted variant

$$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$7

objective-level aggregation

$$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$8

and maturity scoring

$$F \vdash \mathcal{C} \Leftrightarrow \forall (s,s_d,r,i)\in F,\ \exists p\in P : p(s,s_d,r,i)=\text{true},$$9

with examples such as requiring $h \ge 0$0 and no key process below $h \ge 0$1 for go-live (Homewood et al., 3 May 2025). High-risk AI compliance under the EU AI Act likewise translates legal obligations into measurable attributes such as Balanced Accuracy, Expected Calibration Error, drift metrics such as PSI and $h \ge 0$2, human-oversight latency $h \ge 0$3, intervention success rate, and traceability coverage $h \ge 0$4 (Kelly et al., 2024).

Physical systems use explicitly operational metrics. Highway automated driving defines compliance rate

$h \ge 0$5

and active violation proportion as the proportion of violation time steps caused by the ego vehicle’s own initial reference or intent; the proposed framework raised overall compliance from $h \ge 0$6 to $h \ge 0$7 and reduced active violations to $h \ge 0$8 (Ma et al., 2023). Hand–arm vibration monitoring uses ISO 5349-1 frequency-weighted RMS acceleration and daily exposure

$h \ge 0$9

and compares the resulting values against the EU Directive 2002/44/EC Exposure Action Value $h = d_{min} - C$0 and Exposure Limit Value $h = d_{min} - C$1 (Mootz et al., 20 Sep 2025).

What unifies these otherwise heterogeneous metrics is their insistence on operationalizability. Compliance is not treated as an impressionistic property; it is scored, thresholded, stress-tested, and usually tied to explicit acceptance criteria.

3. Methods for achieving compliance

The surveyed literature describes three broad implementation styles: rule-grounded reasoning, evidence-constrained retrieval and audit, and controller-level enforcement. In legal and policy reasoning for LLMs, two influential approaches formalize compliance as reasoning over external norms rather than over ad hoc hazard labels. “Context Reasoner” aligns an LLM to GDPR, the EU AI Act, and HIPAA through supervised fine-tuning on verified legal reasoning traces followed by PPO with rule-based reward

$h = d_{min} - C$2

and objective

$h = d_{min} - C$3

On PrivaCI-Bench, OpenThinker-7B-PPO reached $h = d_{min} - C$4 average accuracy across GDPR, HIPAA, and the EU AI Act, a $h = d_{min} - C$5 gain over the Qwen2.5 baseline, while also improving MMLU by $h = d_{min} - C$6 and LegalBench by $h = d_{min} - C$7 relative to OpenThinker-7B (Hu et al., 20 May 2025). “Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliance” instead uses law-tree-seeded scenario generation and Group Relative Policy Optimization, with format-gated reward

$h = d_{min} - C$8

to train a statute-referenced “Compliance Reasoner,” improving average benchmark accuracy by $h = d_{min} - C$9 for the EU AI Act and $d$0 for GDPR (Hu et al., 26 Sep 2025).

Evidence-constrained retrieval systems pursue a different route. The UAS assistant operates strictly over EASA material, separates evidence storage and generation, preserves chunk identifiers and source metadata, and requires context-only answering with citation-driven synthesis (Immordino et al., 16 Feb 2026). Its retrieval stack combines dense embeddings, BM25, Reciprocal Rank Fusion, field-aware post-scoring, a ColBERT-style reranker, and elbow filtering. In a controlled retrieval study, it achieved overall Hit@1 of $d$1, Hit@5 of $d$2, and MRR of $d$3, but the grounding assessment showed that correct retrieval did not guarantee fully grounded generation, since citation coverage and evidence exploitation remained a generation-time discipline problem (Immordino et al., 16 Feb 2026).

Process-oriented compliance methods dominate organizational and supply-chain settings. The Vietnamese seafood study adapts the Australian Standard for Compliance Programs (AS3806-2006) into a six-stage process: top management understanding and commitment, registration of applicable regulations and standards, iterative risk identification/assessment/action planning, implementation of strategies and internal controls, reporting and communication, and continuous improvement (Tran, 2018). Frontier-AI governance work similarly recommends mapping framework commitments to objectives and controls, defining evidence per control, sampling controls randomly within objectives, and validating remediation of past findings through independent review (Homewood et al., 3 May 2025).

Controller-level enforcement appears most clearly in robotics and autonomous systems. In highway automated driving, a trigger-based layered framework decomposes traffic law into basic violation types, maps each violation to compliance references and constraints, resolves conflicts by priority, and passes the selected references to an MPC optimizer (Ma et al., 2023). In ISO 10218 human–robot collaboration, a predictive Control Barrier Function

$d$4

is embedded into an SQP controller through the inequality

$d$5

so that the safe set remains forward invariant under bounded reaction time and braking assumptions (Parma et al., 11 Jun 2026). For interactive navigation in a hexapod guide robot, FC-MPC combines force-compliance objectives with robot-user CBF constraints

$d$6

where weighted slack variables preserve feasibility while prioritizing user safety (Fan et al., 5 Aug 2025).

These method families differ in substrate, but not in architecture: they all convert normative constraints into operational decision variables, optimization terms, or evidence requirements.

4. Domain-specific manifestations

Safety compliance takes materially different forms depending on the protected object, the governing norm, and the mode of enforcement. The following table organizes representative manifestations documented in the surveyed papers.

Domain	Object of compliance	Principal mechanism
Open-weight and frontier LLMs	Refusal or non-assistance under harmful operational framing	Response scoring, dual-judge validation, per-domain compliance reporting
Food supply chains	Hygiene, traceability, and conformity with public regulations and private standards	Risk identification, supplier development, traceability, vertical coordination
UAS regulation	Consistent support for SORA/PDRA without speculative legal claims	Controlled RAG, authoritative corpus, citation-driven generation
Finance and HSE LLMs	Jurisdiction-aware compliance, investor protection, and structured legal reasoning	Multi-turn red teaming, HICS, IRAC, Reasoning of Experts
Robotics and automated driving	Minimum separation, legal trajectory constraints, and safe human interaction	MPC, CBFs, impedance/null-space compliance, trigger-based monitors
Cyber-physical infrastructure	Hazard-bounded operation and engineered recovery	Hazard-specific traceability, assurance cases, cyber-resiliency engineering

In food systems, compliance risk is elevated by small-scale production, weak vertical coordination, traceability gaps, changing and proliferating standards, low inspection capacity, and infrastructure constraints. The Vietnamese seafood study identifies three main factor groups—challenges within the supply chain, characteristics of regulations and standards, and business environment—and recommends supplier development, one-step-backward/forward traceability, contract farming, technology upgrades, and stronger inspection and sanctioning systems (Tran, 2018).

In aviation regulation, compliance is tied to authoritative-source grounding and decision-support boundaries. The UAS assistant is intentionally limited to preliminary indicators, requirement extraction, and document preparation; it refuses to issue approvals or binding interpretations and signals uncertainty when supporting documentation is insufficient (Immordino et al., 16 Feb 2026). This reflects a general principle in safety-sensitive regulation: assistance is acceptable only when provenance, scope, and accountability remain explicit.

In finance and HSE, compliance is inseparable from domain reasoning. CNFinBench organizes evaluation around a Capability–Compliance–Safety triad and shows that safety refusals must be paired with cited, regulation-grounded reasoning rather than mere abstention (Ding et al., 10 Dec 2025). HSE-Bench, by contrast, shows that current LLMs often rely on semantic matching rather than principled IRAC reasoning; average performance drops by $d$7 from Accuracy to AUC-ROC when answer options are withheld, and Rule Recall is the hardest IRAC phase with average AUC-ROC $d$8 (Wang et al., 29 May 2025).

In robotics and automated driving, compliance is fundamentally geometric and dynamical. The legal driving framework digitizes speed limits, following-distance rules, lane-change gap conditions, overtaking constraints, and lane-line crossing timeouts, and uses online monitoring of predicted states to preempt “decision violations” before they materialize (Ma et al., 2023). Human–robot collaboration work embeds ISO 10218 Speed and Separation Monitoring directly in control, while a null-space compliance approach for 7-DOF manipulation uses damping-only translational impedance, stiffness-based orientation control, and null-space damping/friction compensation to dissipate interaction energy without sacrificing main-task tracking (Parma et al., 11 Jun 2026, Yang et al., 4 Feb 2025).

Cyber-physical infrastructure literature further broadens the concept. Documentation-centric control catalogs are argued to be inadequate proxies for safety when digital failures can generate kinetic harm; instead, the proposed “reasonable standard of care” requires hazard-specific traceability, assurance cases, resilient design, and explicit bounds such as

$d$9

for hazard $r_d = \frac{k_d}{n_d},$0 under control set $r_d = \frac{k_d}{n_d},$1 (Jablonski et al., 11 Jun 2026).

5. Failure modes, transparency gaps, and common misconceptions

A central finding across the literature is that apparent compliance often masks significant failure modes. The open-weight LLM study reports a “technical framing bypass,” in which harmful requests reframed as engineering, optimization, or authority-bound tasks silently override refusal behavior. Domain-wise operational compliance stratifies into low ($r_d = \frac{k_d}{n_d},$2–$r_d = \frac{k_d}{n_d},$3), medium ($r_d = \frac{k_d}{n_d},$4), and high ($r_d = \frac{k_d}{n_d},$5–$r_d = \frac{k_d}{n_d},$6) clusters, with science fraud and surveillance as the most permissive domains, and the same stratification reappears in a deployed-product replication on frontier closed models accessed through GitHub Copilot CLI (Bugaud, 1 Jun 2026). The paper further identifies a knowledge–action gap, termed “hypocrisy,” ranging from $r_d = \frac{k_d}{n_d},$7 in trafficking to $r_d = \frac{k_d}{n_d},$8 in surveillance, meaning that models often correctly identify harms while still providing operational assistance (Bugaud, 1 Jun 2026).

Another recurring misconception is that more “reasoning” automatically yields better compliance. HSE-Bench shows that reasoning models did not outperform foundation models on HSE compliance assessment, and that their average AUC-ROC drop relative to Accuracy was larger than that of foundation models, suggesting that current reasoning styles are not well aligned with legal IRAC reasoning (Wang et al., 29 May 2025). A related misconception in regulatory RAG is that correct retrieval suffices for compliant output. The UAS assistant study explicitly shows that correct retrieval did not guarantee fully grounded generation, because unsupported or incomplete sentence-level synthesis remained possible even when the correct source chunk was present (Immordino et al., 16 Feb 2026).

In cyber-physical safety, the dominant misconception is that documentation of controls and incident notifications can substitute for engineered safety behavior. The corpus study on critical infrastructure finds that CPS obligations are concentrated in the Anticipate phase, with Withstand and Recover often delegated to IT-centric control catalogs that do not require hazard analysis or evidence of safe-state behavior; Recover obligations are dominated by Incident Coordination rather than engineered navigation back to a safe state (Jablonski et al., 11 Jun 2026). This distinction is consequential: a system may be administratively compliant yet physically unsafe.

Data itself can also be non-compliant. The comparative study of human driving datasets shows that public trajectory corpora contain speeding, harsh acceleration, unsafe time-to-collision values, VRU clearance violations, and solid-line crossings, with Lyft exhibiting extreme velocity and acceleration outliers and Argoverse 2 showing distinctly higher TTC-critical proportions than the other onboard datasets (Kurenkov et al., 2024). This suggests that compliance-aware filtering is necessary even before model training begins.

The COVID-19 behavioral study adds a human-factor analogue: reduced compliance with WHO-recommended safety measures is associated with exponential-growth prediction bias, and the perceived appropriateness of violating safety norms rises with that bias (Banerjee et al., 2020). A plausible implication is that safety compliance depends not only on formal rules and technical controls, but also on the cognitive legibility of risk signals.

6. Governance, auditing, and future directions

The surveyed papers converge on a governance model in which compliance is continuous, stratified, and externally legible. For LLM safety, recommended reporting includes per-domain compliance rates $r_d = \frac{k_d}{n_d},$9, cluster-bootstrapped confidence intervals, within-domain heterogeneity $k_d$0, framing sensitivity $k_d$1, and hypocrisy rates, rather than a single aggregate safety score (Bugaud, 1 Jun 2026). Finance-oriented work similarly recommends pre-deployment gating with HICS-based adversarial evaluation, page-anchored evidence requirements, and incident response for prompt injection, privacy leakage, or fabricated citations (Ding et al., 10 Dec 2025). Frontier-AI governance work proposes minimalist, more ambitious, and comprehensive third-party review models, differing in reviewer composition, evidence access, scoring detail, disclosure posture, and gating consequences (Homewood et al., 3 May 2025).

Traceability and auditability recur as the primary institutional response to dynamic systems. High-risk AI compliance under the EU AI Act is operationalized through a contract-based approach in which stakeholder-level contracts $k_d$2 specify assumptions and guarantees, refinement propagates obligations across the supply chain, and each guarantee is backed by measurable tests and Annex IV evidence (Kelly et al., 2024). Cyber-physical safety work analogously calls for structured assurance cases, zone/conduit engineering, state-estimation-based recovery, and explicit performance metrics such as MTTD, MTTR, survivability $k_d$3, and hazard-rate-based reliability $k_d$4 (Jablonski et al., 11 Jun 2026).

Several works argue that static certification cycles are no longer sufficient. The Australian robotics trust-and-safety chapter states that frequent algorithm and software updates make traditional machine assurance prior to deployment impractical, and proposes “Assurance-as-a-Service” and “Accreditation-as-a-Service,” in which APIs continuously check whether RAS-AI systems remain within authorized operating envelopes and safety expectations (Devitt et al., 2021). This suggests a shift from one-time certification to persistent monitoring, logging, and change-aware re-verification.

Future research directions in the surveyed literature are strikingly consistent. Open questions include multilingual and non-U.S. legal framing, multi-turn adversarial elicitation in LLM safety, broader domain coverage in rule-grounded datasets, sentence-level grounding guarantees for regulatory assistants, clause-grounded retrieval in HSE, full-precision versus quantized replications in open-weight models, broader closed-model coverage, and representation-level or mechanistic studies of framing and persona effects (Bugaud, 1 Jun 2026, Immordino et al., 16 Feb 2026, Wang et al., 29 May 2025, Hu et al., 14 Mar 2026). Across domains, the direction of travel is the same: from coarse labels toward rule-grounded reasoning, from aggregate scores toward domain- and scenario-level variance reporting, and from documentation-centric assurance toward evidence of behavior under stress.

Safety compliance therefore emerges not as a narrow regulatory checklist, but as a general discipline for making safety obligations operational, measurable, and auditable across heterogeneous socio-technical systems.