Frontier AI Risk Management Framework (SafeWork-F1)

Updated 25 July 2025

Frontier AI Risk Management Framework (SafeWork-F1) is a comprehensive, multi-layered approach for identifying, assessing, and mitigating risks in advanced AI systems.
It integrates established methodologies from safety-critical industries such as systematic risk modeling, principled threshold setting, and robust auditing to manage deployment safety.
The framework emphasizes defense-in-depth, lifecycle integration, and continuous safety assurance, ensuring proactive risk monitoring and stringent compliance with evolving standards.

The Frontier AI Risk Management Framework (SafeWork-F1-Framework) is a comprehensive and multi-layered approach for identifying, assessing, and mitigating the unique and systemic risks posed by advanced artificial intelligence systems at the so-called “frontier” of capability. Designed to address challenges such as unpredictable dangerous capabilities, rapid model proliferation, and the inherent difficulty of risk assessment in novel AI systems, the SafeWork-F1-Framework draws from best practices in safety-critical industries and tailors established governance, risk, and assurance methodologies to the demands of large-scale AI research and deployment. The framework integrates organizational governance principles, systematic risk modeling, principled threshold setting, continuous safety assurance, and robust auditing—all to ensure that the speed of AI innovation does not outpace society’s ability to manage its associated risks.

1. Key Governance Principles and Organizational Structures

The SafeWork-F1-Framework is built on the foundation of strong corporate governance and independent assurance. It mandates the establishment of a dedicated, organizationally independent internal audit function that reports directly to the board of directors, in line with the Institute of Internal Auditors (IIA) “Three Lines of Defense” model (Schuett, 2023). In this model, risk management is divided into:

The first line: risk owners and management,
The second line: specialized risk and compliance functions,
The third line: an independent internal audit function.

This structure is complemented by a Combined Assurance Framework that brings in coordination with external audits and ethics boards. Internal audits objectively assess risk management practices, challenge management assumptions, and provide a continuous feedback loop to the board. This counters principal–agent problems and ensures oversight is not solely mediated through senior management. The internal audit’s engagement cycle (planning, information gathering, assessment, reporting) and its role as a whistleblower contact point further enhance transparency and effectiveness of risk governance.

2. Systematic Risk Identification, Analysis, and Threshold Setting

The framework prescribes a systematic approach to risk identification, encompassing: classification via established risk taxonomies, open-ended red-teaming, and the construction of detailed risk models akin to those in nuclear or aviation domains (Campos et al., 10 Feb 2025). These models analyze causal chains and loss scenarios, applying methods such as event and fault trees, scenario analysis, and expert-driven forecasting exercises.

A central tenet is the rigorous definition and operationalization of risk thresholds (Koessler et al., 20 Jun 2024, Raman et al., 4 Mar 2025). Risk thresholds are quantitative or semiquantitative criteria that delimit acceptable levels of harm as a function of both likelihood and severity:

$R = P \times S$

where $P$ is the estimated probability of an adverse event and $S$ its severity. Using quantitative benchmarks such as F/N diagrams (frequency–number harm metrics), thresholds are set for critical domains (e.g., cyber offense, CBRN, persuasion/manipulation, autonomous AI R&D) and are categorized as “green” (routine deployment), “yellow” (controlled deployment, strengthened mitigation), or “red” (suspension of development/deployment) zones according to the measured risk (Lab et al., 22 Jul 2025). Intolerable risk thresholds are explicitly formulated (e.g., 25% uplift over baseline attack probability, or $P(\text{deceptive situational awareness}) \geq 0.6$ ) to operationalize “red lines” where proactive intervention is mandatory (Raman et al., 4 Mar 2025).

3. Evaluation, Model Testing, and Benchmarking Practices

Comprehensive model evaluation is at the core of the SafeWork-F1-Framework. Evaluations draw from both automated benchmarks and human-centric studies, including red-teaming, uplift studies, and simulation of advanced risk scenarios (Krishna et al., 7 Jul 2025, Lab et al., 22 Jul 2025). Established evaluation platforms such as OpenCompass and targeted domain-specific benchmarks (e.g., chemical/biological planning, cybersecurity CTFs, autonomous research agents, manipulation and collusion simulations) provide the quantitative infrastructure for capability and risk measurement.

A distinctive feature is the integration of “yellow” (early warning) and “red” (intolerable) thresholds into the deployment and monitoring process, guided by the “AI- $45^\circ$ Law,” which posits that capability and safety should advance in tandem. Risk zones are demarcated using experimental results from diverse models, determining which systems remain fit for deployment or require additional mitigation. The framework also recognizes adversarial phenomena such as model sandbagging and multi-agent collusion, requiring robust methodologies to surface and address these risks during evaluation (Lab et al., 22 Jul 2025).

4. Assurance, Safety Case Development, and Continuous Updating

The framework incorporates a safety case methodology as both an internal and regulatory tool (Buhl et al., 28 Oct 2024, Cârlan et al., 23 Dec 2024, Barrett et al., 9 Feb 2025, Dassanayake et al., 17 Jul 2025). Safety cases are structured arguments that include scope, objectives (often quantitative, e.g., $P(\text{harm}) < 10^{-7}$ year $^{-1}$ ), argumentation (e.g., inability, control, and trustworthiness claims), and supporting evidence from rigorous testing, monitoring, and post-deployment review.

Dynamic safety case management (DSCMS) enables ongoing alignment of safety arguments with system state by leveraging automated consistency checking and Safety Performance Indicators (SPIs) (Cârlan et al., 23 Dec 2024). This mechanism ensures that as model capabilities or threat landscapes change—e.g., through observed incident spikes or shifts in performance metrics—reassessment and revision of the safety case is routine, not exceptional. Visual dashboards and multi-dimensional confidence assessments based on the Assurance 2.0 methodology improve transparency and executive decision-making (Barrett et al., 9 Feb 2025).

5. Defense-in-Depth, Security, and Lifecycle Integration

Adopting principles from cybersecurity, the SafeWork-F1-Framework advocates for a defense-in-depth model comprising three mutually reinforcing approaches (Ee et al., 15 Aug 2024, Guo et al., 7 Apr 2025):

Functional approach: Organization-wide functions such as governance, mapping of risks, measurement (e.g., benchmarks, red-teaming), and active management/mitigation controls.
Lifecycle approach: Embedding risk management activities at every phase, from planning and data collection to model training, evaluation, staged deployment, and post-deployment monitoring.
Threat-based approach: Systematic identification and cataloguing of malicious tactics, techniques, and procedures (TTPs) using frameworks (e.g., MITRE ATT&CK, MITRE ATLAS) tailored for AI systems.

Security protocols, containment measures, and redundant deployment controls (e.g., API filtering, information allow-listing, robust insider threat programs) are put in place. Security benchmarks, human factor modeling (e.g., for manipulation risk), and formal verification are prioritized for both model and hybrid system scenarios, seeking to address the observed asymmetry where attackers may initially benefit more from AI than defenders (Guo et al., 7 Apr 2025).

6. Assurance, Audit, and Compliance Mechanisms

The framework integrates ongoing assurance and compliance mechanisms at multiple levels:

Internal audit: Systematic, independent review ensures governance bodies receive accurate, unbiased accounts of organizational risk posture and responses (Schuett, 2023).
Third-party compliance: Regular external reviews—ranging from minimalist to comprehensive, using concrete control and process maturity frameworks—help operationalize safety commitments and inform stakeholder trust (Homewood et al., 3 May 2025).
Continuous documentation: Drawing on best practices from nuclear energy, aviation, cybersecurity, and healthcare, comprehensive safety documentation is mandated from initial development through to deployment and incident reporting (Kierans et al., 1 May 2025).
Industry insurance and reinsurance: A tripartite insurance system (private, pooled, federally backed) is proposed to “price in” risk, align incentives, and ensure financial accountability for catastrophic events, adapting mechanisms from high-risk precedents such as nuclear and medical liability (Stetler, 2 Apr 2025).

7. Responsible Reporting, Transparency, and Regulatory Linkages

The SafeWork-F1-Framework establishes protocols for responsible, structured, and differentiated reporting to ensure both secure information handling and transparency (Kolt et al., 3 Apr 2024). Reporting categories include: development and deployment details, risk and harm documentation, and mitigation strategies. Sensitive details are shared only with trusted government actors, while summaries and incident reports may be disclosed to independent experts and industry peers. Regulatory integration is facilitated through safe-harbor provisions, systematic documentation standards, and capacity-building for governmental oversight.

Close linkage to international commitments, such as the Frontier AI Safety Commitments and declarations from multilateral summits, shapes the required level and frequency of risk disclosure, as well as mandates for continuous reassessment as outlined by the Paris AI Safety Summit and related agreements (Raman et al., 4 Mar 2025, Krishna et al., 7 Jul 2025).

In summary, the SafeWork-F1-Framework is a holistic, adaptive risk management architecture for frontier AI, synthesizing rigorous governance, systematic hazard analysis, principled threshold setting, and robust assurance mechanisms. By integrating lessons from other high-risk industries and recent advances in AI-specific safety evaluation, it seeks to constrain the deployment of advanced AI to “green” and “yellow” safety zones, proactively manage and review risks in domains such as cyber offense, persuasion, and autonomous research, and embed safety as both a technical and organizational imperative throughout the AI development lifecycle.