- The paper proposes a novel OSCAL extension for AI compliance by integrating 16 tailored properties that encode lifecycle semantics and traceability.
- The paper demonstrates a three-layer, compliance-as-code architecture that generates continuous, verifiable evidence during both training and inference.
- Validation on high-risk applications, including credit scoring and medical imaging, confirms the system’s ability to flag biases and trigger remediation workflows.
Machine-Readable Evidence for AI Compliance: OSCAL Extensions and Compliance-as-Code Architecture
Motivation and Context
The paper "Making AI Compliance Evidence Machine-Readable" (2604.13767) addresses a critical gap in the operationalization of AI governance frameworks, notably the EU AI Act, ISO/IEC 42001, and NIST AI RMF. Existing frameworks enumerate assurance requirements and conformity expectations but lack executable formats to encode controls and produce verifiable, auditable evidence artifacts. The authors propose adapting OSCAL (Open Security Controls Assessment Language), a NIST standard widely deployed for cybersecurity compliance in FedRAMP, as a lingua franca for AI assurance pipelines. The approach bridges fragmented, documentation-centric compliance practices with a machine-readable, pipeline-integrated architecture, enabling evidence generation as a byproduct of model training and evaluation.
OSCAL Extensions for AI Lifecycle Assurance
Central to the proposal is the formal extension of OSCAL’s implemented-requirement schema with 16 property types tailored to AI assurance and traceability semantics. These properties encompass metric function identifiers, comparative operators, thresholds, severity levels, lifecycle phases (training, validation, monitoring, incident), enforcement modes (monitor, warn, block), evaluation windows, contextual targets (system, dataset, model), risk and treatment links, policy and objective references, risk acceptance criteria, threshold justification, and stakeholder consultation records. This specification enables end-to-end traceability—linking technical evidence to organizational risk management artifacts—and produces rationales for acceptance thresholds in line with Art.9.5 of the EU AI Act.
The design intentionally preserves backward compatibility with OSCAL’s JSON schema, exploiting established validation infrastructure and facilitating simultaneous referencing of controls across EU, ISO, and NIST catalogs. This interoperable envelope eliminates the triple-documentation problem highlighted by the authors.
Three-Layer Compliance-as-Code Architecture
The proposed pipeline architecture comprises three layers:
- Policy Layer: Controls are authored as OSCAL assessment plan files, version-controlled alongside training code. Data governance and model performance controls are separated and mapped directly to EU AI Act articles—most notably Arts. 10 (data quality, bias) and 15 (accuracy, robustness).
- Evidence Layer: The SDK wraps ML pipeline execution in a context manager, activating seven probes for code analysis (AST trace), data integrity, supply chain BOMs (CycloneDX), environment fingerprinting, hardware telemetry, carbon emissions, and enforcement verification. All probes operate concurrently and map to specific regulatory articles (Arts. 9-12, 15).
- Enforcement Layer: The engine parses OSCAL controls, dispatches corresponding metric functions from a registry covering fairness, privacy, quality, and performance, evaluates metrics against thresholds, and logs findings. Failed controls trigger automatic risk generation and remediation items (POA&M), as mandated by the EU AI Act’s evidence requirements.
Each pipeline run produces a self-contained, machine-readable bundle: OSCAL Assessment Results, POA&M artifacts, BOMs, hashes, environment metadata, and carbon traces, validated against the NIST OSCAL schema. No modification of ML training code is required.
Validation on High-Risk AI Systems
The architecture is validated on two canonical high-risk systems per Annex III of the EU AI Act:
- Credit Scoring (Area 5b): Logistic regression on the UCI German Credit dataset demonstrates pre-training (class imbalance, disparate impact) and post-training (accuracy, demographic parity) assurance. Numerical results highlight the architecture’s utility: age disparity control fails (0.286 vs 0.50 threshold), triggering a remediation workflow, while group-aware sample reweighting reduces gender bias with accuracy intact.
- Medical Imaging Segmentation (Area 5a): A MONAI-based CT segmentation pipeline for whole-body anatomical structures, emphasizing deployer scenarios with pre-trained weights. OSCAL policies declare performance metrics stratified by cohort, and evidence probes capture supply chain and environment artifacts critical for technical documentation (Annex IV) and regulatory audit.
Both scenarios validate direct integration and continuous evidence production, covering both training and inference modalities. Generated artifacts are version-controlled and auditor-ready.
Implications and Speculative Extensions
Engineering and MLOps
The architecture reframes compliance as an amortized infrastructure cost, generating auditor-verifiable evidence streams during pipeline execution. Immediate feedback on data quality, bias, and metric outcomes enables remediation before market placement and reduces audit friction.
Regulatory and Auditing
Machine-readable OSCAL artifacts can be ingested by conformity assessment bodies or regulator dashboards, facilitating automated verification of control evaluation, threshold adherence, and remediation tracking. This approach strengthens auditor independence and the scalability of high-risk system review.
Standards and Framework Interoperability
The 16 OSCAL property extensions provide concrete foundations for standardization efforts such as CEN/CENELEC prEN 18286 and potential future NIST AI profiles. A single catalog can unify controls across EU, ISO, and NIST frameworks, driving regulatory harmonization and simplifying cross-jurisdictional evidence handling.
Post-Market Monitoring and Agentic AI
While validation targets pre-market obligations (Arts. 9–15), the architecture extends seamlessly to post-market monitoring (Art. 72) and incident reporting (Art. 73), leveraging lifecycle phase and evaluation window semantics. For agentic AI, runtime enforcement and trajectory-level compliance policies can reuse the same OSCAL profiles as new consumers, supporting oversight modalities for high-risk, dynamic systems. Recent governance proposals for agentic AI (Wang et al., 5 Aug 2025, Kaptein et al., 17 Mar 2026, Rath, 7 Jan 2026, Nannini et al., 6 Apr 2026) map naturally onto enforcement modes and continuous evaluation windows specified in OSCAL.
Limitations
The scope excludes organizational requirements (Arts. 16–17), multi-regulation reconciliation, and additional modalities such as NLP, LLM, and recommender systems. OSCAL artifacts are engineer- and regulator-oriented; parallel human-readable formats are necessary for transparency and stakeholder accountability.
Conclusion
This work establishes an executable assurance infrastructure for AI governance, proposing OSCAL as the backbone for machine-readable, pipelined evidence production. Extension properties encode lifecycle semantics, traceability, and threshold rationales, integrating with ML pipelines for continuous and auditor-ready documentation. The validation results on high-risk systems demonstrate practical utility and granularity. The architecture provides a scalable, standards-aligned foundation for pre-market and post-market compliance, with significant implications for engineering practice, regulatory bodies, and future agentic AI governance.