Ethics-Based Auditing in AI Systems

• Ethics-based auditing is a governance mechanism that continuously evaluates AI systems for alignment with ethical principles, emphasizing transparency and accountability.
• It employs both qualitative and quantitative methods, such as stakeholder interviews and bias detection, to yield actionable insights for risk management.
• The process supports informed decision-making by synchronizing technical development with public policy and strategic organizational goals.

Ethics-based auditing is a governance mechanism designed to ensure that AI systems are aligned with relevant ethical principles and norms throughout their lifecycle. Rather than relying solely on voluntary codes or external regulation, ethics-based auditing operates as a structured, continuous, and constructive process embedded within organizations to assess, document, and improve the consistency of AI system behaviors with normative values. This approach not only facilitates accountability and transparency, but also provides essential feedback loops for responsible decision-making, risk management, and the realization of public value in the deployment of AI technologies.

1. Definition and Rationale

Ethics-based auditing is formally defined as a structured process whereby an organization assesses present or past behavior (of AI systems or decision-making processes) for consistency with relevant principles or norms (Mokander et al., 2021). This post-compliance governance tool is intended to bridge the gap between abstract ethical frameworks and their operationalization in complex, autonomous, and evolving AI systems. Traditional oversight mechanisms—from legal compliance to voluntary codes—frequently fail to address the technical opacity, autonomy, and scale of AI decision-making. Ethics-based auditing thus emerges as a necessary instrument for ethical alignment, promoting both procedural regularity and transparency across the AI system lifecycle.

A critical aspect of ethics-based auditing is that it exposes and communicates the embedded normative values within AI artifacts or processes, but does not attempt to codify ethics in a closed or exhaustive manner. The intention is to improve practices, support strategic alignment with public policy, and unlock both economic and societal benefits, rather than offering a panacea for all ethical risks of automation.

2. Functional Roles and Outcome Improvements

Ethics-based auditing delivers value through several interrelated mechanisms (Mokander et al., 2021):

Outcome	Function Description
Decision making	Provides visualization and monitoring of system outcomes, supporting feedback and allocation of accountability
User satisfaction	Increases transparency and procedural fairness by enabling contestability and explanation of AI decisions
Growth potential	Fosters sector-specific governance, enabling proactive harm limitation and sustainable innovation
Law-making	Supplies actionable feedback to inform legislation and standard-setting
Human well-being	Identifies and mitigates threats to society, relieving suffering and promoting societal benefit

These mechanisms are operationalized through both qualitative (e.g., participatory processes, stakeholder interviews) and quantitative (e.g., bias detection, benchmarking) means. Critically, ethics-based auditing does not replace regulatory compliance instruments such as certification or human oversight, but rather complements them to cultivate a broader culture of responsible, trustworthy AI.

3. Design Criteria and Best Practices

To be feasible and effective, ethics-based auditing adheres to several design principles, which collectively constitute the aspirational "gold standard" (Mokander et al., 2021):

1. Continuous and Constructive Process: Auditing is ongoing, providing continuous monitoring, documentation, and adaptation, rather than one-off verification.
2. Systemic Perspective: AI systems are treated as integral elements of broader socio-technical systems, emphasizing context, interdependencies, and alternatives rather than technical isolation.
3. Dialectic (Question-Oriented) Approach: The audit process is dynamic and dialogic, focusing on posing and revisiting the right questions, not merely checking for predetermined answers.
4. Alignment with Public Policy and Incentives: Auditing frameworks are synchronized with organizational goals and reinforced by external public policy incentives for ethically desirable behavior.
5. Strategic and Design-Driven Integration: Ethics-based auditing is embedded into the design process from inception, promoting interpretability and robustness as foundational traits.

Objectivity is further reinforced by ensuring auditor independence from daily line management—audits may be conducted internally, by third parties, or by governmental bodies, but should maintain arm's-length separation from development and operational teams.

4. Processual Structure and Audit Types

Ethics-based auditing is characterized by its integration in both the system development lifecycle and organizational governance processes (Mokander et al., 2021). While the original paper does not propose a stepwise formal algorithm, it distinguishes among several core audit types:

• Functionality Audits: Examine the underlying rationale and logic embedded in decision-making modules.
• Code Audits: Review source code for compliance with ethical requirements and coding best practices.
• Impact Audits: Investigate outcomes and societal effects of the AI system, providing a critical empirical check on claimed benefits or risks.

Audits are envisioned as continuous, holistic, and design-driven iterative processes that engage with the system context, stakeholder values, and evolving risk landscapes.

5. Constraints and Challenges

The paper identifies sixteen distinct constraints impacting ethics-based auditing, grouped thematically (Mokander et al., 2021):

Conceptual Constraints

• Absence of consensus on crucial ethical principles (e.g., what constitutes "fairness" or "justice").
• Necessity of navigating trade-offs and conflicts between normative values.
• Difficulty in quantifying indirect externalities or reductionist loss from simplified auditing.

Technical Constraints

• Opacity and interpretability issues—many AI systems function as "black boxes."
• Risks to data integrity and privacy during audit procedures.
• Standard compliance approaches may not fit agile, rapidly iterating AI cycles.
• Limited representativeness of test environments versus real-world deployment.

Economic and Social Constraints

• Disproportionate burden of audits on particular industries or populations.
• Potential for audits to suppress innovation if not adequately balanced by incentives.
• Adversarial manipulation, where parties seek to game or circumvent audit criteria.
• Power imbalances may block effective corrective actions despite audit findings.

Organizational and Institutional Constraints

• Ambiguity in audit scope and in "who audits whom."
• Insufficient access or information for full evaluation.
• Jurisdictional complexity when AI systems cross regulatory boundaries.

Understanding and accounting for these constraints is positioned as necessary to enable ethics-based auditing to genuinely facilitate ethical AI alignment and to yield its promised economic and societal dividends.

6. Integration with Governance Mechanisms

Ethics-based auditing is explicitly positioned to complement (not replace) other governance mechanisms (Mokander et al., 2021). Regulatory authorities are advised to retain sanctioning power, with oversight typically delegated to independent agencies. The effectiveness of auditing processes depends on their integration into organizational infrastructure, parallel to technical or physical systems. Excessive privatization or self-regulation without public oversight is cautioned against, as is the risk of "ethics washing," where claims of compliance outpace substantive change.

Continuous ethical reflection by participating individuals is regarded as foundational. Auditing alone is not a panacea; rather, it must form part of an ongoing ecosystem of ethical deliberation, legal compliance, and adaptive policymaking.

7. Methodological Foundations and Formalizations

The paper does not introduce explicit mathematical formalizations or domain-specific quantitative metrics for ethics-based auditing. Instead, it outlines broad processual qualities—continuous, holistic, dialectic, strategic, and design-driven auditing—without introducing formulas or stepwise algorithms. The lack of formalism is itself recognized: reductionist mathematical ethics may result in unacceptable information loss or mischaracterization of complex value tensions.

Methodologically, effectiveness is tied to the capacity to reveal and operationalize the normative values in system development, enable stakeholder accountability, and support the public legitimacy of AI governance through auditable, transparent processes.

---

Summary Table: Core Principles and Constraints

Principle/Practice	Description
Continuous, design-driven process	Auditing embedded throughout lifecycle and system design
Systems perspective	AI as part of socio-technical contexts
Dialectic engagement	Emphasis on iterative questioning, not static checklists
Alignment with public policy	Incentivization and synchronization with societal values
Integration and independence	Embedded in governance, with objective operational distance

Constraint Group	Example Constraints
Conceptual	Value pluralism, trade-offs, quantification difficulties
Technical	Black-box opacity, privacy risk, rapid update cycles, limited test representativeness
Economic/Social	Audit burden, innovation impact, adversarial gaming, power asymmetries
Organizational/Legal	Scope ambiguity, information access, transjurisdictional enforcement

---

Ethics-based auditing constitutes a necessary and realistic mechanism for operationalizing and assessing AI ethics. Its strength lies in its structured, ongoing engagement with the socio-technical realities of AI, delivered through continuous, context-sensitive, and participatory processes that traverse technical, organizational, and societal domains. However, the process must remain aware of, and adaptable to, substantial conceptual, technical, and institutional constraints to fulfill its promises in practice (Mokander et al., 2021).