Ethics-Based Auditing to Develop Trustworthy AI (2105.00002v1)

Abstract: A series of recent developments points towards auditing as a promising mechanism to bridge the gap between principles and practice in AI ethics. Building on ongoing discussions concerning ethics-based auditing, we offer three contributions. First, we argue that ethics-based auditing can improve the quality of decision making, increase user satisfaction, unlock growth potential, enable law-making, and relieve human suffering. Second, we highlight current best practices to support the design and implementation of ethics-based auditing: To be feasible and effective, ethics-based auditing should take the form of a continuous and constructive process, approach ethical alignment from a system perspective, and be aligned with public policies and incentives for ethically desirable behaviour. Third, we identify and discuss the constraints associated with ethics-based auditing. Only by understanding and accounting for these constraints can ethics-based auditing facilitate ethical alignment of AI, while enabling society to reap the full economic and social benefits of automation.

This paper, "Ethics-Based Auditing to Develop Trustworthy AI" (Floridi et al., 2021 ), proposes ethics-based auditing as a crucial mechanism to bridge the gap between high-level AI ethics principles and their practical implementation in real-world AI systems. It argues that while many organizations have developed ethical guidelines, translating these principles into actionable, verifiable criteria for design and deployment remains a significant challenge. Auditing provides a structured process to assess whether an AI system's behavior aligns with relevant ethical norms and principles, going beyond mere legal compliance or technical robustness to focus on embedded values.

The paper identifies several types of ethics-based audits:

• Functionality audits: Examining the rationale behind decisions made by the AI system.
• Code audits: Reviewing the source code to identify potential ethical issues.
• Impact audits: Investigating the real-world effects of the algorithm's outputs on individuals and society.

Regardless of whether the auditor is internal, external, or governmental, the key is independence from the day-to-day management of the AI system being audited.

Implementing ethics-based auditing can yield practical benefits for organizations deploying AI. It can:

• Support decision-making by visualizing and monitoring system outcomes.
• Improve transparency by helping explain decisions and enabling recourse.
• Allow for tailored, sector-specific governance approaches.
• Help anticipate and mitigate potential harms, reducing human suffering.
• Clarify accountability by leveraging existing governance structures.
• Manage conflicts of interest, such as restricting access to sensitive data during evaluation.

To be feasible and effective, ethics-based auditing should follow specific best practices:

• Continuous: It's not a one-off event but an ongoing process of monitoring system outputs and performance characteristics. This requires integrating auditing into the AI system's lifecycle.
• Holistic: AI systems are part of larger socio-technical contexts. Auditing should consider the system's interaction with users, the environment, and available alternatives.
• Dialectical: Ethics-based auditing is about asking the right questions to probe ethical alignment, rather than checking against a fixed answer sheet. This implies an iterative process of inquiry and refinement.
• Strategic: The auditing framework must be aligned with organizational policies, incentives, and strategic goals to ensure buy-in and effectiveness. Doing the ethically right thing should be made easy.
• Design-driven: Auditing should provide feedback to the continuous design and redesign process. This encourages building ethical considerations, like interpretability and robustness, into the system from the outset.

Practical Implementation Considerations and Constraints

While promising, implementing ethics-based auditing faces significant practical constraints detailed in the paper. Understanding these is crucial for successful deployment:

• Conceptual Constraints:
  • Lack of consensus: Defining and prioritizing potentially conflicting values like different notions of "fairness" (e.g., demographic parity vs. individual fairness).
  • Quantifying externalities: Difficulty measuring the broader societal impact of AI systems.
  • Information loss: Explanations for complex models often simplify the underlying process, losing detail.
  • Practical Impact: Requires organizations to define specific, context-relevant ethical goals and metrics (e.g., choosing a specific fairness definition and a corresponding metric like statistical parity difference or equalized odds). This often involves difficult stakeholder discussions.
• Technical Constraints:
  • Opacity and Interpretability: Many modern AI models (like deep neural networks) are "black boxes," making it hard to understand why a specific decision was made.
  • Data Integrity/Privacy: Auditing requires access to data (training data, performance data), which introduces privacy and security risks.
  • Agile Development: Traditional, linear compliance audits conflict with agile software development cycles where models are updated frequently.
  • Test vs. Real-World Behavior: AI systems, especially those that learn over time, may behave differently in production than in test environments.
  • Practical Impact: Requires the use of interpretability techniques (e.g., LIME, SHAP) during audits. Secure data handling protocols and possibly differential privacy techniques are needed for audit data. Auditing must be integrated into continuous integration/continuous deployment (CI/CD) pipelines. Continuous monitoring of performance and ethical metrics in production is essential.

# Pseudocode example: Checking for disparate impact during continuous monitoring def check_disparate_impact(predictions, protected_attribute, outcome_attribute): # Calculate selection rates for different groups based on protected_attribute # Example: Check if selection_rate(group_A) / selection_rate(group_B) < 0.8 # Return True if disparate impact detected, False otherwise group_a_selected = sum(1 for p, a in zip(predictions, protected_attribute) if a == 'Group A' and p == 'Selected') group_a_total = sum(1 for a in protected_attribute if a == 'Group A') rate_a = group_a_selected / group_a_total if group_a_total > 0 else 0 group_b_selected = sum(1 for p, a in zip(predictions, protected_attribute) if a == 'Group B' and p == 'Selected') group_b_total = sum(1 for a in protected_attribute if a == 'Group B') rate_b = group_b_selected / group_b_total if group_b_total > 0 else 0 if min(rate_a, rate_b) / max(rate_a, rate_b) < 0.8: # Example 80% rule return True, f"Disparate impact detected: Rate A={rate_a:.2f}, Rate B={rate_b:.2f}" return False, "" # Integrated into monitoring pipeline # periodic_data = get_production_data() # predictions = model.predict(periodic_data) # protected_attributes = periodic_data['sensitive_feature'] # disparate_impact, message = check_disparate_impact(predictions, protected_attributes, 'decision_outcome') # if disparate_impact: # alert_auditing_team(message)

• Economic & Social Constraints:
  • Cost: Audits incur financial and operational costs, potentially burdening smaller entities.
  • Innovation vs. Ethics: Balancing the incentive for rapid innovation with the need for ethical rigor.
  • Adversarial Behavior: AI systems are vulnerable to adversarial attacks or manipulation designed to deceive auditors.
  • Power Structures: Auditing outcomes might be ignored if they conflict with the interests of powerful stakeholders.
  • Practical Impact: Requires careful cost-benefit analysis and potentially tiered auditing requirements. Incentive structures need to reward ethical development. Robustness testing and secure deployment practices are necessary. Requires strong governance structures with the authority to enforce audit findings.
• Organizational & Institutional Constraints:
  • Lack of Clarity: Ambiguity around who is responsible for auditing and what their authority is.
  • Auditor Access: Auditors may lack the necessary technical expertise or access to code, data, or system documentation.
  • Jurisdiction: The global nature of AI systems complicates auditing within national legal frameworks.
  • Practical Impact: Organizations need to define clear roles, responsibilities, and reporting lines for ethical auditing. Establish data access protocols and ensure auditors have appropriate technical skills or support. International cooperation frameworks may be needed for auditing globally deployed systems.

Bridging Principles and Practice

The paper strongly advocates for independent ethics-based auditing, potentially conducted by authorized agencies, to provide credible checks. It positions ethics-based auditing not as a replacement for existing tools like human oversight or regulation, but as a crucial complement.

For practitioners, this means:

1. Defining Scope: Identify which AI systems require ethics-based auditing based on their potential impact and risk.
2. Tailoring Frameworks: Develop or adapt auditing frameworks that are specific to the domain, the AI technology used, and the potential ethical challenges (e.g., fairness in lending, privacy in healthcare, safety in autonomous systems).
3. Integrating Tools: Incorporate technical tools for bias detection, explainability, robustness testing, and continuous monitoring into the development and deployment pipeline.
4. Establishing Governance: Create internal structures and processes for conducting audits, reporting findings, and ensuring that corrective actions are taken. Consider external validation or third-party audits for high-risk systems.
5. Training and Expertise: Ensure teams involved in design, development, deployment, and auditing have the necessary understanding of AI ethics principles and the technical skills to implement and assess them.

Implementing ethics-based auditing requires a shift from reactive problem-solving to proactive, continuous assessment integrated throughout the AI system's lifecycle. By actively addressing the conceptual, technical, economic/social, and organizational constraints outlined in the paper, organizations can move towards developing truly trustworthy AI systems.