What steps should companies take to audit AI systems for bias?

Auditing AI systems for bias requires a structured, multi-stage process combined with cross-functional governance to ensure that engineering practices align with organizational values and ethical expectations.

The practical auditing stages span from early scoping to post-deployment monitoring:

• Scoping and Planning: Clarify the objectives of the audit by reviewing the motivations, intended impact, and desired use cases of the system ^[1]. This stage establishes which ethical principles and standards will guide development ^[2], maps out analogous deployments ^[3], and determines the appropriate risk tolerance and tiers for generative or predictive AI applications ^[4].
• Data and Documentation Review: Review documentation on data collection procedures, training data origins, historical context, and curation processes to understand data limitations and fit ^[5][6]. Auditors evaluate how systemic, statistical, and human biases might be embedded within these datasets ^[7].
• Testing, Evaluation, Validation, and Verification (TEVV): Execute rigorous testing protocols to gauge system compliance against prioritized ethical values and performance benchmarks ^[8][9]. This includes stress-testing vulnerabilities under extreme scenarios, checking for performance disparities across groups, validating model outputs against context, and documenting testing tools and test sets ^[10][11].
• Post-Deployment Monitoring and Feedback Integration: Continuously monitor the functionality and behavior of deployed systems in production ^[12][13]. Organizations should establish feedback channels for end users and impacted communities to report problems, track emergent risks, and incorporate adjudicated feedback back into system design ^[14][15].

Effective execution relies heavily on cross-functional governance, which establishes clear lines of accountability, documentation trails, and organizational structures across multiple lines of defense ^[16][17][18]. This governance involves senior leadership setting the risk tone ^[19], diverse interdisciplinary teams informing risk management ^[20], and internal audit functions providing independent evaluation of first- and second-line controls ^[21].

That governance framework extends directly into operational risk management: