AI bias: 9 questions leaders should ask

AI bias: 9 questions leaders should ask

Artificial intelligence bias can create problems ranging from bad business decisions to injustice. Use these questions to fight off potential biases in your AI systems

59 readers like this


January 29, 2019
Crossing the gap to big data

Management questions to ask about AI bias

If you’re actually worried about those robot overlord scenarios, then it’s probably a good idea to have some controls in place to prevent them from coming true. More seriously, ongoing monitoring, auditing, and optimization of AI applications is absolutely necessary to guard against systemic bias. Like security and other IT concerns, this is an indefinite project.

7. What proportion of resources is appropriate for an organization to devote to assessing potential bias?

“A thoughtful organization will scale the resources dedicated to assessing bias based on the potential impact of any bias and the sensitivity to potential bias of the team charged with developing the AI system,” says Northman, the Tucker Ellis attorney.

This is itself an ongoing project of analysis and optimization.

“As a rule, some of the most important assessment will take place following deployment,” Northman says. “It is critical that an organization study and evaluate the results the AI system produces.”

8. Have you thought deeply about what metrics you use to evaluate your work?

Bottom-line metrics like revenue or traffic are more fertile terrain for AI bias.

AI needs measurement – like any other worthwhile IT endeavor. Glaser from Periscope Data notes that short-term, bottom-line metrics like revenue or traffic are more fertile terrain for AI bias.

“AI systems are less likely to go off the rails if the team is evaluating them using long-term holistic metrics, including pairing metrics so that they balance each other out,” Glaser says.

9. How can we test for bias in training data?

McFarland notes that AI success, as with so many other IT systems, requires ongoing testing, though he acknowledges that teams might encounter challenges here, including costs or population assessment accuracy. But not testing at all is leaving the door wide open for bias.

Frequency bias is a common issue with AI; it’s also one you can test for in your training data and on an ongoing basis.

“Compare counts of different population types in your training data and compare that to the actual distribution of your target population,” McFarland says. “Any skews in the counts can indicate a bias for or against a certain population type.”

Coming full circle, human biases are another common issue. You can test for those, too.

“Have several experts summarize the same document and then compare their results,” McFarland says. “If you have a good summarization framework, these summaries will be mostly interchangeable. Looking for variances across these summaries can help identify biases in the experts.”

[ Want lessons learned from CIOs applying AI? Get the new HBR Analytic Services report, An Executive's Guide to Real-World AI. ]


7 New CIO Rules of Road

CIOs: We welcome you to join the conversation

Related Topics

Submitted By Tomoko Yokoi
May 17, 2019

Digital leaders need a combination of technology, people, and entrepreneurial skills. To nuture these skills in more young women, we need to tackle three challenges.

Submitted By Dr. Manas Fuloria
May 17, 2019

Nagarro’s CEO explains how an AI-driven system is changing the way people interact with a variety of tasks and systems at his company.

Submitted By Curt Carver
May 16, 2019

AI agents now serve as personal assistants for faculty at the University of Alabama at Birmingham. We call them "intrusive advising assistants" – because they reach out to engage students


Email Capture

Keep up with the latest thoughts, strategies, and insights from CIOs & IT leaders.