Drexel Professor Warns of "Latent Risk" as AI Erodes Human Oversight Skills
A professor from Drexel University warns that organizations depending on their staff to identify AI mistakes might be neglecting vital skills. This "latent risk" indicates that diminished participation in decision-making could leave employees ill-equipped to act. This idea is rooted in automation studies, including those examining automation bias and the "out-of-the-loop performance problem." Over four decades ago, cognitive psychologist Lisanne Bainbridge highlighted similar "ironies of automation." In response, Deloitte is revamping graduate audit training through simulations to enhance judgment skills. The professor emphasizes that accountability has not evolved alongside AI's influence, calling for adaptive governance. Organizations should regularly evaluate the human-AI dynamic and ensure essential skills are maintained, advocating for instances where employees make decisions independently of AI guidance.
Key facts
- The author is a professor of management information systems and academic director of Drexel University's Center for Applied AI & Business Analytics
- "Latent risk" describes the possibility that people lose capabilities their organizations still depend on them to exercise
- Cognitive psychologist Lisanne Bainbridge described the "ironies of automation" more than four decades ago
- The "out-of-the-loop performance problem" refers to people becoming less prepared to intervene as they become less involved in a task
- Deloitte is redesigning its graduate audit training with simulations as AI takes over manual work traditionally done by junior auditors
- The author argues authority, responsibility and capability can begin to separate when AI takes on more decisions
- Accuracy and productivity metrics may look good even as the human role in a process changes
- One proposed test is to occasionally ask employees to make a decision without first seeing the AI recommendation
Entities
Artists
- Lisanne Bainbridge
Institutions
- Drexel University
- Center for Applied AI & Business Analytics
- Deloitte
- The Conversation
Locations
- Philadelphia
- United States