Assistant Professor NYU Grossman School of Medicine New York, New York
Abstract: As artificial intelligence (AI) and machine learning (ML) models become more popular in healthcare settings, many groups have documented concerns about bias, fairness, equity, justice, and privacy. These problems are not hypothetical—many models encode and reinscribe long-standing health inequities or fail to meet standards for safety and efficacy. In an effort to promote the responsible and ethical use of AI at a large academic health system, we are developing a governance system that reviews AI/ML models across their lifecycle. Our approach includes a “Responsible AI Checklist” outlining key ethical principles for AI, delineating evaluation criteria, and requiring documentation for review. Alongside this checklist, we are testing and iterating on an accompanying governance structure that promotes accountability and limits administrative burden. Based on our experiences developing and implementing this system, which includes informal interviews with stakeholders and field notes from a range of AI governance and implementation meetings over the past 18 months, we highlight challenges and opportunities to building accountability for the safety, efficacy, and fairness of AI models. We find that expertise and ownership over AI models is often spread between a range of professional roles and responsibilities. At the same time, the potential or actual harms of AI models are often hard to visualize. These factors can lead to a “responsibility vacuum,” and it is critical for AI governance systems to address this vacuum if they are to effectively ensure that AI/ML models are used ethically in healthcare settings and beyond.
After participating in this conference, attendees should be able to:
Analyze the key ethical challenges associated with AI/ML implementation in healthcare by examining issues such as bias, fairness, and the diffusion of responsibility across professional roles.
Evaluate the design and implementation of AI governance structures by assessing the effectiveness of a “Responsible AI Checklist” in promoting accountability while balancing administrative burden.
.png)