On Oct. 22, 2020, the American College of Radiology® (ACR®) Data Science Institute® (DSI) Chief Science Officer, Keith J. Dreyer DO, PhD, FACR, FSIIM, presented to the Food and Drug Administration (FDA) Patient Engagement Advisory Committee on establishing trustworthiness in diagnostic imaging artificial intelligence (AI).
The FDA Patient Engagement Advisory Committee advises the Commissioner regarding agency policies, clinical trial or registry design, patient preferences, benefit-risk determinations, device labeling, patient-reported outcomes, and device-related quality of life measures or health status issues, among other topics. The Oct. 22 meeting focused on “Artificial Intelligence and Machine Learning in Medical Devices,” and covered patient-centered issues such as transparency and trustworthiness of AI, and the need for diversity and inclusion in algorithm training data.
Dreyer noted that AI has challenged the traditional FDA framework for ensuring the safety and effectiveness of medical devices. Evaluations of imaging AI algorithms have shown inconsistency in terms of dataset size and representation, and details such as demographic diversity are often not made available to inform consumers about how algorithms have been trained and evaluated. Additionally, varying imaging input devices within healthcare facilities can dramatically impact real world performance of AI algorithms cleared by FDA. These and other issues have the potential to negatively affect patient trustworthiness of AI-enabled care.
Dreyer provided an overview of the ACR DSI, which was established in 2017 to help stakeholders address usefulness, validation and monitoring of AI algorithms to enhance and expedite innovation while also improving community education and trust in imaging AI tools. The ACR DSI has participated extensively in FDA activities and other AI-focused United States government initiatives.
“At the ACR Data Science Institute, patient safety in the use of AI is a top-level goal,” said Dreyer.