Brandon J. Nelson, PhD
Rongping Zeng, PhD
Marla Sammer, MD, MHA
Donald P. Frush, MD, FACR, FAAP, FSABI
Jana G. Delfino, PhD
As new artificial intelligence (AI) powered tools become available in wider applications ranging from image processing and interpretation to text summarization and more, there has been growing excitement in leveraging these tools to improve healthcare such as through new actionable insights from the growing volumes of clinical data and improved efficiency. Such advances may additionally benefit by reducing physician burnout. Despite the many opportunities posed by AI, a recent white paper by the American College of Radiology’s Pediatric AI Working Group1 points out potential health equity issues raised by the current lack of AI tool development for pediatric patients. The white paper highlights many unique aspects of pediatric care that need to be accounted for in AI development and use while calling upon everyone to take action to “Image IntelliGently™”2 to ensure that pediatric patients can also benefit from medical advancements made possible with AI.
Among these calls to action, is a promotion of transparency in AI model development and testing as it relates to pediatric populations. This transparency between device developers, regulators, and ultimately users is essential to ensuring that pediatric patients also can reliably benefit from AI tools. As AI tools are only in the pioneering stage of entering clinical practice, many of the recommendations from Image IntelliGently™ have yet to be implemented. For example, what can device users do now to promote safe and effective use of AI devices in pediatric patients? A recent article, “An FDA Guide on Indications for Use and Device Reporting of Artificial Intelligence-Enabled Devices: Significance for Pediatric Use” by authors from the US Food and Drug Administration (FDA) and members of the Pediatric AI Work Group describes key steps healthcare providers and patients can take right now to promote informed use of AI/ML-enabled medical devices, ensuring they are used appropriately and perform effectively.3
These steps include accessing a device’s Indications For Use (IFU) which describes what the device does and who it is intended for, as well as the device user manual which often provides information on how the device was developed and tested. Understanding the conditions and patient demographics of the data used in training and testing a device is more important than ever as AI-based devices are susceptible to poor performance when used outside of their training data distribution. This article illustrates the importance of understanding the Indications For Use of a device with a representative scenario of a device user unsure of whether a device is suitable for their pediatric patients and summarizes the steps that user can take to find that information.
As an example, consider Doctor X, who wishes to take an AI-powered chest x-ray screening device they have used in adults and extend its use to pediatric patients coming through their clinic. Doctor X knows that AI-powered devices generally don’t perform well on patients or conditions they weren’t designed for. Unsure of whether their AI screening device was designed to work with pediatric patients, Doctor X decides to pull up the FDA 510(k) database to find the device’s Indications for Use. The Indications for Use (IFU for short) describe the disease the device is intended to diagnose or treat and the intended population for which the device has been demonstrated to be safe and effective. Doctor X enters the device name into the database and finds the IFU inside the retrieved 510(k) summary for the device. Doctor X finds that the device is indicated for both adult and pediatric patients, but no specific age range is given.
The FDA review of a device hinges on the Indications for Use, in that the Indications for Use of a device must be supported by performance data in the premarket application. For Doctor X, this means that their AI screening device, which is indicated for adults and pediatric patients, has been tested and proven to work in both adult and pediatric patient groups.
This is a good start for Doctor X, but pediatrics refers to a large age and size range of patients younger than 22 years old,4 and Doctor X wants to know if the device will work with their younger infant patients. Infants are considerably smaller than adolescents, yet both are considered pediatric patients and thus are included in the indications.
To find more information about how the screening device was tested, Doctor X looks to the 510(k) summary, which should provide a brief summary of the supporting information included in the regulatory submission, as well as the device user manual, typically requested from the manufacturer. Unable to find more in the user manual, Doctor X finally reaches out to the manufacturer to directly request more information on how the device was developed and tested, specifically the age and size range of patients. When dealing with X-ray devices, such as this AI-based chest x-ray screening device, patient size is often more relevant than patient age when it comes to device performance.5
A representative from the manufacturer responds to Doctor X saying that the youngest patient their AI-screening device was tested on was 12 years old. Doctor X often sees patients far younger than this and now knows to be more cautious when interpreting the results from the AI device if they decide to use the device on patients younger than 12.
In this example, Doctor X was diligent in their search for device validation in pediatric patients. Following the steps described in the FDA Indications for Use in AI Devices Guide3 and summarized in the Figure below, they started with information freely available online including the 510(k) summary and IFU. When Doctor X couldn’t find the answer in public documents, they moved on to the user manual and asking the manufacturer about questions that remained.
The scenario described above is based on a real study, where a device indicated for patients 18 years and older was found to have uncertain performance in healthy children and poor performance in those younger than 2 years old.6 This illustrates the importance of understanding the device IFU as well as the conditions and patient characteristics a device was validated on as AI devices can be notoriously brittle outside of these conditions.
While some existing indications for use and 510(k) summaries may be inadequate to fully understand the conditions a device was validated on, the FDA and device users recognize the greater importance in understanding the training and testing data used in AI model development. Thus, new efforts are ongoing to make these documents more descriptive and transparent. In the meantime, device users are encouraged to be diligent and follow the necessary steps to being fully informed on appropriate device usage to ensure devices work as intended on our most vulnerable patients.
This article has only discussed some of what is described in the full FDA Guide on Indications for Use in AI Devices3 and interested readers are encouraged to check out the full article and follow the references below to learn more. The full article also discusses adverse event report and the tools available to users when devices do not work as intended and how these tools promote communication between users, device developers, the FDA, and the public.
Artificial Intelligence represents an exciting new chapter in medical device development, making new actionable insights into patient health and improving efficiency and outcomes. However, since these devices also introduce new challenges around generalizability and new guides on transparency are being developed, it is essential that users leverage all existing means to understand how these devices were developed and tested to ensure safe and effective use for all patients young and old, big and small.
References
1. Sammer MBK, Akbari YS, Barth RA, et al. Use of Artificial Intelligence in Radiology: Impact on Pediatric Patients, a White Paper From the ACR Pediatric AI Workgroup. J Am Coll Radiol. Published online July 25, 2023. doi:10.1016/j.jacr.2023.06.003
2. Image IntelliGentlyTM. Accessed August 26, 2023. https://www.acr.org/Practice-Management-Quality-Informatics/Informatics/Pediatric-Radiology-AI-Resources
3. Nelson BJ, Zeng R, Sammer M, Frush DP, Delfino JG. An FDA Guide on Indications for Use and Device Reporting of Artificial Intelligence-Enabled Devices: Significance for Pediatric Use. J Am Coll Radiol. Published online July 1, 2023. doi:10.1016/j.jacr.2023.06.004
4. Health C for D and R. Premarket Assessment of Pediatric Medical Devices. U.S. Food and Drug Administration. Published March 24, 2020. Accessed March 1, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/premarket-assessment-pediatric-medical-devices
5. Health C for D and R. Pediatric Information for X-ray Imaging Device Premarket Notifications. U.S. Food and Drug Administration. Published March 6, 2019. Accessed March 1, 2023. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/pediatric-information-x-ray-imaging-device-premarket-notifications
6. Shin HJ, Son NH, Kim MJ, Kim EK. Diagnostic performance of artificial intelligence approved for adults for the interpretation of pediatric chest radiographs. Sci Rep. 2022;12(1):10215. doi:10.1038/s41598-022-14519-w
Advocacy to Access: Missouri’s Breast Screening Success
Radiology advocacy plays a key role in improving patient care, according to a new Neiman HPI study published in JACR.
Read more
#FocusedForwardTogether
Discussion of upcoming advocacy efforts taking place during the ACR 2025 annual meeting and how members can participate.
Read more
Lessons from Hurricane Helene and Beyond
Radiology as a profession has weathered countless storms in recent memory. The many unprecedented challenges of the COVID-19 years; seemingly endless CMS cuts and declining reimbursement in the face of soaring imaging volumes and workforce shortages; dangerous scope of practice expansion by unqualified individuals, and the increasingly prevalent corporate practice of medicine.
Read more
