ACR Bulletin

Covering topics relevant to the practice of radiology

Democratizing AI

A multi-site federated learning approach to AI algorithm training can protect patient privacy and help make AI development more generalizable to widespread clinical use.
Jump to Article
Tags
RecommendThanks for recommending this page!

The page you recommended will be added to the "what others are reading" feed on "My ACR".

 BookmarkThanks for bookmarking this page!

The page you bookmarked will be added to the "my reading list" feed on "My ACR".

To accelerate the adoption of AI into clinical practice, the ACR recognizes the need to involve more and more institutions in algorithm training, testing, and validation.

November 30, 2020

The hype around AI and its impact on radiology is growing. More and more AI algorithms are emerging, of which 84 are currently FDA cleared. Preliminary results from the ACR Data Science Institute® (DSI) survey show that, in practice, less than 30% of radiologists are using AI algorithms — including 10% that are using self-developed algorithms.

These numbers are hardly surprising, given the current challenges the industry is facing. The development of AI algorithms is primarily happening in institutions with extensive informatics and data science resources. Furthermore, this development usually happens in single institutions, as it is difficult to share data outside of institutions due to patient privacy concerns, and only a small percentage of FDA-cleared algorithms have undergone external validation. These are key limiting factors, especially because algorithms need widespread exposure to a variety of equipment and patient demographics to be generalizable to widespread clinical use.

To accelerate the adoption of AI into clinical practice, the ACR recognizes the need to involve more and more institutions in algorithm training, testing, and validation. This democratization of AI includes a multi-site federated learning approach for training AI algorithms by including data from a variety of practices — while protecting patient privacy. By including data from multiple sites, federated learning allows AI models to evolve and become less brittle when exposed to the amalgam of equipment and patient demographics that will be seen in actual clinical use. This need for democratization led the DSI to develop the AI-LAB™, the ACR’s data science toolkit that empowers radiologists to use their own patient data to participate in algorithm evaluation and development. The ACR’s revamped IT communication platform, ACR Connect, provides the means to transfer analytical tools and AI algorithms to a variety of sites, so that the sites can then safely use their own data for federated learning. Additionally, AI-LAB and ACR Connect provide the infrastructures to securely access local data for multiple purposes, such as hands-on experience and education in medical imaging AI, model creation or tuning, and model validation. Commercial algorithms seeking FDA clearance or those developed at single institutions seeking to become more generalizable could be transferred and validated at multiple institutions without the need to transfer local data. Finally, the ACR is also working with a number of developers to use AI-LAB as a means to allow facilities to evaluate AI algorithms using their own data prior to purchase.

Preliminary results from the ACR Data Science institute® (DSI) survey show that, in practice, less than 30% of radiologists are using AI algorithms — including 10% that are using self-developed algorithms.

While some of the features of AI-LAB are available now in the ACR cloud, to test these on-premises features, the ACR has deployed AI-LAB on site at seven institutions. The Tufts University School of Medicine’s Lahey Hospital and Medical Center is one of those sites, and during the 2020 Imaging Informatics Summit, Christoph Wald, MD, MBA, PhD, FACR, Adam Medina, and Ali Ardestani, MD, shared their first-hand experience in using AI-LAB. The team emphasized that even though they are a smaller institution with limited institutional IT support, installation and implementation were straightforward. They discussed each fundamental step of their workflow — the hardware selection, the institution’s IT policy, the institutional review board process, and the installation of AI-LAB. According to Wald, professor of radiology at Tufts University Medical School and chair of the ACR Commission on Informatics, the data processing step — which required the identification of series of images, multi-reader assurance of ground truth, and standardized annotation — was the most time-consuming, while the actual algorithm development was surprisingly fast and straight-forward.

The Lahey team’s successful implementation of AI-LAB demonstrates that AI can truly be democratized. The DSI is planning to install AI-LAB in more than 20 additional institutions in the next phase, with hopes for widespread deployment next year. AI-LAB and ACR Connect will seamlessly allow every radiologist and institution to evaluate and use AI to enhance the care they provide their patients.

Author Dan Cohen-Addad, MD, is a radiology resident at SUNY Downstate Medical Center and a member of the ACR Commission on Informatics’ Advisory Council.