AI has the potential to aid radiologists in numerous ways, evidenced in part by the extensive list of radiologist-defined AI use cases compiled by the ACR Data Science Institute^®. Despite this huge potential, there are challenges facing the development of clinically useful AI models.

Many of the challenges stem from the fact that imaging data features vary from facility to facility. This occurs for various reasons. While some are well understood, such as variability in equipment manufacturers or patient populations, many are hard to interpret. We do know that AI models trained on data from a single facility are biased toward that facility’s population and do not generalize well across facilities. In practice, this means that AI models trained on data from Facility A can perform poorly when tested on data from Facility B. Ultimately, this slows the radiology community in developing clinically useful models that minimize bias and health inequity.

One way to get around this issue is to acquire training data in a central location from a diverse array of source facilities. Unfortunately, due to the sensitive nature of medical imaging data, this is often not a viable option. That is where federated learning (FL) can help. It offers a solution where multiple facilities participate in model training, while data remain local to each facility and secure behind their firewalls.

Tackling the Data Sharing Problem

FL is a method of collaborative AI model training that does not require data sharing. During FL, AI models, not data, are shared among participating facilities and a coordinating server. Facilities train models locally before passing them to a coordinating server, which then aggregates multiple facility-trained local models into a single AI model. In this way, FL allows model training across a distributed network of facilities.

AI models created through FL are generally more robust and able to be generalized across facilities than single-facility-trained models because training data derives from more diverse sources and patient populations. The performance of final AI models produced with FL often reach levels of performance similar to models trained on data collected in a central location.

Recently, the ACR participated in an FL initiative that produced a breast-density classification model, which performed better on a test dataset consisting of data from all participating facilities than any of the single-facility-trained local models produced in the study.

Offering Analysis Tools

ACR Connect^® is free ACR-developed software, which can be installed on premises. It offers various medical imaging analysis tools and makes them readily accessible through a series of applications. ACR Connect applications make it possible to participate in cross-site research initiatives and create curated local datasets, without the need for data to leave a site’s protective firewalls.

The AI-LAB™ app in ACR Connect provides even non-technical users with the ability to create or evaluate AI models through an easily navigated graphical interface. Since AI-LAB’s focus is on both educating the radiology community about AI and providing a tool to help radiologists develop robust AI models, AI-LAB offers a simple interface and various background materials to bring radiologists up to speed about AI. This includes videos explaining important AI concepts and content throughout the application. As a result, AI-LAB users without any technical expertise can participate in AI development and validation and the radiology community can build expertise in developing clinically useful models.

Producing Better AI Models

AI-LAB is introducing new FL capabilities. At the conceptual level, the process of FL is fairly straightforward with AI-LAB via ACR Connect. To begin, an ACR central server passes a base model, which was developed previously to perform the same or similar task but found to have non-ideal performance, to the ACR Connect instance at each participating facility. At each facility, the base model is then trained on a local dataset prepared by the site, which is used to create a new site-specific model.

New models from each site will be passed back to the ACR central server for aggregation. The process of aggregation will create a single AI model that will be used as the base model in the next round of training. When repeated over many rounds, this process will produce a final AI model that performs well at all participating facilities. On the ACR side, a custom AI-LAB implementation will leverage NVIDIA FLARE™ , which enables multi-party orchestration, training, model sharing, and model aggregation.

For a more detailed look at how ACR Connect clinical integration services work, visit the ACR Clinical Integration Services page.