ACR 2026: Evaluating AI Within Your Practice

At ACR 2026, the College’s annual meeting, an expert panel — including Tessa Cook, MD, PhD, FACR, incoming chair of the Commission on Informatics, Christoph Wald, MD, PhD, MBA, FACR, chair of the Board of Chancellors, Bernardo Bizzo, MD, PhD, MSc, Data Science and Informatics (DSI) Associate Chief Science Officer, and Keith Dreyer, DO, PhD, FACR, DSI Chief Data Science Officer — held an open-mic Listen and Learn session focused on AI. Dr. Wald welcomed the crowd and introduced Dr. Cook as the incoming chair of the ACR Commission on Informatics.

Dr. Cook dove into the abundance of tools ACR provides to help practices determine which AI tool is right for them and how to integrate it into their practice. AI Central, for example, allows users to compare and verify AI tools available for clinical use so they can compare and contrast tools and analyze the features of each. AI Central has a unique error rate estimation calculator that takes into account observed or reported performance compared to vendor-reported performance. Dr. Cook then shifted the focus to Assess AI, the first national registry for AI tools being used in clinical practice. Assess AI uses data to help determine how well AI is performing in various practice environments and how to best utilize AI modules.

A lunch attendee related their experience communicating with vendors about AI technology modules. The vendor only wanted short answers, while the radiologist wanted to give more detail about their issues with the modules. The panel recommended using a governance model to work with subspecialties to find the right AI module. They also agreed with letting vendors know specific errors the AI module made.

The panel also fielded how to handle AI brokers that offer a free trial to test the technology’s performance. Panelists stressed the importance of having a neutral evaluation strategy when it comes to AI, recommending using Assess AI to size up the technology.

Another attendee asked about properly evaluating AI within their practice to ensure it is working as the vendor stated it would. The panel pointed to the benefits of comparing AI products through AI Central and that practices who sign up can also submit their own data to help others make decisions.

Panelists fielded inquiries about strategies to expedite the legal process. They agreed it is a difficult facet of AI to navigate and comes down to each individual practice and what would truly benefit them.

The panel talked about how AI tools can be helpful for workflows and shared their personal experiences. They encouraged attendees to talk with their radiology partners to let them know that this is the future and it is worth an investment.

Panelists provided this takeaway: while AI won’t replace radiologists, it could change the specialty’s efficiency and how they are billed; it is important we keep collecting data and be wary of recently trained AI technology that could become obsolete.

By Alex Utano, associate editor, ACR Bulletin.