Take Action: Contact Your US Senators to Cosponsor ROOT Act
Support the ROOT Act—Senate cosponsors needed to fix CMS issues with imaging AUC under PAMA. College member advocacy is key.
Read moreA new FDA-funded study conducted by ACR DSI members, FDA researchers, and two medical institutions found that two widely used clinical artificial intelligence (AI) algorithms cleared by the FDA for emergent triage of suspected stroke patients displayed decreased performance in real world settings compared to the manufacturers’ reported performance under certain specific circumstances encountered in local practice.
The paper, “Real-World Performance of Large Vessel Occlusion Artificial Intelligence–Based Computer-Aided Triage and Notification Algorithms—What the Stroke Team Needs to Know,” was recently published in the Journal of the American College of Radiology (JACR).
The study looked at two FDA-cleared computer-aided-triage/notification (CADt) devices for large-vessel occlusion (LVO) detection and used data collected from two US medical centers. For both CADt systems, the researchers observed similar sensitivity to that reported by device manufacturers in the larger cranial vessels, such as the Internal Carotid Artery (ICA) and Middle Cerebral Artery (MCA) M1 (proximal) segments. Decreased performance was observed in the further distal and smaller MCA M2 and all other intracranial first and second order branches. In addition, there were quite a few cases where the algorithm could not interpret the imaging data.
Another issue is related to the local implementation of the technology: the AI output is commonly presented to clinical decision makers on mobile platforms proffered by the same manufacturers. The information is relayed in near real time motivated by a very appropriate desire of all involved to increase speed of triage. In reality, there is the potential risk that the speed and mode of information distribution (1) disintermediates the qualified radiologist from validating the AI results, and (2) results intended purely for triage of patient assessment are mistaken for diagnoses as there is minimal to no warning that the AI results are for triage only. As such — in some practice settings — the real-world implementation of the technology may impact clinical decision making in ways other than what the technology was cleared for by the FDA. Especially when real time expert radiologist interpretation is not available.
The study provides timely clinical evidence illustrating an urgent patient safety concern, which the FDA addressed in April 2022 by sending an urgent “Letter to Health Care Providers” to compel manufacturers to remind providers of the intended use of the algorithms, relaying that LVO CADt devices should never be used as a replacement for informed interpretation by an imaging physician when making important treatment decisions.
Take Action: Contact Your US Senators to Cosponsor ROOT Act
Support the ROOT Act—Senate cosponsors needed to fix CMS issues with imaging AUC under PAMA. College member advocacy is key.
Read moreACR MSS Annual Chair Update
ACR MSS Chair Joshua Baker reflects on MSS milestones, student-led initiatives, and future goals, celebrating mentorship and inclusion at ACR 2025.
Read moreCareer Growth Through Volunteer Work
Dr. Bryant Chang shares how early involvement with his state chapter led to launching a medical student section, mentorship program and a symposium.
Read more