Virtual 2020 SIIM-ACR Data Science Summit

Monitoring and Evaluating AI: Challenges and Practical Implications

Access recordings of all 6 sessions from the Virtual 2020 SIIM-ACR Data Science Summit, held on June 23, and earn up to 5.0 CME.

Whether you are a developer looking for insights from radiology leaders, or a radiologist with an informatics background seeking best practices, the Summit can help you understand best practices to evaluate AI models and provide strategies to overcome these potential barriers to AI adoption.

Course Overview:

  • 5 hours of focused content
  • 6 sessions of brief keynote presentations
  • In depth panel discussions

Course Objectives:

Upon completion of the Summit, the participant will be able to:

  • Identify phases of the AI lifecycle.
  • Explain hurdles and steps to regulatory clearance.
  • Define evaluations of AI and common issues including bias, brittleness, and fairness.
  • Cite the tools and resources available to aid in the review and evaluation of AI models.
  • Explain the steps of algorithm assessment and validation.
  • Outline strategies for evaluating performance and monitoring AI algorithms.

The 2020 summit was held in conjunction with the virtual SIIM Annual Meeting.

Pricing

 Member $50
 Non-Member $75
 Member-in-Training Free

 

Order now

Agenda

Session 1: Regulatory Clearance and Approval of AI Models (1 Hour)

  • AI Life Cycle Management (Keith J. Dreyer, DO, PhD, FACR, FSIIM)
  • FDA Classifications and Requirements for AI Algorithms (Howard Chen, MD, MBA)
  • Intro to AI Validation and Post-Market Monitoring (Bibb Allen Jr., MD, FACR)
    15 minute Panel Discussion

Session 2: Science of Evaluation: Bias, Fairness, Brittleness, and Explainability (1 Hour)

  • Science of Evaluation and Explainability (Jayashree Kalpathy-Cramer, PhD)
  • Brittleness of AI Models (Woojin Kim, MD)
  • Bias and Fairness in AI Models (Monica J. Wood, MD)
    15 minute Panel Discussion

Session 3: ACR Data Science Institute's AI-LAB (30 Minutes)

  • Overview of AI-LAB (Mike Tilkin)

Session 4: Training, Validation, and Generalizability: Lessons Learned (45 Minutes)

  • Pre-Market Assessment and Validation (Judy W. Gichoya, MD, MS)
  • Algorithm Improvement: Customization and Distributed and Federated learning (Peter D. Chang, MD)
  • Standardizing Deployment of AI Algorithms (Neil Tenenholtz, PhD and Brian J. Bialecki, CIIP, CDIP, CAHIMS)
    15 Minute Panel Discussion

Session 5: Evaluating Performance and Monitoring Algorithms (1 Hour)

  • Importance of Testing and Ongoing Monitoring of AI Models (Woojin Kim, MD)
  • Tools for Monitoring Effectiveness of AI Algorithms (Stuart R. Pomerantz, MD)
  • Assess-AI: Monitoring Algorithm Performance in Clinical Practice (Axel W. E. Wismüller, M.D., M.Sc., Ph.D.)
  • Ongoing Clinical Monitoring (Daniel S. Chow, MD)
    15 minute Panel Discussion

Session 6: ACR Data Science Institute's AI-LAB Pilot Sites (45 Minutes)

  • Current Status of AI-LAB Pilot Sites (Laura Coombs, PhD)
  • Lahey Hospital Pilot Site (Ali Ardestani, MD MSc CIIP)
  • Massachusetts General Hospital Pilot Site (Romane Gauriau)
    15 Minute Panel Discussion

CME Information

Earn 5.0 CME

Activity Date: September 1, 2020
Expiration Date: September 1, 2021

Physician Accreditation Statement:
The American College of Radiology is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.

Credit Designation Statement:
The American College of Radiology designates this enduring material for a maximum of 5.0 AMA PRA Category 1 Credits™. Physicians should claim only the credit commensurate with their participation in the activity. 

For information about the accreditation of this program, please contact the ACR at info@acr.org.


Informatics

Informatics_640

Radiologists are leaders in applying technological advances to practical medical use. ACR tools and technology are at the forefront of this effort.

Access tools

Data Science Institute



The ACR Data Science Institute™ facilitates the development and implementation of artificial intelligence (AI) applications that will help radiology professionals provide improved medical care.

Learn more