March 27, 2017

Teaching Imaging Appropriateness

A simulation-based educational program increases medical students’ abilities to identify appropriate imaging.

  • Radiologists at Baylor College of Medicine partnered with ACR and National Decision Support Company to develop a web-based program that uses case vignettes and clinical decision support to teach students and practicing providers about appropriate imaging through simulation education.
  • Known as Radiology-TEACHES (Technology Enhanced Appropriateness Criteria Home for Education Simulation), the program fills an existing imaging appropriateness training gap in traditional medical education curricula.
  • The platform is scalable and can be customized to address gaps in curricula across the continuum of medical education.

Nearly every hospital patient undergoes some type of radiological exam. Yet, undergraduate medical education curricula have traditionally lacked comprehensive instruction about ordering appropriate imaging. This training gap means referring trainees and practicing providers are often unprepared to order appropriate imaging in the clinical setting — and as a result, some patients receive unnecessary imaging that can expose them to undue radiation and costs.

Improving education about appropriate imaging is imperative for better patient care, and all radiologists have an important role to play in advancing such training.

Radiologists at Baylor College of Medicine in Houston are addressing the issue by partnering with clinicians to launch a web-based program called Radiology-TEACHES , which uses case vignettes in ACR’s Radiology Case Management SystemTM (RCMS) integrated with the CareSelect ImagingTM  clinical decision support (CDS) tool to simulate the image ordering process and educate learners about appropriate imaging. (Learn more about clinical decision support in this American Medical Association STEPS ForwardTM  module.)

Baylor radiologists developed Radiology-TEACHES in collaboration with ACR and National Decision Support Company (NDSC) — the licensing agent for CareSelect Imaging, which offers the digital version of the ACR Appropriateness CriteriaTM.

In 2015, the team conducted an initial pilot of the program with 34 medical students, 85 percent of whom indicated that the program should be incorporated into the medical school curriculum.1 Now, the team is expanding the program’s reach throughout the medical community with a multisite pilot project.”

Volunteer Effort

The idea for Radiology-TEACHES was born in 2013. As ACR and NDSC began offering CareSelect Imaging to clinical practices, Marc H. Willis, DO, associate professor of radiology and associate chair for quality improvement at Baylor, started thinking. He knew ACR Select was a valuable tool to help clinicians identify appropriate imaging at the point of order, but he also recognized the potential for leveraging it in the medical education setting. (Read more about ACR Select in this Imaging 3.0 case study.)

“I saw a great opportunity for a program where learners can access case vignettes and simulate the ordering process with the ACR Select product,” says Willis, who is also the chief of musculoskeletal imaging and intervention and associate program director of the diagnostic radiology residency program at Baylor.

Willis shared his idea with his department chair and Baylor’s medical school leadership, who liked the concept but didn’t have the money to fund its development. With his leadership’s support, Willis turned to ACR and NDSC, which both agreed to partner with him on the project.

Soon thereafter, efforts began to build the program’s user interface with a direct link to CareSelect Imaging. “Connecting to CareSelect Imaging was essential to give students access to evidence-based appropriateness criteria, information about how to order contrast, relative radiation dose data, and a cost range for different imaging exams,” Willis explains.

From there, Willis approached one of his colleagues, Karla Sepulveda, MD, associate professor of radiology and fellow associate program director of Baylor’s diagnostic radiology residency, about co-leading the program’s development. Sepulveda, who is also the radiology department’s director of medical student education, agreed, and together she and Willis assembled a team of colleagues to help author the pilot program’s case vignettes.

The team included representatives from eight radiology sub-specialties: musculoskeletal, breast imaging, gastrointestinal imaging, neuroradiology, thoracic and cardiac imaging, genitourinary imaging, women’s imaging, and vascular imaging.

For two years, the team spent countless hours authoring cases in ACR’s RCMS, a computer application for collecting, storing, and distributing educational materials in various formats. (Learn more about RCMS in this Imaging 3.0 case study.) “Without a budget, we primarily worked on this during nights and weekends as volunteers,” Willis says. During that time and since then, the team has authored more than 150 vignettes, covering topics in each of the represented sub-specialty areas. In the future, users will be able to author their own cases to fit specific educational needs

Cross-Domain Collaboration

By January of 2015, Radiology-TEACHES was ready for its initial pilot. Willis approached Baylor’s medical school leadership and faculty about using the program with their students. Nadia J. Ismail, MD, MPH, associate dean of clinical sciences, says she immediately saw value in the project, which aligns with medicine’s shift toward high-value care as well as with Baylor’s efforts to make radiology more explicit in its curriculum. “Radiology is really core to most patient care, so any chance we have to educate our students about it, the better,” she says.

With faculty members on board, Willis and his team recruited volunteers to participate in the pilot from a group of second-year medical students who had completed their clinical classroom work and were entering clinical rotations. Volunteers were recruited through verbal announcements at the start of their regular classes and through emails, including one from the associate dean of Baylor’s Office of Undergraduate Medical Education. As a result of these efforts, 34 students signed up to participate in the project, which did not impact their grades.

Allison M. Khoo, a third-year medical student at Baylor who plans to specialize in interventional radiology, was one of the students who volunteered for the pilot. Khoo says she signed up to learn about the decision-making process for different types of imaging studies — training she wouldn’t have otherwise received as an undergraduate. “Generally, we don’t learn much about imaging, like when to use contrast, what kinds of contrasts are available, the differences between CT and MRI, or the relative costs of different exams,” she explains. “This, unfortunately, opens the door for a lot of overuse and wasteful imaging in the clinical setting.”

Initial Pilot

Willis kicked off the pilot with a 10-minute presentation to student participants. He explained what evidence-based medicine and CDS are and how they can optimize decision-making to improve patient outcomes. Then, the students took a 20-minute pre-assessment exam to gauge their ability to order appropriate imaging before logging into the Radiology-TEACHES portal to review a few practice cases.

Once the students understood how to use the platform, they had two weeks to work through 48 case vignettes within the program. The format is learner-directed, allowing each student to complete the vignettes at his or her own pace.

“One advantage of this educational model is that it creates an asynchronous learning environment, where students can log in remotely and do the cases,” Willis says. “It doesn’t have to fit into classroom time.”

Here’s how the program works:

  1. A student logs into the Radiology-TEACHES web-based portal and reads a case vignette, outlining a patient’s clinical scenario.
  2. Then the student clicks on the CareSelect Imaging tab, where he or she enters the patient’s demographic and clinical information.
  3. From there, CareSelect Imaging provides a list of imaging exams, ranking them from the most to least appropriate based on the appropriateness criteria — just as it does in the clinical setting.
  4. The student then selects the study he or she wants to order and, most importantly, receives immediate decision support feedback before moving to the next case vignette.

Positive Results

At the end of the two-week pilot, participants took a post-assessment exam. Compared to the pre-assessment test, the students showed a “statistically significant” improvement in identifying the most appropriate imaging exams in multiple areas: intermediate cases, advanced cases, and cases covering Choosing Wisely topics.

“Most encouraging was the students’ improvement (p=.0207) in vignettes covering Choosing Wisely  topics, a national campaign aimed at reducing the amount of waste in medicine,” Willis says.

The students also reported feeling more prepared to order appropriate imaging after participating in the program. On the pre-assessment test, 70 percent of participating students said they were “unprepared” or “totally unprepared” to order appropriate imaging. On the post-assessment test, after the simulation module, 53 percent of participants said they were at least “slightly prepared” to order appropriate imaging.

Additionally, the students provided feedback about the program’s value as a curriculum tool, with 96 percent reporting that it provided “some value” or was “very valuable” or “extremely valuable.”

“Radiology-TEACHES really fills a gap in our current working knowledge in medicine,” Khoo says. “I feel fortunate to have had the opportunity to get this robust training early in my career; it has definitely bolstered my interest in becoming a radiologist. Hopefully, more people will use this tool in the future." 

Additional Pilots

To turn that hope for further dissemination into reality, Willis and his team have begun several additional Radiology-TEACHES projects. One of them involves piloting the program with Baylor’s physician assistant students, bringing appropriate imaging education to the allied health arena. Willis notes that this is an important project because, in many practices, physician assistants often order imaging studies.

In another effort, Willis and his team are collaborating with the directors of Baylor’s seven required medical student core clinical clerkships (general surgery, internal medicine, family and community medicine, pediatrics, neurology, obstetrics-gynecology, and psychiatry) to integrate the program into those rotations. This project blossomed after Ismail invited Willis and his team to present Radiology-TEACHES at a clerkship director’s meeting, and Willis received a 2016 Radiological Society of North America Education Scholar Grant to fund the initiative.

Andrew C. Caruso, MD, assistant professor of medicine and director of Baylor’s internal medicine clerkship, says he looks forward to sharing the program with the students in his rotation. “Radiology-TEACHES will expose students to the various imaging studies and help them learn which ones are safe and cost-effective based on a patient’s clinical condition,” he says. “This evidence-based information is vital for them to experience prior to becoming trainees.”

To expand Radiology-TEACHES’ footprint even further, the Baylor, ACR, and NDSC team has initiated a multisite pilot at four other institutions across the nation: University of Chicago, Montefiore Medical Center, Uniformed Services University of the Health Sciences, and Augusta University. The team expects this project to prove that the program’s success at Baylor is transferable to other institutions.

Additionally, Willis and his team are creating case vignettes for ACR’s Radiology Support, Communication, and Alignment Network (R-SCAN™ ), an initiative that brings radiologists and referring physicians together to improve image ordering. Willis is an advisor for R-SCAN, which is funded through the Centers for Medicare and Medicaid Service’s Transforming Clinical Practice Initiative . The modules are slated to be available through R-SCAN in the first quarter of 2017.

Future Efforts

Once these additional projects are completed, the Baylor, ACR, and NDSC team plans to make Radiology-TEACHES available as an educational resource to other academic institutions, hospitals, and practices. Willis anticipates that at that time more medical schools, particularly those with highly engaged radiology departments, will implement the program to fill similar gaps in their curricula. He also expects hospital and medical groups to find value in the program as a continuing education tool to teach practicing providers more about appropriate imaging.

“The goal of Radiology-TEACHES is to prepare current and future medical providers to deliver high-value health care by decreasing waste and increasing patient safety through improved imaging utilization, as outlined in health care’s Triple Aim,” Willis explains. “The platform provides a simulated education environment that is flexible and scalable. It can be used across a wide spectrum — from individual learners to inter-professional teams — ensuring patients receive the best care possible. That’s something we should all be focused on as we transition from volume- to value-based care.”


  1. Willis, MH, Frigini, LA, Lin J, et al. Clinical Decision Support at the Point-of-Order Entry: An Education Simulation Pilot with Medical Students. 2016;23(10):1309-1318. Available at: 


 Discussion Questions

  • How would students at your institution benefit from clinical decision support software and embedded case vignettes to simulate the ordering process?
  • How would you organize a team of volunteers from various radiology subspecialties to author the case vignettes?
  • How would you recruit volunteers, particularly second-year medical students who are entering clinical rotations, to participate in a pilot project like this?

Next Steps

  • Determine whether Radiology-TEACHES is a good fit for your academic institution, medical practice, hospital, or healthcare system.
  • Start engaging collaborators and leadership within your institution to begin exploring ways to integrate this program into your organization’s portfolio of educational resources.
  • Email to learn more and engage in the program.

Creative Commons

Teaching Imaging Appropriateness by American College of Radiology is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Based on a work at Permissions beyond the scope of this license may be available at

Share Your Story

Have a case study idea you’d like to share with the radiology community? To submit your idea, please click here.


Jenny Jones, Imaging 3.0 project specialist

Join the Discussion


Want to join the discussion about how radiologists can lead quality improvement projects for improved image ordering? Let us know your thoughts on Twitter at #imaging3.

#Imaging3 on Twitter

Call for Case Studies


Have a suggestion for a future case study? Please share your idea with us!

Submit your idea »