May 30, 2019

Preparing for PAMA

Radiologists and internal medicine specialists lead development and delivery of a curriculum that teaches medical interns about appropriate image ordering.
  • Radiologists and internal medicine specialists at the University of Chicago developed a curriculum that introduces first-year internal medicine interns to the ACR Appropriateness Criteria® and promotes high-value image ordering.
  • The interns unanimously agreed the curriculum presented helpful information, with 90 percent reporting that they would change their ordering practices as a result.
  • The program provides an example of how to prepare for the Protecting Access to Medicare Act, which requires clinicians to consult appropriate use criteria via clinical decision support tools when ordering advance imaging studies for Medicare Part B patients, starting on Jan. 1, 2020.

Carina W. Yang, MDAppropriate diagnostic imaging is a key element of high-value healthcare. Careful selection of imaging exams can expedite diagnoses, reduce unnecessary testing, decrease radiation exposure and other risks, and diminish healthcare costs. A 2018 analysis of healthcare spending in the United States  compared to other countries concluded that high healthcare costs in this country may trace in part to an overuse of diagnostic imaging.

While the ACR Appropriateness Criteria® (AC)  can help guide appropriate image ordering, many referring providers do not know about the guidelines, let alone use them. But change is on the way. The Protecting Access to Medicare Act (PAMA) mandates that starting Jan. 1, 2020, providers must consult appropriate use criteria (AUC) via a qualified clinical decision support (CDS) mechanism when ordering advanced diagnostic imaging for Medicare Part B patients. In preparation for this change, hospitals across the country must seek ways to integrate these tools into ambulatory clinical practice.

At the University of Chicago, an interdisciplinary team of radiologists and general internists have developed a lecture-based curriculum to raise awareness of the risks and costs of unnecessary tests. The curriculum also gives internal medicine interns tools to make more effective and informed decisions about diagnostic imaging for their patients.

These tools include facts about radiation exposure and case studies from Radiology-TEACHES , a program that radiologists at Baylor College of Medicine developed to simulate the image ordering process and teach medical students about appropriate image ordering using CDS technology. Twenty-four medical schools are using Radiology-TEACHES, including the University of Chicago’s Pritzker School of Medicine.

Now in its second year as part of the University of Chicago’s ambulatory program for internal medicine interns, the curriculum has garnered promising results, with nearly all of the interns who participated in the program reporting that they would change their ordering practices based on what they learned. The team of radiologists and general internists is now adapting the curriculum to other specialty groups, such as hospitalists and emergency medicine specialists who treat non-emergent, ambulatory patients in the ED.

“It’s an area of great need and a lot of people are interested in the topic,” says Joyce W. Tang, MD, MPH, assistant professor of hospital medicine, who has been on the team since the project’s inception. “We have the ability to really improve the appropriateness of ordering, which can be very impactful in a number of ways for patients and for healthcare costs.”

Interdisciplinary Idea

Tang conceived the idea for the curriculum with Carina W. Yang, MD, director of pediatric neuroradiology, when the pair were postgraduate fellows in the University of Chicago’s Medical Education Research, Innovation, Teaching, and Scholarship (MERITS) fellowship program.

Joyce W. Tang, MD, MPHTasked with devising an innovative medical education concept and presenting it to their colleagues in the program, Tang and Yang discussed several hot topics. But when Yang described the frequent questions she gets from referring providers about which tests to order and the number of inappropriate diagnostic imaging orders she sees each day, this struck a chord with Tang, who immediately identified with the general internist side of the image ordering process.

Given the breadth of conditions for which general internists order imaging studies, Tang admitted, “I often wonder if I’m ordering the most appropriate radiological test for a given patient.” Despite the importance of judicious image ordering to quality patient care, it is not usually covered in depth in medical school or even residency.

Recognizing this, Yang introduced Tang to the ACR AC. Tang readily recognized the criteria’s value, which dove-tailed nicely with the Choosing Wisely®  campaign and with the increasing emphasis on high-value imaging. “We realized the problem of incorrect ordering went beyond us and is quite universal,” Tang remembers. They based their curriculum on the AC and called it “Medical Overuse of Radiological Exams: MORE for less” and made a brief presentation to their colleagues in the fellowship program.

Although many ideas conceived as part of the MERITS program do not come to fruition, Vineet Arora, MD, faculty adviser for the MERITS program, saw a real need for the curriculum that Tang and Yang envisioned and suggested that they look into implementing the curriculum.

Arora connected Tang and Yang with Mike K.W. Cheng, MD, an internal medicine resident who was applying to the residents’ version of the MERITS program and who could serve as a resident champion for the project. With interests in medical education, high-value care, and quality improvement, Cheng’s skills and experience meshed well with Tang and Yang’s vision. To round out the multidisciplinary team, Yang brought radiology resident Andrea Magee, MD, onto the project.

Content Research

Still in residency, both Cheng and Magee were able to dedicate time in their schedules to the project and set to work gathering and writing curriculum content with guidance from Yang and Tang. They decided that the training would cover balancing the benefits of diagnostic imaging with the risks of radiation and other exposures, as well as the costs of the tests.

To that end, the team collected cost data for different imaging tests from the hospital billing department and compared it with average annual income levels for the surrounding communities, demonstrating the financial burden of over-testing on patients and the additional costs for adding contrast.

Andrea Magee, MDThey also created a graphic showing typical radiation doses of common diagnostic exams nationally, as well as the increase of radiation exposure in recent years due to the rise in medical testing nationwide. The goal was to not only increase awareness of the potential financial and health consequences of inappropriate testing but also to enable physicians to answer patients’ questions about the risks and benefits of recommended diagnostics.

To further emphasize appropriate image ordering, the team knew it needed to include case studies in its curriculum. Creating case studies can be arduous, but in an online search for ideas, Yang came across an Imaging 3.0 article about Radiology-TEACHES, which uses case vignettes from ACR’s CORTEX case management system and the CareSelect Imaging® CDS tool  to simulate the image ordering process.

Since the University of Chicago’s Pritzker Medical School was among the Radiology-TEACHES pilot sites, Yang and Tang spoke with Chris Straus, MD, who led the radiology clerkship that used the program at the University of Chicago. Straus, in turn, connected the team with Marc H. Willis, DO, MMM, associate professor of radiology at Baylor College of Medicine, who led development of Radiology-TEACHES with ACR support.

Willis and his team agreed to allow the University of Chicago group to use Radiology-TEACHES for the case study part of its curriculum. The Baylor team trained Cheng on the software and continues to provide support as needed. Willis says that while the original focus of Radiology-TEACHES was on medical students, he finds this expansion to interns and residents exciting. “The University of Chicago experience has been a great example of how this educational resource can benefit graduate medical education,” he says.

Curriculum Development

From there, the University of Chicago team began developing a framework for the curriculum. They wanted it to appeal to different types of learners, so they included multiple components: two lectures, simulated case studies using Radiology-TEACHES, pre- and post-test assessments, and a three-month follow-up survey. The curriculum initially targeted internal medicine interns because these physicians are at the earliest stage of their careers, when they tend to be more open to new ideas.

Team members take turns presenting the lectures and leading the discussions with each of the three groups in the internal medicine class. Each time they include one radiologist and one internal medicine specialist for an interdisciplinary approach that Willis finds particularly appealing. “Their program acts as a connector, bringing radiologists and referring providers together to learn and work together in teams with the shared goal of providing high-value care to patients,” he says.

The first lecture introduces the interns to the cost and safety data, as well as the AC. After that lecture, the interns complete a homework assignment that involves practicing image ordering with Radiology-TEACHES. During the second lecture, the interns break into small groups to discuss the cases and review what they have learned. Pre- and post-tests and a follow-up survey document learners’ progress and the curriculum’s effectiveness.

Successful Pilot

To get approval to add the curriculum to the internal medicine program, the team approached the program director for the internal medicine residency’s ambulatory curriculum and the ambulatory chief resident. While it can be difficult to get add-on curriculum approved, Cheng says the administration supported the purpose and vision of the training and, based on initial discussions, allowed the team to add the lectures to the curriculum for one academic year.

Mike K.W. Cheng, MDAs an add-on to the curriculum, the lectures start an hour before the official course, and attendance is optional. To encourage the interns to attend the first session, Cheng sent a reminder email a few days before to explain the course content, noting that residents had developed it.

Cheng later found that many of the interns were intrigued with this aspect, which seemed to add credibility to the class. “They were eager to attend to not only show their support, but also because they knew it was a topic valuable enough that internal medicine residents would want to help develop a curriculum about it.”

In the first year, 23 interns participated in the program and completed the pre- and post-tests. The results showed that before the curriculum was presented, more than a third of the interns did not know about the AC, and only 1 in 4 consulted the guidelines when ordering tests.

After the course, the interns unanimously deemed the curriculum helpful, and more than 90 percent said they would change their ordering practices as a result. Interns had improved performance on the knowledge assessment of appropriate image ordering and were more likely to reference the ACR AC tool while completing the assessment. Interns also reported feeling more comfortable discussing exam costs with patients. Three months after the conclusion of the program, 18 interns completed a follow-up survey. More than three quarters of those interns reported using the AC at least monthly.

Josh G. Waytz, MD, first year resident in internal medicine at the University of Chicago who took the class in the fall of 2018, said coverage of radiological exams during medical school had been brief and broad. He appreciated that this class was tailored to internal medicine and found himself surprised by the cost and radiation exposure data of different imaging methodologies.

“I had no idea that certain X-rays carry much more potent radiation than certain CT scans,” says Waytz, who reported that even between the two lectures he consulted the AC to more appropriately order an ankle X-ray while working in the urgent care clinic. “The AC is a hidden gem and a useful resource that I’ll definitely be using going forward,” he says.

Future Plans

Based on the success of the pilot, residency program leadership asked the development team to repeat the curriculum in 2018. The team has also adapted the curriculum for hospitalists and has plans to adapt it for emergency medicine clinicians and other primary care specialists as well as physician assistants and nurse practitioners.

In addition, the team has worked with the hospital’s IT department to integrate a link to the AC within the EHR so clinicians can easily access the guidelines when ordering exams. In the future, they hope to track whether the curriculum leads to more frequent use of this feature of the EHR and helps physicians comply with the PAMA provisions as the hospital implements a qualifying CDS.

All indications are that the curriculum will help. The curriculum includes hands-on instruction on how to consult the AC when ordering tests in the EHR. It also lets learners “look behind the curtain and gives validity to the Appropriateness Criteria,” says Cheng. “We let them know, this is our experience with it, this is the value we find in it, this is how we use it in the clinic.”

This project has shown that just two hours of training can significantly impact physician image ordering practices, increasing safety and efficacy of diagnostic imaging and helping to deliver high-value healthcare to patients. Team members have presented these results at a variety of conferences and events, including the Society of General Internal Medicine’s 2018 Annual Meeting, High Value Practice Academic Alliance National Conference, and the Radiological Society of North America’s 2018 Annual Meeting.

“It’s been really well received by the primary care communities that we’re a part of and at the conferences we’ve presented at,” says Cheng. “A lot of providers are recognizing the importance of imaging appropriateness and the need to be providers who focus on high-value care and who are accountable to our patients and our organizations. It’s rewarding to help assist them along this path to providing better patient care.”

Creative Commons

Ordering Done Right by American College of Radiology is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Based on a work at . Permissions beyond the scope of this license may be available at .

Share Your Story

Have a case study idea you’d like to share with the radiology community? To submit your idea, please click here.

Next Steps

  • Assemble an interdisciplinary team of radiology and internal medicine specialists to develop a curriculum that encourages ordering providers to use the AC when ordering imaging.
  • Work with program directors to determine what information clinicians need, which specialties and trainees to target, as well as the best way to present the information.
  • Adapt the curriculum to specific clinicians to increase provider knowledge of appropriate use diagnostic imaging and enable more informed conversations about risks and benefits with patients and their families.


Emily Paulsen is a freelance writer.

Join the Discussion


Want to join the discussion about how radiologists can lead quality improvement projects for improved image ordering? Let us know your thoughts on Twitter at #imaging3.

#Imaging3 on Twitter

Call for Case Studies


Have a suggestion for a future case study? Please share your idea with us!

Submit your idea »