What is the history of the ACR Appropriateness Criteria®?

The ACR created the Task Force on Appropriateness Criteria (AC) and appointed panel chairs in late 1993. By 1994, development of nationally accepted, scientifically based guidelines had begun. The Appropriateness Criteria were developed to support the ordering of appropriate imaging for given patient clinical conditions by referring physicians. This groundbreaking effort provided the College’s perspective on how to best use limited health care resources.

In 2000, the task force became the Committee on Appropriateness Criteria under the ACR Commission on Quality and Safety. Together, panel leaders and the chair of the Committee on Appropriateness Criteria act as a steering committee to oversee the activities of consensus panels.

The task force’s processes incorporated attributes for developing acceptable medical practice guidelines used by the Agency for Healthcare Research and Quality (AHRQ) as designed by the Institute of Medicine. From the beginning, the methodology relied on a combination of evidence, and when data from well-controlled trials were insufficient, expert consensus. Importantly, the methodology continues to employ the input of physicians from other medical specialties to provide relevant clinical perspectives.

Currently the ACR Appropriateness Criteria® include 17 panels: 10 diagnostic, one covering interventional radiology, and nine radiation oncology with input from over 20 medical specialty societies.

Are the ACR Appropriateness Criteria evidence-based?

The ACR Appropriateness Criteria® are evidence-based guidelines to assist referring physicians and other providers in making the most appropriate imaging or treatment decision for a specific clinical condition. By employing these guidelines, providers enhance quality of care and contribute to the most efficacious use of medical imaging.

Expert panels led by the ACR in diagnostic imaging, interventional radiology and radiation oncology have developed the guidelines. Each panel includes leaders in radiology and other specialties. There are 197 topics with over 900 variants in the November 2013 version.

What does it mean if a procedure is appropriate for a specific clinical scenario?

The ACR has adopted the definition of appropriateness from the Ambulatory Quality Alliance:

The concept of appropriateness, as applied to health care, balances risk and benefit of a treatment, test, or procedure in the context of available resources for an individual patient with specific characteristics. Appropriateness criteria provide guidance to supplement the clinician’s judgment as to whether a patient is a reasonable candidate for the given treatment, test or procedure.1

1From the AQA Principles for Appropriateness Criteria. These principles are a subset of the general AQA Parameters for Selecting Measures for Physician Performance. They are not to be viewed independently of that document.

What is the rating scheme for appropriateness of the ACR AC?

The ACR methodology of rating appropriateness is based on the RAND / UCLA Appropriateness Method (Fitch 2001). The appropriateness rating scale is an ordinal scale from 1-9 grouped into three categories: 1, 2 or 3 are in the category “usually not appropriate”; 4, 5 or 6 are “may be appropriate”; and 7, 8 or 9 “usually appropriate.”

A more detailed explanation of the complete rating process can be found under Supporting Documents at www.acr.org/ac.

How is the evidence categorized?

The evidence is based on over 5,000 peer reviewed medical publications, each with various strengths of evidence. These represent all relevant publications contained within the medical literature today. The ACR Clinical Decision Support (CDS) committee evaluates each AC reference according to Strength of Evidence.

  • Category 1: The study is well designed and accounts for common biases
  • Category 2: The study is moderately well designed and accounts for most common biases
  • Category 3: There are important study design limitations
  • Category 4: The study is not useful as primary evidence. The article may not be a clinical study or the study design is invalid, or conclusions are based on expert consensus.

Appropriateness Criteria Evidence Categories

97% of AC guidelines are informed by Category 1 or 2 references
3% of AC guidelines are informed by Category 3 references

Is the ACR AC content transparent to the public?

The ACR AC are available on the ACR website for research purposes. Also, the National Guidelines Clearinghouse (NGC), one of the initiatives of AHRQ, is a public resource for evidence-based clinical practice guidelines. The ACR AC topics are submitted, reviewed and posted on the NGC site.

The ACR believes that imaging CDS systems that are used routinely to deliver health care best serve the public when they are widely available, backed by evidence and transparent.

Is there any federal certification of the quality of the evidence?

The ACR AC has been developed using the guidelines published by the Agency for Healthcare Research and Quality (AHRQ). The ACR AC meet all the criteria for evidence-based medicine and as such have been published on the National Guideline Clearing house, which is an initiative of the AHRQ.

What happens when evidence is conflicting?

ACR AC uses an expert review of all relevant literature. Taking into account the quality of evidence for each publication, the expert panel clarifies any discrepancies among the published evidence. All such clarifications are published in the literature review summaries associated with each published ACR AC guideline.

How often is the content updated and revised?

Each AC is revised every three years.

What is ACR Select and how does it differ from the ACR AC®?

The AC has been converted into a digital knowledge base. This has been combined with recommendations from other sources that have been reviewed and vetted by the ACR and have been integrated into the digital knowledge base to create a complete, clinically consumable Clinical Decision Support (CDS) system for imaging. This Imaging CDS system is ACR Select. ACR Select comprises both the platform to manage and deliver an Imaging CDS system, as well as the evidence-based content.

ACR Select makes the ACR AC more accessible to and digitally consumable by health care providers, either directly or through integration with health care IT platforms (such as electronic medical records, clinical decision support systems or computerized physician order entry systems) as part of its Imaging 3.0 toolkit.

How is ACR Select delivered?

The ACR Select platform includes an integration layer such that the content can be incorporated into health care IT platforms and clinical workflows. The ACR Select platform includes a web portal to access the content and augment the workflows of health care IT platforms as required.

National Decision Support Company provides the licensing and technical support of the ACR Select platform into the market so that health care providers can improve the delivery of imaging services and access ACR Select in their clinical workflows.

Why does ACR Select use all available evidence rather than only evidence from controlled trials?

Highest quality evidence may be scarce in some areas of medicine for a variety of reasons. Researchers may not have studies for some diagnostic procedures for specific clinical conditions. If a high-quality study exists, it may not be repeated. Or research may never be performed to study the value of examinations that are unlikely to be appropriate for given conditions.

The ACR includes expert, consensus-driven content in addition to the evidence that exists in ACR Select for these circumstances to ensure a complete and useable Imaging CDS.

Can ACR Select use decision trees and deep algorithmic logic?

The greatest benefit from CDS is in the initial presentation of patients and that is what ACR Select has chosen to focus on. Experience and feedback from users of early Imaging CDS platforms has led the ACR and NDSC to focus on a single-layer, single-question model for the presentation of the CDS logic to the users. Adding clicks and additional questions at the point of order does little to improve the quality of result, and unnecessarily burdens the ordering physician.

The platform and presentation state are, however, decoupled. The platform is multilayered; the presentation model has been purposely “flattened” to make the content useable at CPOE.

Physician adoption is critical to achieving the kind of health care outcomes the ACR believes are possible with ACR Select and CPOE. Creating additional overhead for already busy physicians in the form of decision trees is not conducive to physician adoption.

Should a CDS system be able to provide expert consensus guidance as well as evidence-based guidance?

A CDS system should be able to provide the ordering clinician all information available at the time of order, provided that it also exposes the source and quality of all guidance, including the distinction between evidence-based and expert, consensus-based content. ACR Select can also limit the presentation of CDS based on quality of evidence. Experience has dictated that presenting the most complete set of guidance possible improves the physician experience with CDS.

Practical experience with CDS systems has dictated that presenting the most complete set of guidance possible improves the physician experience.

Why is ACR Select important to me as a radiologist?

When used in the CPOE workflow, ACR Select ensures the most appropriate imaging procedure based upon a structured set of clinical indications.

The rule set is evidence-based and reflects more than 20 years of the ACR’s expertise and collaboration with other medical societies.

The forces at work within the current health care delivery model are moving away from volume-based models towards value-based payment models. Furthermore the reimbursement for imaging services continues to decline without regard to the value of imaging on patient care.

By using ACR Select to guide the ordering physician towards more appropriate imaging, radiologists can be at the forefront of the dialogue regarding how best to utilize imaging in the care cycle and collaborate on new value-driven models for the delivery of imaging services.

ACR Select allows radiology to demonstrate the value of properly delivered imaging services on improving the quality of care. CDS can help drive radiology as a consultative service when clinicians encounter moderate to low level of appropriateness scores.

Bibb Allen, Jr., MD, FACR, has published a presentation outlining the value proposition for radiologists to adopt clinical decision support.