1. Are quality improvement strategies to improve outpatient antibiotic use effective?

Our review found that the vast majority of published studies reported clinically significant improvements in antibiotic treatment and selection. The magnitude of these effects compares favorably to those achieved by quality improvement efforts in other settings. These findings were consistent across diverse patient populations and clinical settings. More than half the included studies were performed in the last decade; the concomitant decline in antibiotic prescribing in US ambulatory practices suggests that efforts to promote judicious antibiotic use are succeeding. However, inappropriate prescribing rates remain high, and inappropriate selection of antibiotics presents a continuing challenge.

In addition to benefiting patients and the community by reducing the adverse consequences of inappropriate antibiotic use, effective interventions may result in cost savings, although definitive evidence is lacking.

2. What are the critical components of effective intervention strategies to improve outpatient antibiotic use?

Antibiotic treatment studies. Within the antibiotic treatment studies, we were able to compare studies using clinician education alone with those using clinician and patient education. The addition of patient education to clinician education did not result in greater reductions in antibiotic prescribing, a finding that withstood evaluation for potential negative confounding. No distinct differences emerged among strategies, with the exception of a possible trend toward greater effectiveness in studies using active educational strategies. This finding was supported by systematic evaluation of studies that were not incorporated in the median effects analysis. ^{122, 126, 128}

Potential negative confounding by unmeasured variables and limited power may affect our finding of an apparent lack of additional benefit of patient education over clinician education alone. Although studies included in median effects analyses did not show an additional benefit, two large population-based studies ^{122, 126} not incorporated in median effects analyses did demonstrate reductions in antimicrobial prescribing with a combined clinician and patient educational intervention. Of note, these studies were conducted in the U.S., but most included studies were from outside the U.S. It is conceivable that different practice styles, levels of baseline prescribing rates, and patient expectation of antibiotic treatment may make patient education more important in the U.S. than outside the country.

Antibiotic selection studies . Most selection trials eligible for median effects analysis employed one of two main QI strategies: clinician education alone, or clinician education in combination with audit and feedback. The absolute difference in median effect between these strategies was substantial; interventions adding audit and feedback to clinician education were less effective than interventions employing clinician education alone. The effectiveness of other types of QI strategies is difficult to systematically assess, since there were only three studies for all other categories of QI strategies.

The surprising finding that adding audit and feedback to clinician education results in smaller effects on prescribing may be explained in part by confounding. Interventions employing clinician education alone were substantially more likely to have below-median sample size; these smaller studies had overall larger effect sizes. While this may reflect publication bias, it is also possible that low sample size is acting as a proxy for studies that use local relationships and leadership to achieve greater clinician “buy-in” than studies spread over many sites. In addition to publication bias and sample size already noted, there may be other subtle confounders. For example, interventions that employed audit and feedback may have used less intensive methods to implement the clinician education component, spreading their energies among several intervention strategies rather than focusing on one. These possibilities are intriguing, but not directly testable in this analysis. A prudent conclusion may be that we are unable to definitively assess the relative efficacy of clinician education alone vs. clinician education combined with audit and feedback, but that we found no evidence to suggest that the combined approach is superior to clinician education alone.

Unlike the antibiotic treatment studies, there was no prominent association between the presence of active vs. passive types of educational interventions and antibiotic prescribing outcomes. It is difficult to assess whether this lack of association is true, or is merely negatively confounded by other observed or unobserved variables, compounded by limited power to detect differences due to low sample sizes. Other findings suggest that active educational interventions may in fact produce better outcomes. First, the differences in median effect size between active and passive techniques were in the expected direction, and of similar magnitude to differences observed in treatment studies, in which a trend (P=0.11) toward an association was observed, suggesting insufficient power. In addition, studies that compare two or more interventions to one another can be viewed as controlling for the many measured and unmeasured confounders that make inter-study comparisons so challenging. ¹⁶⁰ In each of these cases, the active group outperformed the passive group. ^{135, 152}

Our overall results thus supply cautious support for including active clinician education in quality improvement efforts for improving antibiotic use. Combining other strategies with clinician education does not necessarily improve outcomes, but our ability to detect such differences was limited.

3. Which patients and conditions should be targeted in order to exert the maximal impact on antibiotic prescribing?

We did not find any evidence that targeting specific patient populations resulted in significant differences in study effects, in either antibiotic treatment or antibiotic selection studies.

When selecting an intervention target, stakeholders and policymakers may wish to consider the potential population-level intervention effect, rather than simply the target-specific effect. Interventions that have highly significant effects on antibiotic prescribing for single conditions or limited patient populations may not necessarily exert large effects on overall antibiotic prescribing rates. At the population level, targeting all ARIs appears to translate into larger reductions in antibiotic consumption than focusing on a single condition or patient age range, and this might be an important factor in the willingness of purchasers/payors of health care to invest in appropriate antibiotic use interventions.

Interventions targeting antibiotic selection appeared to be equally effective across different disease processes and patient populations. Thus, stakeholders should determine the quality gap in their unique patient population, and target interventions appropriately.

4. What are the limitations of current research in this field and what areas require further study?

Included studies rarely reported important measures of the potential harms of the intervention. These include the potential for increased use of health services (e.g., return visits due to persistent symptoms) and adverse clinical consequences (e.g., increasing rates of serious infections due to undertreatment). Patient satisfaction does not appear to be affected in the limited number of studies that did report this outcome.

More importantly, very few of the included studies (and none of the US-based treatment studies) documented the resources required for completion of the intervention and measurements. Only four ^{102, 121, 143, 144} studies reported the cost savings resulting from changing prescribing behavior. Thus, although an active clinician education intervention may be more effective than a passive intervention, we are unable to make any statement regarding the cost-effectiveness of such an intervention. Given the apparent benefit of other, potentially less resource-intensive interventions, further research should include formal reporting of both costs of and cost savings from QI interventions. Further studies should also clearly document intervention intensity and reach, the baseline quality gap, and any other local factors that could have affected the intervention or outcomes measurement.

Changes in antibiotic resistance rates should be monitored not only for declines in resistance rates, but also for changes in the rate of rise in existing or new resistance patterns. This will require longer-term followup than measured in most studies. Based on the best evidence and mathematical modeling to date, it is unlikely that reductions in antibiotic consumption will lead to major reductions in levels of antibiotic resistance among community-acquired bacterial infections. ³³ – ³⁵ However, ecological studies do support the notion that the amount of antibiotic consumption in a community directly influences how rapidly new resistance emerges or rises. ^{7, 37} As the benefits from preventing antimicrobial resistance will be seen largely by the community at large, health care organizations may be reluctant to invest heavily in programs to reduce antimicrobial use without a clear business case. A demonstration that these types of interventions can recover their implementation costs through savings in antibiotic costs would be helpful. Our crude analysis in Figure 7 indicates that many of these interventions have the potential to be cost-neutral or cost-saving, depending on the cost of the intervention, the reduction in antibiotic prescriptions and the cost of antibiotics.

Study design and quality should be improved. Studies that formally evaluate the cost effectiveness of interventions to improve antibiotic treatment and selection are needed, and studies should evaluate the potential harms of such interventions.