NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Criteria for Distinguishing Effectiveness From Efficacy Trials in Systematic Reviews
Technical Reviews, No. 12
Investigators: Gerald Gartlehner, MD, MPH, Richard A Hansen, PhD, Daniel Nissman, MD, MPH, Kathleen N Lohr, PhD, and Timothy S Carey, MD, MPH.
Author Information and AffiliationsStructured Abstract
Objectives:
To propose and test a simple instrument based on seven criteria of study design to distinguish effectiveness (pragmatic) from efficacy (explanatory) trials while conducting systematic reviews.
Design:
Currently, no validated definition of effectiveness studies exists. We asked the directors of 12 Evidence-based Practice Centers (EPCs) to select six studies each: four that they considered to be examples of effectiveness trials and two considered efficacy studies. We then applied our proposed criteria to test the construct validity using the selected studies as if they had been identified by a gold standard.
Results:
Based on the rationale to identify effectiveness studies reliably with minimal false positives (i.e., a high specificity), a cut-off of six criteria produced the most desirable balance between sensitivity and specificity. This setting produced a specificity of 0.83 and a sensitivity of 0.72.
Conclusions:
When applied in a standardized manner, our proposed criteria can provide a valid and simple tool to distinguish effectiveness from efficacy studies. The applicability of systematic reviews can improve when analysts place more emphasis on the generalizability of included studies. In addition, clinicians can also use our criteria to determine the external validity of individual studies given an appropriate population of interest.
Contents
Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services.1 Contract No. 290-02-0016. Prepared by: RTI-International-University of North Carolina Evidence-based Practice Center, Research Triangle Park, NC.
Suggested citation:
Gartlehner G, Hansen RA, Nissman D, Lohr KN, Carey TS. Criteria for Distinguishing Effectiveness From Efficacy Trials in Systematic Reviews. Technical Review 12 (Prepared by the RTI-International-University of North Carolina Evidence-based Practice Center under Contract No. 290-02-0016.) AHRQ Publication No. 06-0046. Rockville, MD: Agency for Healthcare Research and Quality. April 2006.
This report is based on research conducted by the RTI-International-University of North Carolina Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-02-0016). The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
The information in this report is intended to help clinicians, employers, policymakers, and others make informed decisions about the provision of health care services. This report is intended as a reference and not as a substitute for clinical judgment.
This report may be used, in whole or in part, as the basis for the development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.
- 1
540 Gaither Road, Rockville, MD 20850. www
.ahrq.gov
- NLM CatalogRelated NLM Catalog Entries
- A simple and valid tool distinguished efficacy from effectiveness studies.[J Clin Epidemiol. 2006]A simple and valid tool distinguished efficacy from effectiveness studies.Gartlehner G, Hansen RA, Nissman D, Lohr KN, Carey TS. J Clin Epidemiol. 2006 Oct; 59(10):1040-8. Epub 2006 Aug 4.
- Review Screening for Chronic Obstructive Pulmonary Disease: A Systematic Evidence Review for the U.S. Preventive Services Task Force[ 2016]Review Screening for Chronic Obstructive Pulmonary Disease: A Systematic Evidence Review for the U.S. Preventive Services Task ForceGuirguis-Blake JM, Senger CA, Webber EM, Mularski R, Whitlock EP. 2016 Apr
- How has the impact of 'care pathway technologies' on service integration in stroke care been measured and what is the strength of the evidence to support their effectiveness in this respect?[Int J Evid Based Healthc. 2008]How has the impact of 'care pathway technologies' on service integration in stroke care been measured and what is the strength of the evidence to support their effectiveness in this respect?Allen D, Rixson L. Int J Evid Based Healthc. 2008 Mar; 6(1):78-110.
- Review Inadequate reporting of trials compromises the applicability of systematic reviews.[Int J Technol Assess Health Ca...]Review Inadequate reporting of trials compromises the applicability of systematic reviews.Gartlehner G, Thieda P, Hansen RA, Morgan LC, Shumate JA, Nissman DB. Int J Technol Assess Health Care. 2009 Jul; 25(3):323-30.
- Review Screening for Preeclampsia: A Systematic Evidence Review for the U.S. Preventive Services Task Force[ 2017]Review Screening for Preeclampsia: A Systematic Evidence Review for the U.S. Preventive Services Task ForceHenderson JT, Thompson JH, Burda BU, Cantor A, Beil T, Whitlock EP. 2017 Apr
- Criteria for Distinguishing Effectiveness From Efficacy Trials in Systematic Rev...Criteria for Distinguishing Effectiveness From Efficacy Trials in Systematic Reviews
Your browsing activity is empty.
Activity recording is turned off.
See more...