NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Empirical Evaluation of the Association Between Methodological Shortcomings and Estimates of Adverse Events
Technical Reviews, No. 13
Investigators: Roger Chou, MD, Rongwei Fu, PhD, Susan Carson, MPH, Somnath Saha, MD, MPH, and Mark Helfand, MD, MPH.
Author Information and AffiliationsStructured Abstract
Objective:
Accurate harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Numerous deficiencies in the quality and reporting of harms associated with clinical interventions have been reported. Little is known, however, about whether such perceived methodological shortcomings are associated with lower estimates of harms.
Study Design and Setting:
Studies reporting harms associated with (1) carotid endarterectomy (CEA) for symptomatic stenosis, (2) rofecoxib for arthritis, and (3) CEA for asymptomatic stenosis were identified using published systematic reviews. A standardized abstraction form, including eight pre-defined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms. A quality-rating instrument for studies reporting harms was developed based on the results of the analyses of studies of CEA for symptomatic stenosis. The quality-rating instrument was tested on the other data sets.
Results:
In 111 studies of CEA for symptomatic stenosis, meeting five of the eight quality criteria was associated with significantly higher rates of stroke or death. A quality-rating instrument with four of the five criteria predicted adverse events (5.7 percent in studies rated “adequate”, compared to 3.9 percent in studies rated “inadequate” [p=0.0003]). In multivariate analyses, the quality rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted lower estimates of risk for myocardial infarction in 16 trials of rofecoxib, and none of the quality criteria predicted lower estimates of stroke of death in 18 studies of CEA for asymptomatic stenosis. Evaluation of the latter two data sets was limited by small numbers of trials and low rates of evaluated adverse events.
Conclusions:
The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should carefully consider the potential effects of individual methodological shortcomings when evaluating estimates of harms associated with clinical interventions. However, we were unable to develop a generic summary quality-rating instrument for studies of harms because specific methodological shortcomings were not consistently predictive across data sets.
Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services.1 Contract No. 290-02-0024. Prepared by: Oregon Evidence-based Practice Center, Portland, OR.
Suggested citation:
Chou R, Fu R, Carson S, Saha S, Helfand M. Empirical Evaluation of the Association Between Methodological Shortcomings and Estimates of Adverse Events. Technical Review No. 13. (Prepared by the Oregon Evidence-based Practice Center under Contract No. 290-02-0024.) AHRQ Publication No. 07-0003. Rockville, MD: Agency for Healthcare Research and Quality. June 2006.
This report is based on research conducted by the Oregon Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-02-0024). The findings and conclusions in this document are those of the authors, who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
The information in this report is intended to help clinicians, employers, policymakers, and others make informed decisions about the provision of health care services. This report is intended as a reference and not as a substitute for clinical judgment.
This report may be used, in whole or in part, as the basis for the development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.
Dr. Saha was supported by a Research Career Development award from the Health Services Research and Development Service of the Department of Veterans Affairs and by a Generalist Physician Faculty Scholar award from the Robert Wood Johnson Foundation.
- 1
540 Gaither Road, Rockville, MD 20850. www
.ahrq.gov
- NLM CatalogRelated NLM Catalog Entries
- Review Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions.[J Clin Epidemiol. 2007]Review Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions.Chou R, Fu R, Carson S, Saha S, Helfand M. J Clin Epidemiol. 2007 Jan; 60(1):18-28. Epub 2006 Sep 7.
- Review Lipid Screening in Childhood and Adolescence for Detection of Familial Hypercholesterolemia: A Systematic Evidence Review for the U.S. Preventive Services Task Force[ 2016]Review Lipid Screening in Childhood and Adolescence for Detection of Familial Hypercholesterolemia: A Systematic Evidence Review for the U.S. Preventive Services Task ForceLozano P, Henrikson NB, Dunn J, Morrison CC, Nguyen M, Blasi PR, Anderson ML, Whitlock E. 2016 Aug
- Review Screening for Asymptomatic Carotid Artery Stenosis: A Systematic Review and Meta-Analysis for the U.S. Preventive Services Task Force[ 2014]Review Screening for Asymptomatic Carotid Artery Stenosis: A Systematic Review and Meta-Analysis for the U.S. Preventive Services Task ForceJonas DE, Feltner C, Amick HR, Sheridan S, Zheng ZJ, Watford DJ, Carter JL, Rowe CJ, Harris R. 2014 Jul
- Review Screening for Chronic Obstructive Pulmonary Disease: A Systematic Evidence Review for the U.S. Preventive Services Task Force[ 2016]Review Screening for Chronic Obstructive Pulmonary Disease: A Systematic Evidence Review for the U.S. Preventive Services Task ForceGuirguis-Blake JM, Senger CA, Webber EM, Mularski R, Whitlock EP. 2016 Apr
- Review Low-Dose Aspirin for the Prevention of Morbidity and Mortality From Preeclampsia: A Systematic Evidence Review for the U.S. Preventive Services Task Force[ 2014]Review Low-Dose Aspirin for the Prevention of Morbidity and Mortality From Preeclampsia: A Systematic Evidence Review for the U.S. Preventive Services Task ForceHenderson JT, Whitlock EP, O'Conner E, Senger CA, Thompson JH, Rowland MG. 2014 Apr
- Empirical Evaluation of the Association Between Methodological Shortcomings and ...Empirical Evaluation of the Association Between Methodological Shortcomings and Estimates of Adverse Events
Your browsing activity is empty.
Activity recording is turned off.
See more...