Critical evaluation of the scientific content in clinical practice guidelines

Zaid M. Abdelsattar, Bradley N Reames, Scott E. Regenbogen, Samantha Hendren, Sandra L. Wong

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

BACKGROUND: Increasing pressures to provide high-quality evidence-based cancer care have driven the rapid proliferation of clinical practice guidelines (CPGs). The quality and validity of CPGs have been questioned, and adherence to guidelines is relatively low. The purpose of this study was to critically evaluate the development process and scientific content of CPGs. METHODS: CPGs addressing management of rectal cancer were evaluated. We quantitatively assessed guideline quality with the validated Appraisal of Guidelines Research & Evaluation (AGREE II) instrument. We identified 21 independent processes of care using the nominal group technique. We then compared the evidence base and scientific agreement for the management recommendations for these processes of care. RESULTS: The quality and content of rectal cancer CPGs varied widely. Mean overall AGREE II scores ranged from 27% to 90%. Across the 5 CPGs, average scores were highest for the clarity of presentation domain (85%; range, 58% to 99%) and lowest for the applicability domain (21%; range, 8% to 56%). Randomized controlled trials represented a small proportion of citations (median, 18%; range, 13%-35%), 78% of the recommendations were based on low-or moderate-quality evidence, and the CPGs only had 11 references in common with the highest-rated CPG. There were conflicting recommendations for 13 of the 21 care processes assessed (62%). CONCLUSIONS: There is significant variation in CPG development processes and scientific content. With conflicting recommendations between CPGs, there is no reliable resource to guide high-quality evidence-based cancer care. The quality and consistency of CPGs are in need of improvement.

Original languageEnglish (US)
Pages (from-to)783-789
Number of pages7
JournalCancer
Volume121
Issue number5
DOIs
StatePublished - Mar 1 2015

Fingerprint

Practice Guidelines
Rectal Neoplasms
Guidelines
Guideline Adherence
Neoplasms
Randomized Controlled Trials
Pressure

Keywords

  • Evidence-based medicine
  • Health services
  • Organizations
  • Practice guidelines
  • Rectal neoplasms

ASJC Scopus subject areas

  • Oncology
  • Cancer Research

Cite this

Abdelsattar, Z. M., Reames, B. N., Regenbogen, S. E., Hendren, S., & Wong, S. L. (2015). Critical evaluation of the scientific content in clinical practice guidelines. Cancer, 121(5), 783-789. https://doi.org/10.1002/cncr.29124

Critical evaluation of the scientific content in clinical practice guidelines. / Abdelsattar, Zaid M.; Reames, Bradley N; Regenbogen, Scott E.; Hendren, Samantha; Wong, Sandra L.

In: Cancer, Vol. 121, No. 5, 01.03.2015, p. 783-789.

Research output: Contribution to journalArticle

Abdelsattar, ZM, Reames, BN, Regenbogen, SE, Hendren, S & Wong, SL 2015, 'Critical evaluation of the scientific content in clinical practice guidelines', Cancer, vol. 121, no. 5, pp. 783-789. https://doi.org/10.1002/cncr.29124
Abdelsattar, Zaid M. ; Reames, Bradley N ; Regenbogen, Scott E. ; Hendren, Samantha ; Wong, Sandra L. / Critical evaluation of the scientific content in clinical practice guidelines. In: Cancer. 2015 ; Vol. 121, No. 5. pp. 783-789.
@article{9608a346d83f4514bd2f3174d84782d4,
title = "Critical evaluation of the scientific content in clinical practice guidelines",
abstract = "BACKGROUND: Increasing pressures to provide high-quality evidence-based cancer care have driven the rapid proliferation of clinical practice guidelines (CPGs). The quality and validity of CPGs have been questioned, and adherence to guidelines is relatively low. The purpose of this study was to critically evaluate the development process and scientific content of CPGs. METHODS: CPGs addressing management of rectal cancer were evaluated. We quantitatively assessed guideline quality with the validated Appraisal of Guidelines Research & Evaluation (AGREE II) instrument. We identified 21 independent processes of care using the nominal group technique. We then compared the evidence base and scientific agreement for the management recommendations for these processes of care. RESULTS: The quality and content of rectal cancer CPGs varied widely. Mean overall AGREE II scores ranged from 27{\%} to 90{\%}. Across the 5 CPGs, average scores were highest for the clarity of presentation domain (85{\%}; range, 58{\%} to 99{\%}) and lowest for the applicability domain (21{\%}; range, 8{\%} to 56{\%}). Randomized controlled trials represented a small proportion of citations (median, 18{\%}; range, 13{\%}-35{\%}), 78{\%} of the recommendations were based on low-or moderate-quality evidence, and the CPGs only had 11 references in common with the highest-rated CPG. There were conflicting recommendations for 13 of the 21 care processes assessed (62{\%}). CONCLUSIONS: There is significant variation in CPG development processes and scientific content. With conflicting recommendations between CPGs, there is no reliable resource to guide high-quality evidence-based cancer care. The quality and consistency of CPGs are in need of improvement.",
keywords = "Evidence-based medicine, Health services, Organizations, Practice guidelines, Rectal neoplasms",
author = "Abdelsattar, {Zaid M.} and Reames, {Bradley N} and Regenbogen, {Scott E.} and Samantha Hendren and Wong, {Sandra L.}",
year = "2015",
month = "3",
day = "1",
doi = "10.1002/cncr.29124",
language = "English (US)",
volume = "121",
pages = "783--789",
journal = "Cancer",
issn = "0008-543X",
publisher = "John Wiley and Sons Inc.",
number = "5",

}

TY - JOUR

T1 - Critical evaluation of the scientific content in clinical practice guidelines

AU - Abdelsattar, Zaid M.

AU - Reames, Bradley N

AU - Regenbogen, Scott E.

AU - Hendren, Samantha

AU - Wong, Sandra L.

PY - 2015/3/1

Y1 - 2015/3/1

N2 - BACKGROUND: Increasing pressures to provide high-quality evidence-based cancer care have driven the rapid proliferation of clinical practice guidelines (CPGs). The quality and validity of CPGs have been questioned, and adherence to guidelines is relatively low. The purpose of this study was to critically evaluate the development process and scientific content of CPGs. METHODS: CPGs addressing management of rectal cancer were evaluated. We quantitatively assessed guideline quality with the validated Appraisal of Guidelines Research & Evaluation (AGREE II) instrument. We identified 21 independent processes of care using the nominal group technique. We then compared the evidence base and scientific agreement for the management recommendations for these processes of care. RESULTS: The quality and content of rectal cancer CPGs varied widely. Mean overall AGREE II scores ranged from 27% to 90%. Across the 5 CPGs, average scores were highest for the clarity of presentation domain (85%; range, 58% to 99%) and lowest for the applicability domain (21%; range, 8% to 56%). Randomized controlled trials represented a small proportion of citations (median, 18%; range, 13%-35%), 78% of the recommendations were based on low-or moderate-quality evidence, and the CPGs only had 11 references in common with the highest-rated CPG. There were conflicting recommendations for 13 of the 21 care processes assessed (62%). CONCLUSIONS: There is significant variation in CPG development processes and scientific content. With conflicting recommendations between CPGs, there is no reliable resource to guide high-quality evidence-based cancer care. The quality and consistency of CPGs are in need of improvement.

AB - BACKGROUND: Increasing pressures to provide high-quality evidence-based cancer care have driven the rapid proliferation of clinical practice guidelines (CPGs). The quality and validity of CPGs have been questioned, and adherence to guidelines is relatively low. The purpose of this study was to critically evaluate the development process and scientific content of CPGs. METHODS: CPGs addressing management of rectal cancer were evaluated. We quantitatively assessed guideline quality with the validated Appraisal of Guidelines Research & Evaluation (AGREE II) instrument. We identified 21 independent processes of care using the nominal group technique. We then compared the evidence base and scientific agreement for the management recommendations for these processes of care. RESULTS: The quality and content of rectal cancer CPGs varied widely. Mean overall AGREE II scores ranged from 27% to 90%. Across the 5 CPGs, average scores were highest for the clarity of presentation domain (85%; range, 58% to 99%) and lowest for the applicability domain (21%; range, 8% to 56%). Randomized controlled trials represented a small proportion of citations (median, 18%; range, 13%-35%), 78% of the recommendations were based on low-or moderate-quality evidence, and the CPGs only had 11 references in common with the highest-rated CPG. There were conflicting recommendations for 13 of the 21 care processes assessed (62%). CONCLUSIONS: There is significant variation in CPG development processes and scientific content. With conflicting recommendations between CPGs, there is no reliable resource to guide high-quality evidence-based cancer care. The quality and consistency of CPGs are in need of improvement.

KW - Evidence-based medicine

KW - Health services

KW - Organizations

KW - Practice guidelines

KW - Rectal neoplasms

UR - http://www.scopus.com/inward/record.url?scp=84923291139&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923291139&partnerID=8YFLogxK

U2 - 10.1002/cncr.29124

DO - 10.1002/cncr.29124

M3 - Article

VL - 121

SP - 783

EP - 789

JO - Cancer

JF - Cancer

SN - 0008-543X

IS - 5

ER -