Evaluation of the effectiveness of a surgical checklist in Medicare patients

Bradley N. Reames, Christopher P. Scally, Jyothi R. Thumma, Justin B. Dimick

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Background: Surgical checklists are increasingly used to improve compliance with evidence-based processes in the perioperative period. Although enthusiasm exists for using checklists to improve outcomes, recent studies have questioned their effectiveness in large populations. Objective: We sought to examine the association of Keystone Surgery, a statewide implementation of an evidence-based checklist and Comprehensive Unit-based Safety Program, on surgical outcomes and health care costs. Methods: We performed a study using national Medicare claims data for patients undergoing general and vascular surgery (n = 1,002,241) from 2006 to 2011. A difference-in-differences approach was used to evaluate whether implementation was associated with improved surgical outcomes and decreased costs when compared with a national cohort of nonparticipating hospitals. Propensity score matching was used to select 10 control hospitals for each participating hospital. Costs were assessed using pricestandardized 30-day Medicare payments for acute hospitalizations, readmissions, and high-cost outliers. Results: Keystone Surgery implementation in participating centers (N = 95 hospitals) was not associated with improved outcomes. Difference-in-differences analysis accounting for trends in nonparticipating hospitals (N = 950 hospitals) revealed no differences in adjusted rates of 30-day mortality [relative risk (RR) = 1.03; 95% confidence intervals (CI), 0.97-1.10], any complication (RR = 1.03; 95% CI, 0.99-1.07), reoperations (RR = 0.89; 95% CI, 0.56-1.22), or readmissions (RR = 1.01; 95% CI, 0.97-1.05). Medicare payments for the index admission increased following implementation (516 average increase in payments; 95% CI, 210-823 increase), as did readmission payments (564 increase; 95% CI, 89-1040 increase). High-outlier payments (965 increase; 95% CI, 974decrease to 2904 increase) did not change. Conclusions: Implementation of Keystone Surgery in Michigan was not associated with improved outcomes or decreased costs in Medicare patients.

Original languageEnglish (US)
Pages (from-to)87-94
Number of pages8
JournalMedical Care
Volume53
Issue number1
DOIs
StatePublished - Jan 20 2015

Fingerprint

Medicare
Checklist
Confidence Intervals
Costs and Cost Analysis
Propensity Score
Perioperative Period
Reoperation
Health Care Costs
Blood Vessels
Hospitalization
Outcome Assessment (Health Care)
Safety
Mortality
Population

Keywords

  • Administrative Data
  • Cost Analysis
  • Effectiveness
  • Observational Studies
  • Outcomes Research
  • Quality Mprovement
  • Surgery

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health

Cite this

Evaluation of the effectiveness of a surgical checklist in Medicare patients. / Reames, Bradley N.; Scally, Christopher P.; Thumma, Jyothi R.; Dimick, Justin B.

In: Medical Care, Vol. 53, No. 1, 20.01.2015, p. 87-94.

Research output: Contribution to journalArticle

Reames, Bradley N. ; Scally, Christopher P. ; Thumma, Jyothi R. ; Dimick, Justin B. / Evaluation of the effectiveness of a surgical checklist in Medicare patients. In: Medical Care. 2015 ; Vol. 53, No. 1. pp. 87-94.
@article{1ab8b4bf0e84499eb23d25080b3b43b5,
title = "Evaluation of the effectiveness of a surgical checklist in Medicare patients",
abstract = "Background: Surgical checklists are increasingly used to improve compliance with evidence-based processes in the perioperative period. Although enthusiasm exists for using checklists to improve outcomes, recent studies have questioned their effectiveness in large populations. Objective: We sought to examine the association of Keystone Surgery, a statewide implementation of an evidence-based checklist and Comprehensive Unit-based Safety Program, on surgical outcomes and health care costs. Methods: We performed a study using national Medicare claims data for patients undergoing general and vascular surgery (n = 1,002,241) from 2006 to 2011. A difference-in-differences approach was used to evaluate whether implementation was associated with improved surgical outcomes and decreased costs when compared with a national cohort of nonparticipating hospitals. Propensity score matching was used to select 10 control hospitals for each participating hospital. Costs were assessed using pricestandardized 30-day Medicare payments for acute hospitalizations, readmissions, and high-cost outliers. Results: Keystone Surgery implementation in participating centers (N = 95 hospitals) was not associated with improved outcomes. Difference-in-differences analysis accounting for trends in nonparticipating hospitals (N = 950 hospitals) revealed no differences in adjusted rates of 30-day mortality [relative risk (RR) = 1.03; 95{\%} confidence intervals (CI), 0.97-1.10], any complication (RR = 1.03; 95{\%} CI, 0.99-1.07), reoperations (RR = 0.89; 95{\%} CI, 0.56-1.22), or readmissions (RR = 1.01; 95{\%} CI, 0.97-1.05). Medicare payments for the index admission increased following implementation (516 average increase in payments; 95{\%} CI, 210-823 increase), as did readmission payments (564 increase; 95{\%} CI, 89-1040 increase). High-outlier payments (965 increase; 95{\%} CI, 974decrease to 2904 increase) did not change. Conclusions: Implementation of Keystone Surgery in Michigan was not associated with improved outcomes or decreased costs in Medicare patients.",
keywords = "Administrative Data, Cost Analysis, Effectiveness, Observational Studies, Outcomes Research, Quality Mprovement, Surgery",
author = "Reames, {Bradley N.} and Scally, {Christopher P.} and Thumma, {Jyothi R.} and Dimick, {Justin B.}",
year = "2015",
month = "1",
day = "20",
doi = "10.1097/MLR.0000000000000277",
language = "English (US)",
volume = "53",
pages = "87--94",
journal = "Medical Care",
issn = "0025-7079",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - Evaluation of the effectiveness of a surgical checklist in Medicare patients

AU - Reames, Bradley N.

AU - Scally, Christopher P.

AU - Thumma, Jyothi R.

AU - Dimick, Justin B.

PY - 2015/1/20

Y1 - 2015/1/20

N2 - Background: Surgical checklists are increasingly used to improve compliance with evidence-based processes in the perioperative period. Although enthusiasm exists for using checklists to improve outcomes, recent studies have questioned their effectiveness in large populations. Objective: We sought to examine the association of Keystone Surgery, a statewide implementation of an evidence-based checklist and Comprehensive Unit-based Safety Program, on surgical outcomes and health care costs. Methods: We performed a study using national Medicare claims data for patients undergoing general and vascular surgery (n = 1,002,241) from 2006 to 2011. A difference-in-differences approach was used to evaluate whether implementation was associated with improved surgical outcomes and decreased costs when compared with a national cohort of nonparticipating hospitals. Propensity score matching was used to select 10 control hospitals for each participating hospital. Costs were assessed using pricestandardized 30-day Medicare payments for acute hospitalizations, readmissions, and high-cost outliers. Results: Keystone Surgery implementation in participating centers (N = 95 hospitals) was not associated with improved outcomes. Difference-in-differences analysis accounting for trends in nonparticipating hospitals (N = 950 hospitals) revealed no differences in adjusted rates of 30-day mortality [relative risk (RR) = 1.03; 95% confidence intervals (CI), 0.97-1.10], any complication (RR = 1.03; 95% CI, 0.99-1.07), reoperations (RR = 0.89; 95% CI, 0.56-1.22), or readmissions (RR = 1.01; 95% CI, 0.97-1.05). Medicare payments for the index admission increased following implementation (516 average increase in payments; 95% CI, 210-823 increase), as did readmission payments (564 increase; 95% CI, 89-1040 increase). High-outlier payments (965 increase; 95% CI, 974decrease to 2904 increase) did not change. Conclusions: Implementation of Keystone Surgery in Michigan was not associated with improved outcomes or decreased costs in Medicare patients.

AB - Background: Surgical checklists are increasingly used to improve compliance with evidence-based processes in the perioperative period. Although enthusiasm exists for using checklists to improve outcomes, recent studies have questioned their effectiveness in large populations. Objective: We sought to examine the association of Keystone Surgery, a statewide implementation of an evidence-based checklist and Comprehensive Unit-based Safety Program, on surgical outcomes and health care costs. Methods: We performed a study using national Medicare claims data for patients undergoing general and vascular surgery (n = 1,002,241) from 2006 to 2011. A difference-in-differences approach was used to evaluate whether implementation was associated with improved surgical outcomes and decreased costs when compared with a national cohort of nonparticipating hospitals. Propensity score matching was used to select 10 control hospitals for each participating hospital. Costs were assessed using pricestandardized 30-day Medicare payments for acute hospitalizations, readmissions, and high-cost outliers. Results: Keystone Surgery implementation in participating centers (N = 95 hospitals) was not associated with improved outcomes. Difference-in-differences analysis accounting for trends in nonparticipating hospitals (N = 950 hospitals) revealed no differences in adjusted rates of 30-day mortality [relative risk (RR) = 1.03; 95% confidence intervals (CI), 0.97-1.10], any complication (RR = 1.03; 95% CI, 0.99-1.07), reoperations (RR = 0.89; 95% CI, 0.56-1.22), or readmissions (RR = 1.01; 95% CI, 0.97-1.05). Medicare payments for the index admission increased following implementation (516 average increase in payments; 95% CI, 210-823 increase), as did readmission payments (564 increase; 95% CI, 89-1040 increase). High-outlier payments (965 increase; 95% CI, 974decrease to 2904 increase) did not change. Conclusions: Implementation of Keystone Surgery in Michigan was not associated with improved outcomes or decreased costs in Medicare patients.

KW - Administrative Data

KW - Cost Analysis

KW - Effectiveness

KW - Observational Studies

KW - Outcomes Research

KW - Quality Mprovement

KW - Surgery

UR - http://www.scopus.com/inward/record.url?scp=84919341273&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84919341273&partnerID=8YFLogxK

U2 - 10.1097/MLR.0000000000000277

DO - 10.1097/MLR.0000000000000277

M3 - Article

C2 - 25464163

AN - SCOPUS:84919341273

VL - 53

SP - 87

EP - 94

JO - Medical Care

JF - Medical Care

SN - 0025-7079

IS - 1

ER -