American college of surgeons resident objective structured clinical examination: A national program to assess clinical readiness of entering postgraduate year 1 surgery residents

Ranjan Sudan, Thomas G. Lynch, Donald A. Risucci, Patrice G. Blair, Ajit K. Sachdeva

Research output: Contribution to journalReview article

10 Citations (Scopus)

Abstract

OBJECTIVE:: This report describes the development, initial implementation, and reliability of American College of Surgeons Resident Objective Structured Clinical Examination (ACS OSCE). BACKGROUND:: Variability in clinical knowledge and skills of entering surgery residents has been demonstrated. The ACS OSCE was developed to evaluate and help remediate residents' knowledge and skills in managing patients with life-threatening conditions. METHODS:: A task force of surgeons and professional educators developed 10 standardized clinical case stations, evaluation checklists, and rating scales. Standardized patients (SPs) evaluated each resident's clinical skills (history taking, physical examination, communication, and SP-global scores). Residents completed checklists on diagnosis and management. Coefficient alpha and item-total correlations were used, respectively, to assess internal consistency of metrics and station validity. The resident's overall performance for each station was calculated by combining scores of the individual skills. Analysis of variance compared performance across different institutions. RESULTS:: A total of 103 postgraduate year 1 residents from 7 institutions completed the OSCE. Reliability coefficients of skills ranged from 0.38 for diagnosis to 0.68 for global scores. For overall performance on individual stations, the reliability coefficients ranged from 0.51 to 0.82. Using total percent correct scores from highly reliable stations (α > 0.8), wide variability in resident performance was demonstrated within and between the 7 institutions. CONCLUSIONS:: The ACS OSCE was successfully implemented across diverse institutions. It had moderate reliability and demonstrated variability among entering surgery residents. The ACS OSCE is now available for broader implementation. It should help reduce resident variability and address the requirements of Accreditation Council for Graduate Medical Education for resident supervision.

Original languageEnglish (US)
Pages (from-to)65-71
Number of pages7
JournalAnnals of surgery
Volume260
Issue number1
DOIs
StatePublished - Jul 2014

Fingerprint

Clinical Competence
Checklist
Graduate Medical Education
Accreditation
Advisory Committees
Physical Examination
Analysis of Variance
Communication
Surgeons

Keywords

  • Accreditation Council for Graduate Medical Education
  • OSCE
  • clinical skills
  • patient safety
  • reliability
  • resident training
  • surgery

ASJC Scopus subject areas

  • Surgery

Cite this

American college of surgeons resident objective structured clinical examination : A national program to assess clinical readiness of entering postgraduate year 1 surgery residents. / Sudan, Ranjan; Lynch, Thomas G.; Risucci, Donald A.; Blair, Patrice G.; Sachdeva, Ajit K.

In: Annals of surgery, Vol. 260, No. 1, 07.2014, p. 65-71.

Research output: Contribution to journalReview article

Sudan, Ranjan ; Lynch, Thomas G. ; Risucci, Donald A. ; Blair, Patrice G. ; Sachdeva, Ajit K. / American college of surgeons resident objective structured clinical examination : A national program to assess clinical readiness of entering postgraduate year 1 surgery residents. In: Annals of surgery. 2014 ; Vol. 260, No. 1. pp. 65-71.
@article{edfd55ceb4cc4271acd496657a46fbe2,
title = "American college of surgeons resident objective structured clinical examination: A national program to assess clinical readiness of entering postgraduate year 1 surgery residents",
abstract = "OBJECTIVE:: This report describes the development, initial implementation, and reliability of American College of Surgeons Resident Objective Structured Clinical Examination (ACS OSCE). BACKGROUND:: Variability in clinical knowledge and skills of entering surgery residents has been demonstrated. The ACS OSCE was developed to evaluate and help remediate residents' knowledge and skills in managing patients with life-threatening conditions. METHODS:: A task force of surgeons and professional educators developed 10 standardized clinical case stations, evaluation checklists, and rating scales. Standardized patients (SPs) evaluated each resident's clinical skills (history taking, physical examination, communication, and SP-global scores). Residents completed checklists on diagnosis and management. Coefficient alpha and item-total correlations were used, respectively, to assess internal consistency of metrics and station validity. The resident's overall performance for each station was calculated by combining scores of the individual skills. Analysis of variance compared performance across different institutions. RESULTS:: A total of 103 postgraduate year 1 residents from 7 institutions completed the OSCE. Reliability coefficients of skills ranged from 0.38 for diagnosis to 0.68 for global scores. For overall performance on individual stations, the reliability coefficients ranged from 0.51 to 0.82. Using total percent correct scores from highly reliable stations (α > 0.8), wide variability in resident performance was demonstrated within and between the 7 institutions. CONCLUSIONS:: The ACS OSCE was successfully implemented across diverse institutions. It had moderate reliability and demonstrated variability among entering surgery residents. The ACS OSCE is now available for broader implementation. It should help reduce resident variability and address the requirements of Accreditation Council for Graduate Medical Education for resident supervision.",
keywords = "Accreditation Council for Graduate Medical Education, OSCE, clinical skills, patient safety, reliability, resident training, surgery",
author = "Ranjan Sudan and Lynch, {Thomas G.} and Risucci, {Donald A.} and Blair, {Patrice G.} and Sachdeva, {Ajit K.}",
year = "2014",
month = "7",
doi = "10.1097/SLA.0000000000000325",
language = "English (US)",
volume = "260",
pages = "65--71",
journal = "Annals of Surgery",
issn = "0003-4932",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - American college of surgeons resident objective structured clinical examination

T2 - A national program to assess clinical readiness of entering postgraduate year 1 surgery residents

AU - Sudan, Ranjan

AU - Lynch, Thomas G.

AU - Risucci, Donald A.

AU - Blair, Patrice G.

AU - Sachdeva, Ajit K.

PY - 2014/7

Y1 - 2014/7

N2 - OBJECTIVE:: This report describes the development, initial implementation, and reliability of American College of Surgeons Resident Objective Structured Clinical Examination (ACS OSCE). BACKGROUND:: Variability in clinical knowledge and skills of entering surgery residents has been demonstrated. The ACS OSCE was developed to evaluate and help remediate residents' knowledge and skills in managing patients with life-threatening conditions. METHODS:: A task force of surgeons and professional educators developed 10 standardized clinical case stations, evaluation checklists, and rating scales. Standardized patients (SPs) evaluated each resident's clinical skills (history taking, physical examination, communication, and SP-global scores). Residents completed checklists on diagnosis and management. Coefficient alpha and item-total correlations were used, respectively, to assess internal consistency of metrics and station validity. The resident's overall performance for each station was calculated by combining scores of the individual skills. Analysis of variance compared performance across different institutions. RESULTS:: A total of 103 postgraduate year 1 residents from 7 institutions completed the OSCE. Reliability coefficients of skills ranged from 0.38 for diagnosis to 0.68 for global scores. For overall performance on individual stations, the reliability coefficients ranged from 0.51 to 0.82. Using total percent correct scores from highly reliable stations (α > 0.8), wide variability in resident performance was demonstrated within and between the 7 institutions. CONCLUSIONS:: The ACS OSCE was successfully implemented across diverse institutions. It had moderate reliability and demonstrated variability among entering surgery residents. The ACS OSCE is now available for broader implementation. It should help reduce resident variability and address the requirements of Accreditation Council for Graduate Medical Education for resident supervision.

AB - OBJECTIVE:: This report describes the development, initial implementation, and reliability of American College of Surgeons Resident Objective Structured Clinical Examination (ACS OSCE). BACKGROUND:: Variability in clinical knowledge and skills of entering surgery residents has been demonstrated. The ACS OSCE was developed to evaluate and help remediate residents' knowledge and skills in managing patients with life-threatening conditions. METHODS:: A task force of surgeons and professional educators developed 10 standardized clinical case stations, evaluation checklists, and rating scales. Standardized patients (SPs) evaluated each resident's clinical skills (history taking, physical examination, communication, and SP-global scores). Residents completed checklists on diagnosis and management. Coefficient alpha and item-total correlations were used, respectively, to assess internal consistency of metrics and station validity. The resident's overall performance for each station was calculated by combining scores of the individual skills. Analysis of variance compared performance across different institutions. RESULTS:: A total of 103 postgraduate year 1 residents from 7 institutions completed the OSCE. Reliability coefficients of skills ranged from 0.38 for diagnosis to 0.68 for global scores. For overall performance on individual stations, the reliability coefficients ranged from 0.51 to 0.82. Using total percent correct scores from highly reliable stations (α > 0.8), wide variability in resident performance was demonstrated within and between the 7 institutions. CONCLUSIONS:: The ACS OSCE was successfully implemented across diverse institutions. It had moderate reliability and demonstrated variability among entering surgery residents. The ACS OSCE is now available for broader implementation. It should help reduce resident variability and address the requirements of Accreditation Council for Graduate Medical Education for resident supervision.

KW - Accreditation Council for Graduate Medical Education

KW - OSCE

KW - clinical skills

KW - patient safety

KW - reliability

KW - resident training

KW - surgery

UR - http://www.scopus.com/inward/record.url?scp=84902252158&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84902252158&partnerID=8YFLogxK

U2 - 10.1097/SLA.0000000000000325

DO - 10.1097/SLA.0000000000000325

M3 - Review article

C2 - 24263326

AN - SCOPUS:84902252158

VL - 260

SP - 65

EP - 71

JO - Annals of Surgery

JF - Annals of Surgery

SN - 0003-4932

IS - 1

ER -