Using elements from an acute abdominal pain Objective Structured Clinical Examination (OSCE) leads to more standardized grading in the surgical clerkship for third-year medical students

John L. Falcone, Kimberly D Schenarts, Peter F. Ferson, Hollis D. Day

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Background: There is poor reliability in the Likert-based assessments of patient interaction and general knowledge base for medical students in the surgical clerkship. The Objective Structured Clinical Examination (OSCE) can be used to assess these competencies. Objective: We hypothesize that using OSCE performance to replace the current Likert-based patient interaction and general knowledge base assessments will not affect the pass/fail rate for third-year medical students in the surgical clerkship. Methods: In this retrospective study, third-year medical student clerkship data from a three-station acute abdominal pain OSCE were collected from the 20092010 academic year. New patient interaction and general knowledge base assessments were derived from the performance data and substituted for original assessments to generate new clerkship scores and ordinal grades. Two-sided nonparametric statistics were used for comparative analyses, using an α = 0.05. Results: Seventy third-year medical students (50.0% female) were evaluated. A sign test showed a difference in the original (4.45/5) and the new (4.20/5) median patient interaction scores (p < 0.01). A sign test did not show a difference in the original (4.00/5) and the new (4.11/5) median general knowledge base scores (p = 0.28). Nine clerkship grades changed between these different grading schemes (p = 0.045), with an overall agreement of 87.1% and a kappa statistic of 0.81. There were no differences in the pass/fail rate (p > 0.99). Conclusions: We conclude that there are no differences in pass/fail rate, but there is a more standardized distribution of patient interaction assessments and utilization of the full spectrum of possible passing grades. We recommend that the current patient interaction assessment for third-year medical students in the surgical clerkship be replaced with that found through trained standardized patients in this three-station acute abdominal pain OSCE.

Original languageEnglish (US)
Pages (from-to)408-413
Number of pages6
JournalJournal of Surgical Education
Volume68
Issue number5
DOIs
StatePublished - Sep 1 2011

Fingerprint

Acute Pain
grading
Medical Students
Abdominal Pain
medical student
pain
examination
assessment basis
interaction
Knowledge Bases
performance
Nonparametric Statistics
utilization
statistics
Retrospective Studies

Keywords

  • acute abdomen
  • clinical clerkship
  • educational assessment
  • educational measurement
  • general surgery
  • medical students

ASJC Scopus subject areas

  • Surgery
  • Education

Cite this

@article{c63aa7d559eb42e593b00f0c581ef25f,
title = "Using elements from an acute abdominal pain Objective Structured Clinical Examination (OSCE) leads to more standardized grading in the surgical clerkship for third-year medical students",
abstract = "Background: There is poor reliability in the Likert-based assessments of patient interaction and general knowledge base for medical students in the surgical clerkship. The Objective Structured Clinical Examination (OSCE) can be used to assess these competencies. Objective: We hypothesize that using OSCE performance to replace the current Likert-based patient interaction and general knowledge base assessments will not affect the pass/fail rate for third-year medical students in the surgical clerkship. Methods: In this retrospective study, third-year medical student clerkship data from a three-station acute abdominal pain OSCE were collected from the 20092010 academic year. New patient interaction and general knowledge base assessments were derived from the performance data and substituted for original assessments to generate new clerkship scores and ordinal grades. Two-sided nonparametric statistics were used for comparative analyses, using an α = 0.05. Results: Seventy third-year medical students (50.0{\%} female) were evaluated. A sign test showed a difference in the original (4.45/5) and the new (4.20/5) median patient interaction scores (p < 0.01). A sign test did not show a difference in the original (4.00/5) and the new (4.11/5) median general knowledge base scores (p = 0.28). Nine clerkship grades changed between these different grading schemes (p = 0.045), with an overall agreement of 87.1{\%} and a kappa statistic of 0.81. There were no differences in the pass/fail rate (p > 0.99). Conclusions: We conclude that there are no differences in pass/fail rate, but there is a more standardized distribution of patient interaction assessments and utilization of the full spectrum of possible passing grades. We recommend that the current patient interaction assessment for third-year medical students in the surgical clerkship be replaced with that found through trained standardized patients in this three-station acute abdominal pain OSCE.",
keywords = "acute abdomen, clinical clerkship, educational assessment, educational measurement, general surgery, medical students",
author = "Falcone, {John L.} and Schenarts, {Kimberly D} and Ferson, {Peter F.} and Day, {Hollis D.}",
year = "2011",
month = "9",
day = "1",
doi = "10.1016/j.jsurg.2011.05.008",
language = "English (US)",
volume = "68",
pages = "408--413",
journal = "Journal of Surgical Education",
issn = "1931-7204",
publisher = "Elsevier Inc.",
number = "5",

}

TY - JOUR

T1 - Using elements from an acute abdominal pain Objective Structured Clinical Examination (OSCE) leads to more standardized grading in the surgical clerkship for third-year medical students

AU - Falcone, John L.

AU - Schenarts, Kimberly D

AU - Ferson, Peter F.

AU - Day, Hollis D.

PY - 2011/9/1

Y1 - 2011/9/1

N2 - Background: There is poor reliability in the Likert-based assessments of patient interaction and general knowledge base for medical students in the surgical clerkship. The Objective Structured Clinical Examination (OSCE) can be used to assess these competencies. Objective: We hypothesize that using OSCE performance to replace the current Likert-based patient interaction and general knowledge base assessments will not affect the pass/fail rate for third-year medical students in the surgical clerkship. Methods: In this retrospective study, third-year medical student clerkship data from a three-station acute abdominal pain OSCE were collected from the 20092010 academic year. New patient interaction and general knowledge base assessments were derived from the performance data and substituted for original assessments to generate new clerkship scores and ordinal grades. Two-sided nonparametric statistics were used for comparative analyses, using an α = 0.05. Results: Seventy third-year medical students (50.0% female) were evaluated. A sign test showed a difference in the original (4.45/5) and the new (4.20/5) median patient interaction scores (p < 0.01). A sign test did not show a difference in the original (4.00/5) and the new (4.11/5) median general knowledge base scores (p = 0.28). Nine clerkship grades changed between these different grading schemes (p = 0.045), with an overall agreement of 87.1% and a kappa statistic of 0.81. There were no differences in the pass/fail rate (p > 0.99). Conclusions: We conclude that there are no differences in pass/fail rate, but there is a more standardized distribution of patient interaction assessments and utilization of the full spectrum of possible passing grades. We recommend that the current patient interaction assessment for third-year medical students in the surgical clerkship be replaced with that found through trained standardized patients in this three-station acute abdominal pain OSCE.

AB - Background: There is poor reliability in the Likert-based assessments of patient interaction and general knowledge base for medical students in the surgical clerkship. The Objective Structured Clinical Examination (OSCE) can be used to assess these competencies. Objective: We hypothesize that using OSCE performance to replace the current Likert-based patient interaction and general knowledge base assessments will not affect the pass/fail rate for third-year medical students in the surgical clerkship. Methods: In this retrospective study, third-year medical student clerkship data from a three-station acute abdominal pain OSCE were collected from the 20092010 academic year. New patient interaction and general knowledge base assessments were derived from the performance data and substituted for original assessments to generate new clerkship scores and ordinal grades. Two-sided nonparametric statistics were used for comparative analyses, using an α = 0.05. Results: Seventy third-year medical students (50.0% female) were evaluated. A sign test showed a difference in the original (4.45/5) and the new (4.20/5) median patient interaction scores (p < 0.01). A sign test did not show a difference in the original (4.00/5) and the new (4.11/5) median general knowledge base scores (p = 0.28). Nine clerkship grades changed between these different grading schemes (p = 0.045), with an overall agreement of 87.1% and a kappa statistic of 0.81. There were no differences in the pass/fail rate (p > 0.99). Conclusions: We conclude that there are no differences in pass/fail rate, but there is a more standardized distribution of patient interaction assessments and utilization of the full spectrum of possible passing grades. We recommend that the current patient interaction assessment for third-year medical students in the surgical clerkship be replaced with that found through trained standardized patients in this three-station acute abdominal pain OSCE.

KW - acute abdomen

KW - clinical clerkship

KW - educational assessment

KW - educational measurement

KW - general surgery

KW - medical students

UR - http://www.scopus.com/inward/record.url?scp=80051542646&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80051542646&partnerID=8YFLogxK

U2 - 10.1016/j.jsurg.2011.05.008

DO - 10.1016/j.jsurg.2011.05.008

M3 - Article

VL - 68

SP - 408

EP - 413

JO - Journal of Surgical Education

JF - Journal of Surgical Education

SN - 1931-7204

IS - 5

ER -