Musculoskeletal ultrasound objective structured clinical examination: An assessment of the test

Eugene Y. Kissin, Peter C. Grayson, Amy C Cannella, Paul J. Demarco, Amy Evangelisto, Janak Goyal, Rany Al Haj, Jay Higgs, Daniel G. Malone, Midori J. Nishio, Darren Tabechian, Gurjit S. Kaeley

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

Objective To determine the reliability and validity of an objective structured clinical examination (OSCE) for musculoskeletal ultrasound (MSUS). Methods A 9-station OSCE was administered to 35 rheumatology fellows trained in MSUS and to 3 expert faculty (controls). Participants were unaware of joint health (5 diseased/4 healthy). Faculty assessors (n = 9) graded image quality with predefined checklists and a 0-5 global rating, blinded to who performed the study. Interrater reliability, correlation between a written multiple choice question examination (MCQ) and OSCE performance, and comparison of fellow OSCE results with those of the faculty were measured to determine OSCE reliability, concurrent validity, and construct validity. Results Assessors' interrater reliability was good (intraclass correlation coefficient [ICC] 0.7). Score reliability was good in the normal wrist and ankle stations (ICC 0.7) and moderate in the abnormal wrist and ankle stations (ICC 0.4). MCQ grades significantly correlated with OSCE grades (r = 0.52, P < 0.01). The fellows in the bottom quartile of the MCQ scored 3.07 on the OSCE, significantly worse than the top quartile fellows (3.32) and the faculty (3.29; P < 0.01). Scores also significantly discriminated bottom quartile fellows from faculty in the normal wrist and ankle stations (3.38 versus 3.78; P < 0.01), but not in the abnormal stations (3.37 versus 3.49; P = 0.08). Conclusion MSUS OSCE is a reliable and valid method for evaluation of MSUS skill. Normal joint assessment stations are more reliable than abnormal joint assessment stations and better discriminate poorly performing fellows from faculty. Therefore, MSUS OSCE with normal joints can be used for the assessment of MSUS skill competency.

Original languageEnglish (US)
Pages (from-to)2-6
Number of pages5
JournalArthritis Care and Research
Volume66
Issue number1
DOIs
StatePublished - Jan 1 2014

Fingerprint

Joints
Wrist
Ankle
Reproducibility of Results
Rheumatology
Checklist
Health

ASJC Scopus subject areas

  • Rheumatology

Cite this

Kissin, E. Y., Grayson, P. C., Cannella, A. C., Demarco, P. J., Evangelisto, A., Goyal, J., ... Kaeley, G. S. (2014). Musculoskeletal ultrasound objective structured clinical examination: An assessment of the test. Arthritis Care and Research, 66(1), 2-6. https://doi.org/10.1002/acr.22105

Musculoskeletal ultrasound objective structured clinical examination : An assessment of the test. / Kissin, Eugene Y.; Grayson, Peter C.; Cannella, Amy C; Demarco, Paul J.; Evangelisto, Amy; Goyal, Janak; Al Haj, Rany; Higgs, Jay; Malone, Daniel G.; Nishio, Midori J.; Tabechian, Darren; Kaeley, Gurjit S.

In: Arthritis Care and Research, Vol. 66, No. 1, 01.01.2014, p. 2-6.

Research output: Contribution to journalArticle

Kissin, EY, Grayson, PC, Cannella, AC, Demarco, PJ, Evangelisto, A, Goyal, J, Al Haj, R, Higgs, J, Malone, DG, Nishio, MJ, Tabechian, D & Kaeley, GS 2014, 'Musculoskeletal ultrasound objective structured clinical examination: An assessment of the test', Arthritis Care and Research, vol. 66, no. 1, pp. 2-6. https://doi.org/10.1002/acr.22105
Kissin, Eugene Y. ; Grayson, Peter C. ; Cannella, Amy C ; Demarco, Paul J. ; Evangelisto, Amy ; Goyal, Janak ; Al Haj, Rany ; Higgs, Jay ; Malone, Daniel G. ; Nishio, Midori J. ; Tabechian, Darren ; Kaeley, Gurjit S. / Musculoskeletal ultrasound objective structured clinical examination : An assessment of the test. In: Arthritis Care and Research. 2014 ; Vol. 66, No. 1. pp. 2-6.
@article{923d53376e3d4d859209f164b39fe3e1,
title = "Musculoskeletal ultrasound objective structured clinical examination: An assessment of the test",
abstract = "Objective To determine the reliability and validity of an objective structured clinical examination (OSCE) for musculoskeletal ultrasound (MSUS). Methods A 9-station OSCE was administered to 35 rheumatology fellows trained in MSUS and to 3 expert faculty (controls). Participants were unaware of joint health (5 diseased/4 healthy). Faculty assessors (n = 9) graded image quality with predefined checklists and a 0-5 global rating, blinded to who performed the study. Interrater reliability, correlation between a written multiple choice question examination (MCQ) and OSCE performance, and comparison of fellow OSCE results with those of the faculty were measured to determine OSCE reliability, concurrent validity, and construct validity. Results Assessors' interrater reliability was good (intraclass correlation coefficient [ICC] 0.7). Score reliability was good in the normal wrist and ankle stations (ICC 0.7) and moderate in the abnormal wrist and ankle stations (ICC 0.4). MCQ grades significantly correlated with OSCE grades (r = 0.52, P < 0.01). The fellows in the bottom quartile of the MCQ scored 3.07 on the OSCE, significantly worse than the top quartile fellows (3.32) and the faculty (3.29; P < 0.01). Scores also significantly discriminated bottom quartile fellows from faculty in the normal wrist and ankle stations (3.38 versus 3.78; P < 0.01), but not in the abnormal stations (3.37 versus 3.49; P = 0.08). Conclusion MSUS OSCE is a reliable and valid method for evaluation of MSUS skill. Normal joint assessment stations are more reliable than abnormal joint assessment stations and better discriminate poorly performing fellows from faculty. Therefore, MSUS OSCE with normal joints can be used for the assessment of MSUS skill competency.",
author = "Kissin, {Eugene Y.} and Grayson, {Peter C.} and Cannella, {Amy C} and Demarco, {Paul J.} and Amy Evangelisto and Janak Goyal and {Al Haj}, Rany and Jay Higgs and Malone, {Daniel G.} and Nishio, {Midori J.} and Darren Tabechian and Kaeley, {Gurjit S.}",
year = "2014",
month = "1",
day = "1",
doi = "10.1002/acr.22105",
language = "English (US)",
volume = "66",
pages = "2--6",
journal = "Arthritis and Rheumatism",
issn = "2151-4658",
publisher = "John Wiley and Sons Inc.",
number = "1",

}

TY - JOUR

T1 - Musculoskeletal ultrasound objective structured clinical examination

T2 - An assessment of the test

AU - Kissin, Eugene Y.

AU - Grayson, Peter C.

AU - Cannella, Amy C

AU - Demarco, Paul J.

AU - Evangelisto, Amy

AU - Goyal, Janak

AU - Al Haj, Rany

AU - Higgs, Jay

AU - Malone, Daniel G.

AU - Nishio, Midori J.

AU - Tabechian, Darren

AU - Kaeley, Gurjit S.

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Objective To determine the reliability and validity of an objective structured clinical examination (OSCE) for musculoskeletal ultrasound (MSUS). Methods A 9-station OSCE was administered to 35 rheumatology fellows trained in MSUS and to 3 expert faculty (controls). Participants were unaware of joint health (5 diseased/4 healthy). Faculty assessors (n = 9) graded image quality with predefined checklists and a 0-5 global rating, blinded to who performed the study. Interrater reliability, correlation between a written multiple choice question examination (MCQ) and OSCE performance, and comparison of fellow OSCE results with those of the faculty were measured to determine OSCE reliability, concurrent validity, and construct validity. Results Assessors' interrater reliability was good (intraclass correlation coefficient [ICC] 0.7). Score reliability was good in the normal wrist and ankle stations (ICC 0.7) and moderate in the abnormal wrist and ankle stations (ICC 0.4). MCQ grades significantly correlated with OSCE grades (r = 0.52, P < 0.01). The fellows in the bottom quartile of the MCQ scored 3.07 on the OSCE, significantly worse than the top quartile fellows (3.32) and the faculty (3.29; P < 0.01). Scores also significantly discriminated bottom quartile fellows from faculty in the normal wrist and ankle stations (3.38 versus 3.78; P < 0.01), but not in the abnormal stations (3.37 versus 3.49; P = 0.08). Conclusion MSUS OSCE is a reliable and valid method for evaluation of MSUS skill. Normal joint assessment stations are more reliable than abnormal joint assessment stations and better discriminate poorly performing fellows from faculty. Therefore, MSUS OSCE with normal joints can be used for the assessment of MSUS skill competency.

AB - Objective To determine the reliability and validity of an objective structured clinical examination (OSCE) for musculoskeletal ultrasound (MSUS). Methods A 9-station OSCE was administered to 35 rheumatology fellows trained in MSUS and to 3 expert faculty (controls). Participants were unaware of joint health (5 diseased/4 healthy). Faculty assessors (n = 9) graded image quality with predefined checklists and a 0-5 global rating, blinded to who performed the study. Interrater reliability, correlation between a written multiple choice question examination (MCQ) and OSCE performance, and comparison of fellow OSCE results with those of the faculty were measured to determine OSCE reliability, concurrent validity, and construct validity. Results Assessors' interrater reliability was good (intraclass correlation coefficient [ICC] 0.7). Score reliability was good in the normal wrist and ankle stations (ICC 0.7) and moderate in the abnormal wrist and ankle stations (ICC 0.4). MCQ grades significantly correlated with OSCE grades (r = 0.52, P < 0.01). The fellows in the bottom quartile of the MCQ scored 3.07 on the OSCE, significantly worse than the top quartile fellows (3.32) and the faculty (3.29; P < 0.01). Scores also significantly discriminated bottom quartile fellows from faculty in the normal wrist and ankle stations (3.38 versus 3.78; P < 0.01), but not in the abnormal stations (3.37 versus 3.49; P = 0.08). Conclusion MSUS OSCE is a reliable and valid method for evaluation of MSUS skill. Normal joint assessment stations are more reliable than abnormal joint assessment stations and better discriminate poorly performing fellows from faculty. Therefore, MSUS OSCE with normal joints can be used for the assessment of MSUS skill competency.

UR - http://www.scopus.com/inward/record.url?scp=84891137147&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84891137147&partnerID=8YFLogxK

U2 - 10.1002/acr.22105

DO - 10.1002/acr.22105

M3 - Article

C2 - 23925978

AN - SCOPUS:84891137147

VL - 66

SP - 2

EP - 6

JO - Arthritis and Rheumatism

JF - Arthritis and Rheumatism

SN - 2151-4658

IS - 1

ER -