A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies

C. Scott Smith, Magdalena Morris, Chris Francovich, Rick Tivis, Roger Bush, Shelley Schoepflin Sanders, Jeremy Graham, Alex Niven, Mari Kai, Christopher Knight, Joseph Hardman, Kelly J Caverzagie, William Iobst

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

PURPOSE: The Accreditation Council for Graduate Medical Education's (ACGME's) six-competency framework has not been validated across multiple stakeholders and sites. The objective of this study was to perform a multisite validation with five stakeholder groups. METHOD: This was a cross-sectional, observational study carried out from October to December, 2011, in the internal medicine residency continuity clinics of eight internal medicine residency programs in the Pacific Northwest, including a VA, two academic medical centers, a military medical center, and four private hospitals. The authors performed a cultural consensus analysis (CCA) and a convergent-discriminant analysis using previously developed statements based on internal medicine milestones related to the six competencies. Ten participants were included from each of five stakeholder groups: patients, nurses, residents, faculty members, and administrators from each training site (total: 400 participants). RESULTS: Moderate to high agreement and coherence for all groups were observed (CCA eigenvalue ratios ranging from 2.16 to 3.20); however, high differences in ranking order were seen between groups in four of the CCA statements, which may suggest between-group tension in these areas. Analyses revealed excellent construct validity (Zcontrast score of 5.323, P < .0001) for the six-competency framework. Average Spearman correlation between same-node statements was 0.012, and between different-node statements it was -0.096. CONCLUSIONS: The ACGME's six-competency framework has reasonable face and construct validity across multiple stakeholders and sites. Stakeholders appear to share a single mental model of competence in this learning environment. Data patterns suggest possible improvements to the competency-milestone framework.

Original languageEnglish (US)
Pages (from-to)997-1001
Number of pages5
JournalAcademic Medicine
Volume88
Issue number7
DOIs
StatePublished - Jan 1 2013

Fingerprint

Graduate Medical Education
Accreditation
Internal Medicine
accreditation
Consensus
stakeholder
graduate
Internship and Residency
Northwestern United States
medicine
construct validity
education
Private Hospitals
Group
Discriminant Analysis
Administrative Personnel
Reproducibility of Results
Mental Competency
Observational Studies
Cross-Sectional Studies

ASJC Scopus subject areas

  • Education

Cite this

Smith, C. S., Morris, M., Francovich, C., Tivis, R., Bush, R., Sanders, S. S., ... Iobst, W. (2013). A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies. Academic Medicine, 88(7), 997-1001. https://doi.org/10.1097/ACM.0b013e3182951efc

A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies. / Smith, C. Scott; Morris, Magdalena; Francovich, Chris; Tivis, Rick; Bush, Roger; Sanders, Shelley Schoepflin; Graham, Jeremy; Niven, Alex; Kai, Mari; Knight, Christopher; Hardman, Joseph; Caverzagie, Kelly J; Iobst, William.

In: Academic Medicine, Vol. 88, No. 7, 01.01.2013, p. 997-1001.

Research output: Contribution to journalArticle

Smith, CS, Morris, M, Francovich, C, Tivis, R, Bush, R, Sanders, SS, Graham, J, Niven, A, Kai, M, Knight, C, Hardman, J, Caverzagie, KJ & Iobst, W 2013, 'A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies', Academic Medicine, vol. 88, no. 7, pp. 997-1001. https://doi.org/10.1097/ACM.0b013e3182951efc
Smith, C. Scott ; Morris, Magdalena ; Francovich, Chris ; Tivis, Rick ; Bush, Roger ; Sanders, Shelley Schoepflin ; Graham, Jeremy ; Niven, Alex ; Kai, Mari ; Knight, Christopher ; Hardman, Joseph ; Caverzagie, Kelly J ; Iobst, William. / A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies. In: Academic Medicine. 2013 ; Vol. 88, No. 7. pp. 997-1001.
@article{b42f110133e94ed4a706f61adb9a45b7,
title = "A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies",
abstract = "PURPOSE: The Accreditation Council for Graduate Medical Education's (ACGME's) six-competency framework has not been validated across multiple stakeholders and sites. The objective of this study was to perform a multisite validation with five stakeholder groups. METHOD: This was a cross-sectional, observational study carried out from October to December, 2011, in the internal medicine residency continuity clinics of eight internal medicine residency programs in the Pacific Northwest, including a VA, two academic medical centers, a military medical center, and four private hospitals. The authors performed a cultural consensus analysis (CCA) and a convergent-discriminant analysis using previously developed statements based on internal medicine milestones related to the six competencies. Ten participants were included from each of five stakeholder groups: patients, nurses, residents, faculty members, and administrators from each training site (total: 400 participants). RESULTS: Moderate to high agreement and coherence for all groups were observed (CCA eigenvalue ratios ranging from 2.16 to 3.20); however, high differences in ranking order were seen between groups in four of the CCA statements, which may suggest between-group tension in these areas. Analyses revealed excellent construct validity (Zcontrast score of 5.323, P < .0001) for the six-competency framework. Average Spearman correlation between same-node statements was 0.012, and between different-node statements it was -0.096. CONCLUSIONS: The ACGME's six-competency framework has reasonable face and construct validity across multiple stakeholders and sites. Stakeholders appear to share a single mental model of competence in this learning environment. Data patterns suggest possible improvements to the competency-milestone framework.",
author = "Smith, {C. Scott} and Magdalena Morris and Chris Francovich and Rick Tivis and Roger Bush and Sanders, {Shelley Schoepflin} and Jeremy Graham and Alex Niven and Mari Kai and Christopher Knight and Joseph Hardman and Caverzagie, {Kelly J} and William Iobst",
year = "2013",
month = "1",
day = "1",
doi = "10.1097/ACM.0b013e3182951efc",
language = "English (US)",
volume = "88",
pages = "997--1001",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "7",

}

TY - JOUR

T1 - A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies

AU - Smith, C. Scott

AU - Morris, Magdalena

AU - Francovich, Chris

AU - Tivis, Rick

AU - Bush, Roger

AU - Sanders, Shelley Schoepflin

AU - Graham, Jeremy

AU - Niven, Alex

AU - Kai, Mari

AU - Knight, Christopher

AU - Hardman, Joseph

AU - Caverzagie, Kelly J

AU - Iobst, William

PY - 2013/1/1

Y1 - 2013/1/1

N2 - PURPOSE: The Accreditation Council for Graduate Medical Education's (ACGME's) six-competency framework has not been validated across multiple stakeholders and sites. The objective of this study was to perform a multisite validation with five stakeholder groups. METHOD: This was a cross-sectional, observational study carried out from October to December, 2011, in the internal medicine residency continuity clinics of eight internal medicine residency programs in the Pacific Northwest, including a VA, two academic medical centers, a military medical center, and four private hospitals. The authors performed a cultural consensus analysis (CCA) and a convergent-discriminant analysis using previously developed statements based on internal medicine milestones related to the six competencies. Ten participants were included from each of five stakeholder groups: patients, nurses, residents, faculty members, and administrators from each training site (total: 400 participants). RESULTS: Moderate to high agreement and coherence for all groups were observed (CCA eigenvalue ratios ranging from 2.16 to 3.20); however, high differences in ranking order were seen between groups in four of the CCA statements, which may suggest between-group tension in these areas. Analyses revealed excellent construct validity (Zcontrast score of 5.323, P < .0001) for the six-competency framework. Average Spearman correlation between same-node statements was 0.012, and between different-node statements it was -0.096. CONCLUSIONS: The ACGME's six-competency framework has reasonable face and construct validity across multiple stakeholders and sites. Stakeholders appear to share a single mental model of competence in this learning environment. Data patterns suggest possible improvements to the competency-milestone framework.

AB - PURPOSE: The Accreditation Council for Graduate Medical Education's (ACGME's) six-competency framework has not been validated across multiple stakeholders and sites. The objective of this study was to perform a multisite validation with five stakeholder groups. METHOD: This was a cross-sectional, observational study carried out from October to December, 2011, in the internal medicine residency continuity clinics of eight internal medicine residency programs in the Pacific Northwest, including a VA, two academic medical centers, a military medical center, and four private hospitals. The authors performed a cultural consensus analysis (CCA) and a convergent-discriminant analysis using previously developed statements based on internal medicine milestones related to the six competencies. Ten participants were included from each of five stakeholder groups: patients, nurses, residents, faculty members, and administrators from each training site (total: 400 participants). RESULTS: Moderate to high agreement and coherence for all groups were observed (CCA eigenvalue ratios ranging from 2.16 to 3.20); however, high differences in ranking order were seen between groups in four of the CCA statements, which may suggest between-group tension in these areas. Analyses revealed excellent construct validity (Zcontrast score of 5.323, P < .0001) for the six-competency framework. Average Spearman correlation between same-node statements was 0.012, and between different-node statements it was -0.096. CONCLUSIONS: The ACGME's six-competency framework has reasonable face and construct validity across multiple stakeholders and sites. Stakeholders appear to share a single mental model of competence in this learning environment. Data patterns suggest possible improvements to the competency-milestone framework.

UR - http://www.scopus.com/inward/record.url?scp=84879985356&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84879985356&partnerID=8YFLogxK

U2 - 10.1097/ACM.0b013e3182951efc

DO - 10.1097/ACM.0b013e3182951efc

M3 - Article

VL - 88

SP - 997

EP - 1001

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 7

ER -