Psychometric testing of the pecka grading rubric for evaluating higher-order thinking in distance learning

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

This article describes development of the Pecka Grading Rubric (PGR) as a strategy to facilitate and evaluate students' higher-order thinking in discussion boards. The purpose of this study was to describe psychometric properties of the PGR. Rubric reliability was pilot tested on a discussion board assignment used by 15 senior student registered nurse anesthetist enrolled in an Advanced Principles of Anesthesia course. Interrater and intrarater reliabilities were tested using an interclass correlation coefficient (ICC) to evaluate absolute agreement of scoring. Raters gave each category a score, scores of the categories were summed, and a total score was calculated for the entire rubric. Interrater (ICC = 0.939, P < .001) and intrarater (ICC = 0.902 to 0.994, P < .001) reliabilities were excellent for total point scores. A content validity index was used to evaluate content validity. Raters evaluated content validity of each cell of the PGR. The content validity index (0.8-1.0) was acceptable. Known-group validity was evaluated by comparing graduate student registered nurse anesthetists (N = 7) with undergraduate senior nursing students (N = 13). Beginning evidence indicates a valid and reliable instrument that measures higher-order thinking in the student registered nurse anesthetist.

Original languageEnglish (US)
Pages (from-to)449-456
Number of pages8
JournalAANA Journal
Volume82
Issue number6
StatePublished - Jan 1 2014

Fingerprint

Distance Education
Nurse Anesthetists
Psychometrics
Students
Nurses
Nursing Students
Anesthesia
Thinking

Keywords

  • Distance learning
  • Grading rubric
  • Higherorder thinking
  • Reliability
  • Validity

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Psychometric testing of the pecka grading rubric for evaluating higher-order thinking in distance learning. / Pecka, Shannon; Schmid, Kendra K; Pozehl, Bunny.

In: AANA Journal, Vol. 82, No. 6, 01.01.2014, p. 449-456.

Research output: Contribution to journalArticle

@article{5a51c7249f15412f9de627a9a9b73467,
title = "Psychometric testing of the pecka grading rubric for evaluating higher-order thinking in distance learning",
abstract = "This article describes development of the Pecka Grading Rubric (PGR) as a strategy to facilitate and evaluate students' higher-order thinking in discussion boards. The purpose of this study was to describe psychometric properties of the PGR. Rubric reliability was pilot tested on a discussion board assignment used by 15 senior student registered nurse anesthetist enrolled in an Advanced Principles of Anesthesia course. Interrater and intrarater reliabilities were tested using an interclass correlation coefficient (ICC) to evaluate absolute agreement of scoring. Raters gave each category a score, scores of the categories were summed, and a total score was calculated for the entire rubric. Interrater (ICC = 0.939, P < .001) and intrarater (ICC = 0.902 to 0.994, P < .001) reliabilities were excellent for total point scores. A content validity index was used to evaluate content validity. Raters evaluated content validity of each cell of the PGR. The content validity index (0.8-1.0) was acceptable. Known-group validity was evaluated by comparing graduate student registered nurse anesthetists (N = 7) with undergraduate senior nursing students (N = 13). Beginning evidence indicates a valid and reliable instrument that measures higher-order thinking in the student registered nurse anesthetist.",
keywords = "Distance learning, Grading rubric, Higherorder thinking, Reliability, Validity",
author = "Shannon Pecka and Schmid, {Kendra K} and Bunny Pozehl",
year = "2014",
month = "1",
day = "1",
language = "English (US)",
volume = "82",
pages = "449--456",
journal = "AANA Journal",
issn = "0094-6354",
publisher = "AANA Publishing Inc.",
number = "6",

}

TY - JOUR

T1 - Psychometric testing of the pecka grading rubric for evaluating higher-order thinking in distance learning

AU - Pecka, Shannon

AU - Schmid, Kendra K

AU - Pozehl, Bunny

PY - 2014/1/1

Y1 - 2014/1/1

N2 - This article describes development of the Pecka Grading Rubric (PGR) as a strategy to facilitate and evaluate students' higher-order thinking in discussion boards. The purpose of this study was to describe psychometric properties of the PGR. Rubric reliability was pilot tested on a discussion board assignment used by 15 senior student registered nurse anesthetist enrolled in an Advanced Principles of Anesthesia course. Interrater and intrarater reliabilities were tested using an interclass correlation coefficient (ICC) to evaluate absolute agreement of scoring. Raters gave each category a score, scores of the categories were summed, and a total score was calculated for the entire rubric. Interrater (ICC = 0.939, P < .001) and intrarater (ICC = 0.902 to 0.994, P < .001) reliabilities were excellent for total point scores. A content validity index was used to evaluate content validity. Raters evaluated content validity of each cell of the PGR. The content validity index (0.8-1.0) was acceptable. Known-group validity was evaluated by comparing graduate student registered nurse anesthetists (N = 7) with undergraduate senior nursing students (N = 13). Beginning evidence indicates a valid and reliable instrument that measures higher-order thinking in the student registered nurse anesthetist.

AB - This article describes development of the Pecka Grading Rubric (PGR) as a strategy to facilitate and evaluate students' higher-order thinking in discussion boards. The purpose of this study was to describe psychometric properties of the PGR. Rubric reliability was pilot tested on a discussion board assignment used by 15 senior student registered nurse anesthetist enrolled in an Advanced Principles of Anesthesia course. Interrater and intrarater reliabilities were tested using an interclass correlation coefficient (ICC) to evaluate absolute agreement of scoring. Raters gave each category a score, scores of the categories were summed, and a total score was calculated for the entire rubric. Interrater (ICC = 0.939, P < .001) and intrarater (ICC = 0.902 to 0.994, P < .001) reliabilities were excellent for total point scores. A content validity index was used to evaluate content validity. Raters evaluated content validity of each cell of the PGR. The content validity index (0.8-1.0) was acceptable. Known-group validity was evaluated by comparing graduate student registered nurse anesthetists (N = 7) with undergraduate senior nursing students (N = 13). Beginning evidence indicates a valid and reliable instrument that measures higher-order thinking in the student registered nurse anesthetist.

KW - Distance learning

KW - Grading rubric

KW - Higherorder thinking

KW - Reliability

KW - Validity

UR - http://www.scopus.com/inward/record.url?scp=84914811427&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84914811427&partnerID=8YFLogxK

M3 - Article

VL - 82

SP - 449

EP - 456

JO - AANA Journal

JF - AANA Journal

SN - 0094-6354

IS - 6

ER -