Validity and Reliability of Baseline Testing in a Standardized Environment

Kathryn L. Higgins, Todd Caze, Arthur Maerlender

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Objective: The Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) is a computerized neuropsychological test battery commonly used to determine cognitive recovery from concussion based on comparing post-injury scores to baseline scores. This model is based on the premise that ImPACT baseline test scores are a valid and reliable measure of optimal cognitive function at baseline. Growing evidence suggests that this premise may not be accurate and a large contributor to invalid and unreliable baseline test scores may be the protocol and environment in which baseline tests are administered. This study examined the effects of a standardized environment and administration protocol on the reliability and performance validity of athletes' baseline test scores on ImPACT by comparing scores obtained in two different group-testing settings.

Method: Three hundred-sixty one Division 1 cohort-matched collegiate athletes' baseline data were assessed using a variety of indicators of potential performance invalidity; internal reliability was also examined.

Results: Thirty-one to thirty-nine percent of the baseline cases had at least one indicator of low performance validity, but there were no significant differences in validity indicators based on environment in which the testing was conducted. Internal consistency reliability scores were in the acceptable to good range, with no significant differences between administration conditions.

Conclusions: These results suggest that athletes may be reliably performing at levels lower than their best effort would produce.

Original languageEnglish (US)
Pages (from-to)437-443
Number of pages7
JournalArchives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists
Volume33
Issue number4
DOIs
StatePublished - Jun 1 2018

Fingerprint

Reproducibility of Results
Athletes
Neuropsychological Tests
Cognition
Wounds and Injuries

ASJC Scopus subject areas

  • Neuropsychology and Physiological Psychology
  • Clinical Psychology
  • Psychiatry and Mental health

Cite this

@article{4cd20d5c31924e9c84c3681853cfc5d3,
title = "Validity and Reliability of Baseline Testing in a Standardized Environment",
abstract = "Objective: The Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) is a computerized neuropsychological test battery commonly used to determine cognitive recovery from concussion based on comparing post-injury scores to baseline scores. This model is based on the premise that ImPACT baseline test scores are a valid and reliable measure of optimal cognitive function at baseline. Growing evidence suggests that this premise may not be accurate and a large contributor to invalid and unreliable baseline test scores may be the protocol and environment in which baseline tests are administered. This study examined the effects of a standardized environment and administration protocol on the reliability and performance validity of athletes' baseline test scores on ImPACT by comparing scores obtained in two different group-testing settings.Method: Three hundred-sixty one Division 1 cohort-matched collegiate athletes' baseline data were assessed using a variety of indicators of potential performance invalidity; internal reliability was also examined.Results: Thirty-one to thirty-nine percent of the baseline cases had at least one indicator of low performance validity, but there were no significant differences in validity indicators based on environment in which the testing was conducted. Internal consistency reliability scores were in the acceptable to good range, with no significant differences between administration conditions.Conclusions: These results suggest that athletes may be reliably performing at levels lower than their best effort would produce.",
author = "Higgins, {Kathryn L.} and Todd Caze and Arthur Maerlender",
year = "2018",
month = "6",
day = "1",
doi = "10.1093/arclin/acx071",
language = "English (US)",
volume = "33",
pages = "437--443",
journal = "Archives of Clinical Neuropsychology",
issn = "0887-6177",
publisher = "Elsevier BV",
number = "4",

}

TY - JOUR

T1 - Validity and Reliability of Baseline Testing in a Standardized Environment

AU - Higgins, Kathryn L.

AU - Caze, Todd

AU - Maerlender, Arthur

PY - 2018/6/1

Y1 - 2018/6/1

N2 - Objective: The Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) is a computerized neuropsychological test battery commonly used to determine cognitive recovery from concussion based on comparing post-injury scores to baseline scores. This model is based on the premise that ImPACT baseline test scores are a valid and reliable measure of optimal cognitive function at baseline. Growing evidence suggests that this premise may not be accurate and a large contributor to invalid and unreliable baseline test scores may be the protocol and environment in which baseline tests are administered. This study examined the effects of a standardized environment and administration protocol on the reliability and performance validity of athletes' baseline test scores on ImPACT by comparing scores obtained in two different group-testing settings.Method: Three hundred-sixty one Division 1 cohort-matched collegiate athletes' baseline data were assessed using a variety of indicators of potential performance invalidity; internal reliability was also examined.Results: Thirty-one to thirty-nine percent of the baseline cases had at least one indicator of low performance validity, but there were no significant differences in validity indicators based on environment in which the testing was conducted. Internal consistency reliability scores were in the acceptable to good range, with no significant differences between administration conditions.Conclusions: These results suggest that athletes may be reliably performing at levels lower than their best effort would produce.

AB - Objective: The Immediate Postconcussion Assessment and Cognitive Testing (ImPACT) is a computerized neuropsychological test battery commonly used to determine cognitive recovery from concussion based on comparing post-injury scores to baseline scores. This model is based on the premise that ImPACT baseline test scores are a valid and reliable measure of optimal cognitive function at baseline. Growing evidence suggests that this premise may not be accurate and a large contributor to invalid and unreliable baseline test scores may be the protocol and environment in which baseline tests are administered. This study examined the effects of a standardized environment and administration protocol on the reliability and performance validity of athletes' baseline test scores on ImPACT by comparing scores obtained in two different group-testing settings.Method: Three hundred-sixty one Division 1 cohort-matched collegiate athletes' baseline data were assessed using a variety of indicators of potential performance invalidity; internal reliability was also examined.Results: Thirty-one to thirty-nine percent of the baseline cases had at least one indicator of low performance validity, but there were no significant differences in validity indicators based on environment in which the testing was conducted. Internal consistency reliability scores were in the acceptable to good range, with no significant differences between administration conditions.Conclusions: These results suggest that athletes may be reliably performing at levels lower than their best effort would produce.

UR - http://www.scopus.com/inward/record.url?scp=85053732527&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053732527&partnerID=8YFLogxK

U2 - 10.1093/arclin/acx071

DO - 10.1093/arclin/acx071

M3 - Article

C2 - 28961684

AN - SCOPUS:85053732527

VL - 33

SP - 437

EP - 443

JO - Archives of Clinical Neuropsychology

JF - Archives of Clinical Neuropsychology

SN - 0887-6177

IS - 4

ER -