Item selection counts

A comparison of empirical key and rational scale validities in theory-based and non-theory-based item pools

Roni Reiter-Palmon, Mary Shane Connelly

Research output: Contribution to journalReview article

12 Citations (Scopus)

Abstract

Little explicit attention has been given to the impact of item pools on the validities and cross-validities of different background data scoring approaches. This study tests the idea that pools c f items theoretically related to the performance of interest will outperform pools of items with no hypothesized relationship with the criterion. Validities and cross-validities of rational scales and empirical keys created from theory- and non-theory-based item pools were compared for 3 criteria. When size of the item pools was held constant, theory-based empirical keys (correlational and vertical percent) and rational scales showed larger validities and cross-validities than non-theory-based empirical keys (correlational and vertical percent) and showed minimal shrinkage in cross-validities. Even when item pool for the non-theory-based keys was expanded to include all items in the instrument, the theory-based keys showed comparable or slightly better validities and cross-validities for 2 of the 3 criteria, including college GPA, which was separated from the predictors by 4 years.

Original languageEnglish (US)
Pages (from-to)143-151
Number of pages9
JournalJournal of Applied Psychology
Volume85
Issue number1
DOIs
StatePublished - Jan 1 2000

ASJC Scopus subject areas

  • Applied Psychology

Cite this

Item selection counts : A comparison of empirical key and rational scale validities in theory-based and non-theory-based item pools. / Reiter-Palmon, Roni; Connelly, Mary Shane.

In: Journal of Applied Psychology, Vol. 85, No. 1, 01.01.2000, p. 143-151.

Research output: Contribution to journalReview article

@article{f4446ff4c7484dcbba9d6fb5d9461a55,
title = "Item selection counts: A comparison of empirical key and rational scale validities in theory-based and non-theory-based item pools",
abstract = "Little explicit attention has been given to the impact of item pools on the validities and cross-validities of different background data scoring approaches. This study tests the idea that pools c f items theoretically related to the performance of interest will outperform pools of items with no hypothesized relationship with the criterion. Validities and cross-validities of rational scales and empirical keys created from theory- and non-theory-based item pools were compared for 3 criteria. When size of the item pools was held constant, theory-based empirical keys (correlational and vertical percent) and rational scales showed larger validities and cross-validities than non-theory-based empirical keys (correlational and vertical percent) and showed minimal shrinkage in cross-validities. Even when item pool for the non-theory-based keys was expanded to include all items in the instrument, the theory-based keys showed comparable or slightly better validities and cross-validities for 2 of the 3 criteria, including college GPA, which was separated from the predictors by 4 years.",
author = "Roni Reiter-Palmon and Connelly, {Mary Shane}",
year = "2000",
month = "1",
day = "1",
doi = "10.1037/0021-9010.85.1.143",
language = "English (US)",
volume = "85",
pages = "143--151",
journal = "Journal of Applied Psychology",
issn = "0021-9010",
publisher = "American Psychological Association Inc.",
number = "1",

}

TY - JOUR

T1 - Item selection counts

T2 - A comparison of empirical key and rational scale validities in theory-based and non-theory-based item pools

AU - Reiter-Palmon, Roni

AU - Connelly, Mary Shane

PY - 2000/1/1

Y1 - 2000/1/1

N2 - Little explicit attention has been given to the impact of item pools on the validities and cross-validities of different background data scoring approaches. This study tests the idea that pools c f items theoretically related to the performance of interest will outperform pools of items with no hypothesized relationship with the criterion. Validities and cross-validities of rational scales and empirical keys created from theory- and non-theory-based item pools were compared for 3 criteria. When size of the item pools was held constant, theory-based empirical keys (correlational and vertical percent) and rational scales showed larger validities and cross-validities than non-theory-based empirical keys (correlational and vertical percent) and showed minimal shrinkage in cross-validities. Even when item pool for the non-theory-based keys was expanded to include all items in the instrument, the theory-based keys showed comparable or slightly better validities and cross-validities for 2 of the 3 criteria, including college GPA, which was separated from the predictors by 4 years.

AB - Little explicit attention has been given to the impact of item pools on the validities and cross-validities of different background data scoring approaches. This study tests the idea that pools c f items theoretically related to the performance of interest will outperform pools of items with no hypothesized relationship with the criterion. Validities and cross-validities of rational scales and empirical keys created from theory- and non-theory-based item pools were compared for 3 criteria. When size of the item pools was held constant, theory-based empirical keys (correlational and vertical percent) and rational scales showed larger validities and cross-validities than non-theory-based empirical keys (correlational and vertical percent) and showed minimal shrinkage in cross-validities. Even when item pool for the non-theory-based keys was expanded to include all items in the instrument, the theory-based keys showed comparable or slightly better validities and cross-validities for 2 of the 3 criteria, including college GPA, which was separated from the predictors by 4 years.

UR - http://www.scopus.com/inward/record.url?scp=0034131552&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034131552&partnerID=8YFLogxK

U2 - 10.1037/0021-9010.85.1.143

DO - 10.1037/0021-9010.85.1.143

M3 - Review article

VL - 85

SP - 143

EP - 151

JO - Journal of Applied Psychology

JF - Journal of Applied Psychology

SN - 0021-9010

IS - 1

ER -