Using proxy measures and other correlates of survey outcomes to adjust for non-response: Examples from multiple surveys

F. Kreuter, K. Olson, J. Wagner, T. Yan, T. M. Ezzati-Rice, C. Casas-Cordero, M. Lemay, A. Peytchev, R. M. Groves, T. E. Raghunathan

Research output: Contribution to journalArticle

84 Citations (Scopus)

Abstract

Non-response weighting is a commonly used method to adjust for bias due to unit non-response in surveys. Theory and simulations show that, to reduce bias effectively without increasing variance, a covariate that is used for non-response weighting adjustment needs to be highly associated with both the response indicator and the survey outcome variable. In practice, these requirements pose a challenge that is often overlooked, because those covariates are often not observed or may not exist. Surveys have recently begun to collect supplementary data, such as interviewer observations and other proxy measures of key survey outcome variables. To the extent that these auxiliary variables are highly correlated with the actual outcomes, these variables are promising candidates for non-response adjustment. In the present study, we examine traditional covariates and new auxiliary variables for the National Survey of Family Growth, the Medical Expenditure Panel Survey, the American National Election Survey, the European Social Surveys and the University of Michigan Transportation Research Institute survey. We provide empirical estimates of the association between proxy measures and response to the survey request as well as the actual survey outcome variables. We also compare unweighted and weighted estimates under various non-response models. Our results from multiple surveys with multiple recruitment protocols from multiple organizations on multiple topics show the difficulty of finding suitable covariates for non-response adjustment and the need to improve the quality of auxiliary data.

Original languageEnglish (US)
Pages (from-to)389-407
Number of pages19
JournalJournal of the Royal Statistical Society. Series A: Statistics in Society
Volume173
Issue number2
DOIs
StatePublished - Apr 1 2010

Fingerprint

Non-response
response behavior
Correlate
Covariates
Adjustment
Auxiliary Variables
weighting
Weighting
Correlates
ESS
Weighted Estimates
trend
research facility
Elections
expenditures
candidacy
election

Keywords

  • Interviewer observations
  • Non-response adjustment
  • Non-response bias
  • Paradata
  • Response propensity weights

ASJC Scopus subject areas

  • Statistics and Probability
  • Social Sciences (miscellaneous)
  • Economics and Econometrics
  • Statistics, Probability and Uncertainty

Cite this

Using proxy measures and other correlates of survey outcomes to adjust for non-response : Examples from multiple surveys. / Kreuter, F.; Olson, K.; Wagner, J.; Yan, T.; Ezzati-Rice, T. M.; Casas-Cordero, C.; Lemay, M.; Peytchev, A.; Groves, R. M.; Raghunathan, T. E.

In: Journal of the Royal Statistical Society. Series A: Statistics in Society, Vol. 173, No. 2, 01.04.2010, p. 389-407.

Research output: Contribution to journalArticle

Kreuter, F. ; Olson, K. ; Wagner, J. ; Yan, T. ; Ezzati-Rice, T. M. ; Casas-Cordero, C. ; Lemay, M. ; Peytchev, A. ; Groves, R. M. ; Raghunathan, T. E. / Using proxy measures and other correlates of survey outcomes to adjust for non-response : Examples from multiple surveys. In: Journal of the Royal Statistical Society. Series A: Statistics in Society. 2010 ; Vol. 173, No. 2. pp. 389-407.
@article{557c008697dd440e8fca069b60fcaf95,
title = "Using proxy measures and other correlates of survey outcomes to adjust for non-response: Examples from multiple surveys",
abstract = "Non-response weighting is a commonly used method to adjust for bias due to unit non-response in surveys. Theory and simulations show that, to reduce bias effectively without increasing variance, a covariate that is used for non-response weighting adjustment needs to be highly associated with both the response indicator and the survey outcome variable. In practice, these requirements pose a challenge that is often overlooked, because those covariates are often not observed or may not exist. Surveys have recently begun to collect supplementary data, such as interviewer observations and other proxy measures of key survey outcome variables. To the extent that these auxiliary variables are highly correlated with the actual outcomes, these variables are promising candidates for non-response adjustment. In the present study, we examine traditional covariates and new auxiliary variables for the National Survey of Family Growth, the Medical Expenditure Panel Survey, the American National Election Survey, the European Social Surveys and the University of Michigan Transportation Research Institute survey. We provide empirical estimates of the association between proxy measures and response to the survey request as well as the actual survey outcome variables. We also compare unweighted and weighted estimates under various non-response models. Our results from multiple surveys with multiple recruitment protocols from multiple organizations on multiple topics show the difficulty of finding suitable covariates for non-response adjustment and the need to improve the quality of auxiliary data.",
keywords = "Interviewer observations, Non-response adjustment, Non-response bias, Paradata, Response propensity weights",
author = "F. Kreuter and K. Olson and J. Wagner and T. Yan and Ezzati-Rice, {T. M.} and C. Casas-Cordero and M. Lemay and A. Peytchev and Groves, {R. M.} and Raghunathan, {T. E.}",
year = "2010",
month = "4",
day = "1",
doi = "10.1111/j.1467-985X.2009.00621.x",
language = "English (US)",
volume = "173",
pages = "389--407",
journal = "Journal of the Royal Statistical Society. Series A: Statistics in Society",
issn = "0964-1998",
publisher = "Wiley-Blackwell",
number = "2",

}

TY - JOUR

T1 - Using proxy measures and other correlates of survey outcomes to adjust for non-response

T2 - Examples from multiple surveys

AU - Kreuter, F.

AU - Olson, K.

AU - Wagner, J.

AU - Yan, T.

AU - Ezzati-Rice, T. M.

AU - Casas-Cordero, C.

AU - Lemay, M.

AU - Peytchev, A.

AU - Groves, R. M.

AU - Raghunathan, T. E.

PY - 2010/4/1

Y1 - 2010/4/1

N2 - Non-response weighting is a commonly used method to adjust for bias due to unit non-response in surveys. Theory and simulations show that, to reduce bias effectively without increasing variance, a covariate that is used for non-response weighting adjustment needs to be highly associated with both the response indicator and the survey outcome variable. In practice, these requirements pose a challenge that is often overlooked, because those covariates are often not observed or may not exist. Surveys have recently begun to collect supplementary data, such as interviewer observations and other proxy measures of key survey outcome variables. To the extent that these auxiliary variables are highly correlated with the actual outcomes, these variables are promising candidates for non-response adjustment. In the present study, we examine traditional covariates and new auxiliary variables for the National Survey of Family Growth, the Medical Expenditure Panel Survey, the American National Election Survey, the European Social Surveys and the University of Michigan Transportation Research Institute survey. We provide empirical estimates of the association between proxy measures and response to the survey request as well as the actual survey outcome variables. We also compare unweighted and weighted estimates under various non-response models. Our results from multiple surveys with multiple recruitment protocols from multiple organizations on multiple topics show the difficulty of finding suitable covariates for non-response adjustment and the need to improve the quality of auxiliary data.

AB - Non-response weighting is a commonly used method to adjust for bias due to unit non-response in surveys. Theory and simulations show that, to reduce bias effectively without increasing variance, a covariate that is used for non-response weighting adjustment needs to be highly associated with both the response indicator and the survey outcome variable. In practice, these requirements pose a challenge that is often overlooked, because those covariates are often not observed or may not exist. Surveys have recently begun to collect supplementary data, such as interviewer observations and other proxy measures of key survey outcome variables. To the extent that these auxiliary variables are highly correlated with the actual outcomes, these variables are promising candidates for non-response adjustment. In the present study, we examine traditional covariates and new auxiliary variables for the National Survey of Family Growth, the Medical Expenditure Panel Survey, the American National Election Survey, the European Social Surveys and the University of Michigan Transportation Research Institute survey. We provide empirical estimates of the association between proxy measures and response to the survey request as well as the actual survey outcome variables. We also compare unweighted and weighted estimates under various non-response models. Our results from multiple surveys with multiple recruitment protocols from multiple organizations on multiple topics show the difficulty of finding suitable covariates for non-response adjustment and the need to improve the quality of auxiliary data.

KW - Interviewer observations

KW - Non-response adjustment

KW - Non-response bias

KW - Paradata

KW - Response propensity weights

UR - http://www.scopus.com/inward/record.url?scp=77950949283&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77950949283&partnerID=8YFLogxK

U2 - 10.1111/j.1467-985X.2009.00621.x

DO - 10.1111/j.1467-985X.2009.00621.x

M3 - Article

AN - SCOPUS:77950949283

VL - 173

SP - 389

EP - 407

JO - Journal of the Royal Statistical Society. Series A: Statistics in Society

JF - Journal of the Royal Statistical Society. Series A: Statistics in Society

SN - 0964-1998

IS - 2

ER -