Within-Household Selection in Mail Surveys: Explicit questions are better than cover letter instructions

Research output: Contribution to journalArticle

Abstract

Randomly selecting a single adult within a household is one of the biggest challenges facing mail surveys. Yet obtaining a probability sample of adults within households is critical to having a probability sample of the US adult population. In this paper, we experimentally test three alternative placements of the within-household selection instructions in the National Health, Wellbeing, and Perspectives study (sample n = 6,000; respondent n = 998): (1) a standard cover letter informing the household to ask the person with the next birthday to complete the survey (control); (2) the control cover letter plus an instruction on the front cover of the questionnaire itself to have the adult with the next birthday complete the survey; and (3) the control cover letter plus an explicit yes/ no question asking whether the individual is the adult in the household who will have the next birthday. Although the version with an explicit question had a two-point decrease in response rates relative to not having any instruction, the explicit question significantly improves selection accuracy relative to the other two designs, yields a sample composition closer to national benchmarks, and does not affect item nonresponse rates. Accurately selected respondents also differ from inaccurately selected respondents on questions related to household tasks. Survey practitioners are encouraged to use active tasks such as explicit questions rather than passive tasks such as embedded instructions as part of the within-household selection process.

Original languageEnglish (US)
Pages (from-to)688-713
Number of pages26
JournalPublic Opinion Quarterly
Volume81
Issue number3
DOIs
StatePublished - Jan 1 2017

Fingerprint

mail survey
instruction
response behavior
Health
Household
Letters
Chemical analysis
human being
questionnaire
health
Birthday

ASJC Scopus subject areas

  • Communication
  • History
  • Sociology and Political Science
  • Social Sciences(all)
  • History and Philosophy of Science

Cite this

@article{6d2044eba16840e394b88c6f97d5dbd9,
title = "Within-Household Selection in Mail Surveys: Explicit questions are better than cover letter instructions",
abstract = "Randomly selecting a single adult within a household is one of the biggest challenges facing mail surveys. Yet obtaining a probability sample of adults within households is critical to having a probability sample of the US adult population. In this paper, we experimentally test three alternative placements of the within-household selection instructions in the National Health, Wellbeing, and Perspectives study (sample n = 6,000; respondent n = 998): (1) a standard cover letter informing the household to ask the person with the next birthday to complete the survey (control); (2) the control cover letter plus an instruction on the front cover of the questionnaire itself to have the adult with the next birthday complete the survey; and (3) the control cover letter plus an explicit yes/ no question asking whether the individual is the adult in the household who will have the next birthday. Although the version with an explicit question had a two-point decrease in response rates relative to not having any instruction, the explicit question significantly improves selection accuracy relative to the other two designs, yields a sample composition closer to national benchmarks, and does not affect item nonresponse rates. Accurately selected respondents also differ from inaccurately selected respondents on questions related to household tasks. Survey practitioners are encouraged to use active tasks such as explicit questions rather than passive tasks such as embedded instructions as part of the within-household selection process.",
author = "Kristen Olson and Smyth, {Jolene D.}",
year = "2017",
month = "1",
day = "1",
doi = "10.1093/poq/nfx025",
language = "English (US)",
volume = "81",
pages = "688--713",
journal = "Public Opinion Quarterly",
issn = "0033-362X",
publisher = "Oxford University Press",
number = "3",

}

TY - JOUR

T1 - Within-Household Selection in Mail Surveys

T2 - Explicit questions are better than cover letter instructions

AU - Olson, Kristen

AU - Smyth, Jolene D.

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Randomly selecting a single adult within a household is one of the biggest challenges facing mail surveys. Yet obtaining a probability sample of adults within households is critical to having a probability sample of the US adult population. In this paper, we experimentally test three alternative placements of the within-household selection instructions in the National Health, Wellbeing, and Perspectives study (sample n = 6,000; respondent n = 998): (1) a standard cover letter informing the household to ask the person with the next birthday to complete the survey (control); (2) the control cover letter plus an instruction on the front cover of the questionnaire itself to have the adult with the next birthday complete the survey; and (3) the control cover letter plus an explicit yes/ no question asking whether the individual is the adult in the household who will have the next birthday. Although the version with an explicit question had a two-point decrease in response rates relative to not having any instruction, the explicit question significantly improves selection accuracy relative to the other two designs, yields a sample composition closer to national benchmarks, and does not affect item nonresponse rates. Accurately selected respondents also differ from inaccurately selected respondents on questions related to household tasks. Survey practitioners are encouraged to use active tasks such as explicit questions rather than passive tasks such as embedded instructions as part of the within-household selection process.

AB - Randomly selecting a single adult within a household is one of the biggest challenges facing mail surveys. Yet obtaining a probability sample of adults within households is critical to having a probability sample of the US adult population. In this paper, we experimentally test three alternative placements of the within-household selection instructions in the National Health, Wellbeing, and Perspectives study (sample n = 6,000; respondent n = 998): (1) a standard cover letter informing the household to ask the person with the next birthday to complete the survey (control); (2) the control cover letter plus an instruction on the front cover of the questionnaire itself to have the adult with the next birthday complete the survey; and (3) the control cover letter plus an explicit yes/ no question asking whether the individual is the adult in the household who will have the next birthday. Although the version with an explicit question had a two-point decrease in response rates relative to not having any instruction, the explicit question significantly improves selection accuracy relative to the other two designs, yields a sample composition closer to national benchmarks, and does not affect item nonresponse rates. Accurately selected respondents also differ from inaccurately selected respondents on questions related to household tasks. Survey practitioners are encouraged to use active tasks such as explicit questions rather than passive tasks such as embedded instructions as part of the within-household selection process.

UR - http://www.scopus.com/inward/record.url?scp=85029805864&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029805864&partnerID=8YFLogxK

U2 - 10.1093/poq/nfx025

DO - 10.1093/poq/nfx025

M3 - Article

AN - SCOPUS:85029805864

VL - 81

SP - 688

EP - 713

JO - Public Opinion Quarterly

JF - Public Opinion Quarterly

SN - 0033-362X

IS - 3

ER -