Open-ended questions in web surveys

Jolene D Smyth, Don A. Dillman, Leah Melani Christian, Mallory Mcbride

Research output: Contribution to journalArticle

70 Citations (Scopus)

Abstract

Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys.

Original languageEnglish (US)
Pages (from-to)325-337
Number of pages13
JournalPublic Opinion Quarterly
Volume73
Issue number2
DOIs
StatePublished - May 1 2009

Fingerprint

response behavior
random sample
Gages
manipulation
World Wide Web
inclusion
interview
time
Inclusion
Undergraduate
Salient
Length
Response Time
Manipulation

ASJC Scopus subject areas

  • Communication
  • History
  • Sociology and Political Science
  • Social Sciences(all)
  • History and Philosophy of Science

Cite this

Smyth, J. D., Dillman, D. A., Christian, L. M., & Mcbride, M. (2009). Open-ended questions in web surveys. Public Opinion Quarterly, 73(2), 325-337. https://doi.org/10.1093/poq/nfp029

Open-ended questions in web surveys. / Smyth, Jolene D; Dillman, Don A.; Christian, Leah Melani; Mcbride, Mallory.

In: Public Opinion Quarterly, Vol. 73, No. 2, 01.05.2009, p. 325-337.

Research output: Contribution to journalArticle

Smyth, JD, Dillman, DA, Christian, LM & Mcbride, M 2009, 'Open-ended questions in web surveys', Public Opinion Quarterly, vol. 73, no. 2, pp. 325-337. https://doi.org/10.1093/poq/nfp029
Smyth JD, Dillman DA, Christian LM, Mcbride M. Open-ended questions in web surveys. Public Opinion Quarterly. 2009 May 1;73(2):325-337. https://doi.org/10.1093/poq/nfp029
Smyth, Jolene D ; Dillman, Don A. ; Christian, Leah Melani ; Mcbride, Mallory. / Open-ended questions in web surveys. In: Public Opinion Quarterly. 2009 ; Vol. 73, No. 2. pp. 325-337.
@article{4338530de4c6432c8d07c18874b13d3b,
title = "Open-ended questions in web surveys",
abstract = "Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys.",
author = "Smyth, {Jolene D} and Dillman, {Don A.} and Christian, {Leah Melani} and Mallory Mcbride",
year = "2009",
month = "5",
day = "1",
doi = "10.1093/poq/nfp029",
language = "English (US)",
volume = "73",
pages = "325--337",
journal = "Public Opinion Quarterly",
issn = "0033-362X",
publisher = "Oxford University Press",
number = "2",

}

TY - JOUR

T1 - Open-ended questions in web surveys

AU - Smyth, Jolene D

AU - Dillman, Don A.

AU - Christian, Leah Melani

AU - Mcbride, Mallory

PY - 2009/5/1

Y1 - 2009/5/1

N2 - Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys.

AB - Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys.

UR - http://www.scopus.com/inward/record.url?scp=67650351483&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67650351483&partnerID=8YFLogxK

U2 - 10.1093/poq/nfp029

DO - 10.1093/poq/nfp029

M3 - Article

VL - 73

SP - 325

EP - 337

JO - Public Opinion Quarterly

JF - Public Opinion Quarterly

SN - 0033-362X

IS - 2

ER -