Comparing check-all and forced-choice question formats in Web surveys

Jolene D. Smyth, Don A. Dillman, Leah Melani Christian, Michael J. Stern

Research output: Contribution to journalReview article

117 Scopus citations

Abstract

For survey researchers, it is common practice to use the check-all question format in Web and mail surveys but to convert to the forced-choice question format in telephone surveys. The assumption underlying this practice is that respondents will answer the two formats similarly. In this research note we report results from 16 experimental comparisons in two Web surveys and a paper survey conducted in 2002 and 2003 that test whether the check-all and forced-choice formats produce similar results. In all 16 comparisons, we find that the two question formats do not perform similarly; respondents endorse more options and take longer to answer in the forced-choice format than in the check-all format. These findings suggest that the forced-choice question format encourages deeper processing of response options and, as such, is preferable to the check-all format, which may encourage a weak satisficing response strategy. Additional analyses show that neither acquiescence bias nor item nonresponse seem to pose substantial problems for use of the forced-choice question format in Web surveys.

Original languageEnglish (US)
Pages (from-to)66-77
Number of pages12
JournalPublic Opinion Quarterly
Volume70
Issue number1
DOIs
StatePublished - Mar 1 2006

    Fingerprint

ASJC Scopus subject areas

  • Communication
  • History
  • Sociology and Political Science
  • Social Sciences(all)
  • History and Philosophy of Science

Cite this