Effects of Stem and Response Order on Response Patterns in Satisfaction Ratings

Jolene D. Smyth, Glenn D. Israel, Milton G. Newberry, Richard G. Hull

Research output: Contribution to journalArticle

Abstract

Considerable research has examined the effect of response option order in ordinal bipolar questions such as satisfaction questions. However, no research we know of has examined the effect of the order of presentation of concepts in the question stem or whether stem order moderates response option order. In this article, we experimentally test the main and interaction effects of both stem and response option order for items in self-administered surveys on response distributions and answer changes in eight satisfied/dissatisfied questions. We find consistent evidence that response option order impacts answers. We also find that the order of “satisfied” or “dissatisfied” in the question stem impacts response distributions for four of our eight items but does not moderate the effect of response option order. We discuss the implications of our findings for questionnaire design and secondary data analyses.

Original languageEnglish (US)
Pages (from-to)260-276
Number of pages17
JournalField Methods
Volume31
Issue number3
DOIs
StatePublished - Aug 1 2019

Fingerprint

rating
questionnaire
interaction
evidence

ASJC Scopus subject areas

  • Anthropology

Cite this

Effects of Stem and Response Order on Response Patterns in Satisfaction Ratings. / Smyth, Jolene D.; Israel, Glenn D.; Newberry, Milton G.; Hull, Richard G.

In: Field Methods, Vol. 31, No. 3, 01.08.2019, p. 260-276.

Research output: Contribution to journalArticle

Smyth, Jolene D. ; Israel, Glenn D. ; Newberry, Milton G. ; Hull, Richard G. / Effects of Stem and Response Order on Response Patterns in Satisfaction Ratings. In: Field Methods. 2019 ; Vol. 31, No. 3. pp. 260-276.
@article{6ddbeb6ed8964889a66a3c9bae1ba23a,
title = "Effects of Stem and Response Order on Response Patterns in Satisfaction Ratings",
abstract = "Considerable research has examined the effect of response option order in ordinal bipolar questions such as satisfaction questions. However, no research we know of has examined the effect of the order of presentation of concepts in the question stem or whether stem order moderates response option order. In this article, we experimentally test the main and interaction effects of both stem and response option order for items in self-administered surveys on response distributions and answer changes in eight satisfied/dissatisfied questions. We find consistent evidence that response option order impacts answers. We also find that the order of “satisfied” or “dissatisfied” in the question stem impacts response distributions for four of our eight items but does not moderate the effect of response option order. We discuss the implications of our findings for questionnaire design and secondary data analyses.",
author = "Smyth, {Jolene D.} and Israel, {Glenn D.} and Newberry, {Milton G.} and Hull, {Richard G.}",
year = "2019",
month = "8",
day = "1",
doi = "10.1177/1525822X19860648",
language = "English (US)",
volume = "31",
pages = "260--276",
journal = "Field Methods",
issn = "1525-822X",
publisher = "SAGE Publications Inc.",
number = "3",

}

TY - JOUR

T1 - Effects of Stem and Response Order on Response Patterns in Satisfaction Ratings

AU - Smyth, Jolene D.

AU - Israel, Glenn D.

AU - Newberry, Milton G.

AU - Hull, Richard G.

PY - 2019/8/1

Y1 - 2019/8/1

N2 - Considerable research has examined the effect of response option order in ordinal bipolar questions such as satisfaction questions. However, no research we know of has examined the effect of the order of presentation of concepts in the question stem or whether stem order moderates response option order. In this article, we experimentally test the main and interaction effects of both stem and response option order for items in self-administered surveys on response distributions and answer changes in eight satisfied/dissatisfied questions. We find consistent evidence that response option order impacts answers. We also find that the order of “satisfied” or “dissatisfied” in the question stem impacts response distributions for four of our eight items but does not moderate the effect of response option order. We discuss the implications of our findings for questionnaire design and secondary data analyses.

AB - Considerable research has examined the effect of response option order in ordinal bipolar questions such as satisfaction questions. However, no research we know of has examined the effect of the order of presentation of concepts in the question stem or whether stem order moderates response option order. In this article, we experimentally test the main and interaction effects of both stem and response option order for items in self-administered surveys on response distributions and answer changes in eight satisfied/dissatisfied questions. We find consistent evidence that response option order impacts answers. We also find that the order of “satisfied” or “dissatisfied” in the question stem impacts response distributions for four of our eight items but does not moderate the effect of response option order. We discuss the implications of our findings for questionnaire design and secondary data analyses.

UR - http://www.scopus.com/inward/record.url?scp=85068613622&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068613622&partnerID=8YFLogxK

U2 - 10.1177/1525822X19860648

DO - 10.1177/1525822X19860648

M3 - Article

AN - SCOPUS:85068613622

VL - 31

SP - 260

EP - 276

JO - Field Methods

JF - Field Methods

SN - 1525-822X

IS - 3

ER -