The effects of respondent and question characteristics on respondent answering behaviors in telephone interviews

Kristen Olson, Jolene D. Smyth, Amanda Ganshert

Research output: Contribution to journalArticle

Abstract

In a standardized telephone interview, respondents ideally are able to provide an answer that easily fits the response task. Deviations from this ideal question answering behavior are behavioral manifestations of breakdowns in the cognitive response process and partially reveal mechanisms underlying measurement error, but little is known about what question characteristics or types of respondents are associated with what types of deviations. Evaluations of question problems tend to look at one question characteristic at a time; yet questions are comprised of multiple characteristics, some of which are easier to experimentally manipulate (e.g., presence of a definition) than others (e.g., attitude versus behavior). All of these characteristics can affect how respondents answer questions. Using a landline telephone interview, we use cross-classified random effects logistic regression models to simultaneously evaluate the effects of multiple question and respondent characteristics on six different respondent behaviors. We find that most of the variability in these respondent answering behaviors is associated with the questions rather than the respondents themselves. Question characteristics that affect the comprehension and mapping stages of the cognitive response process are consistently associated with answering behaviors, whereas attitude questions do not consistently differ from behavioral questions. We also find that sensitive questions are more likely to yield adequate answers and fewer problems in reporting or clarification requests than nonsensitive questions. Additionally, older respondents are less likely to answer adequately. Our findings suggest that survey designers should focus on questionnaire features related to comprehension and mapping to minimize interactional and data quality problems in surveys and should train interviewers on how to resolve these reporting problems.

Original languageEnglish (US)
Pages (from-to)275-308
Number of pages34
JournalJournal of Survey Statistics and Methodology
Volume7
Issue number2
DOIs
StatePublished - Jan 1 2019

Fingerprint

telephone interview
Telephone
Measurement errors
Logistics
comprehension
Deviation
Likely
data quality
Question Answering
Logistic Regression Model
Data Quality
telephone
Random Effects
Measurement Error
Questionnaire
logistics
Breakdown
Resolve
regression
Tend

Keywords

  • Interviewer-respondent interaction
  • Question features
  • Respondent behaviors
  • Telephone surveys

ASJC Scopus subject areas

  • Statistics and Probability
  • Social Sciences (miscellaneous)
  • Statistics, Probability and Uncertainty
  • Applied Mathematics

Cite this

@article{18906a7d32d04a1fa4571ae83d08caa6,
title = "The effects of respondent and question characteristics on respondent answering behaviors in telephone interviews",
abstract = "In a standardized telephone interview, respondents ideally are able to provide an answer that easily fits the response task. Deviations from this ideal question answering behavior are behavioral manifestations of breakdowns in the cognitive response process and partially reveal mechanisms underlying measurement error, but little is known about what question characteristics or types of respondents are associated with what types of deviations. Evaluations of question problems tend to look at one question characteristic at a time; yet questions are comprised of multiple characteristics, some of which are easier to experimentally manipulate (e.g., presence of a definition) than others (e.g., attitude versus behavior). All of these characteristics can affect how respondents answer questions. Using a landline telephone interview, we use cross-classified random effects logistic regression models to simultaneously evaluate the effects of multiple question and respondent characteristics on six different respondent behaviors. We find that most of the variability in these respondent answering behaviors is associated with the questions rather than the respondents themselves. Question characteristics that affect the comprehension and mapping stages of the cognitive response process are consistently associated with answering behaviors, whereas attitude questions do not consistently differ from behavioral questions. We also find that sensitive questions are more likely to yield adequate answers and fewer problems in reporting or clarification requests than nonsensitive questions. Additionally, older respondents are less likely to answer adequately. Our findings suggest that survey designers should focus on questionnaire features related to comprehension and mapping to minimize interactional and data quality problems in surveys and should train interviewers on how to resolve these reporting problems.",
keywords = "Interviewer-respondent interaction, Question features, Respondent behaviors, Telephone surveys",
author = "Kristen Olson and Smyth, {Jolene D.} and Amanda Ganshert",
year = "2019",
month = "1",
day = "1",
doi = "10.1093/jssam/smy006",
language = "English (US)",
volume = "7",
pages = "275--308",
journal = "Journal of Survey Statistics and Methodology",
issn = "2325-0984",
publisher = "Oxford University Press",
number = "2",

}

TY - JOUR

T1 - The effects of respondent and question characteristics on respondent answering behaviors in telephone interviews

AU - Olson, Kristen

AU - Smyth, Jolene D.

AU - Ganshert, Amanda

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In a standardized telephone interview, respondents ideally are able to provide an answer that easily fits the response task. Deviations from this ideal question answering behavior are behavioral manifestations of breakdowns in the cognitive response process and partially reveal mechanisms underlying measurement error, but little is known about what question characteristics or types of respondents are associated with what types of deviations. Evaluations of question problems tend to look at one question characteristic at a time; yet questions are comprised of multiple characteristics, some of which are easier to experimentally manipulate (e.g., presence of a definition) than others (e.g., attitude versus behavior). All of these characteristics can affect how respondents answer questions. Using a landline telephone interview, we use cross-classified random effects logistic regression models to simultaneously evaluate the effects of multiple question and respondent characteristics on six different respondent behaviors. We find that most of the variability in these respondent answering behaviors is associated with the questions rather than the respondents themselves. Question characteristics that affect the comprehension and mapping stages of the cognitive response process are consistently associated with answering behaviors, whereas attitude questions do not consistently differ from behavioral questions. We also find that sensitive questions are more likely to yield adequate answers and fewer problems in reporting or clarification requests than nonsensitive questions. Additionally, older respondents are less likely to answer adequately. Our findings suggest that survey designers should focus on questionnaire features related to comprehension and mapping to minimize interactional and data quality problems in surveys and should train interviewers on how to resolve these reporting problems.

AB - In a standardized telephone interview, respondents ideally are able to provide an answer that easily fits the response task. Deviations from this ideal question answering behavior are behavioral manifestations of breakdowns in the cognitive response process and partially reveal mechanisms underlying measurement error, but little is known about what question characteristics or types of respondents are associated with what types of deviations. Evaluations of question problems tend to look at one question characteristic at a time; yet questions are comprised of multiple characteristics, some of which are easier to experimentally manipulate (e.g., presence of a definition) than others (e.g., attitude versus behavior). All of these characteristics can affect how respondents answer questions. Using a landline telephone interview, we use cross-classified random effects logistic regression models to simultaneously evaluate the effects of multiple question and respondent characteristics on six different respondent behaviors. We find that most of the variability in these respondent answering behaviors is associated with the questions rather than the respondents themselves. Question characteristics that affect the comprehension and mapping stages of the cognitive response process are consistently associated with answering behaviors, whereas attitude questions do not consistently differ from behavioral questions. We also find that sensitive questions are more likely to yield adequate answers and fewer problems in reporting or clarification requests than nonsensitive questions. Additionally, older respondents are less likely to answer adequately. Our findings suggest that survey designers should focus on questionnaire features related to comprehension and mapping to minimize interactional and data quality problems in surveys and should train interviewers on how to resolve these reporting problems.

KW - Interviewer-respondent interaction

KW - Question features

KW - Respondent behaviors

KW - Telephone surveys

UR - http://www.scopus.com/inward/record.url?scp=85072175681&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072175681&partnerID=8YFLogxK

U2 - 10.1093/jssam/smy006

DO - 10.1093/jssam/smy006

M3 - Article

AN - SCOPUS:85072175681

VL - 7

SP - 275

EP - 308

JO - Journal of Survey Statistics and Methodology

JF - Journal of Survey Statistics and Methodology

SN - 2325-0984

IS - 2

ER -