Mixed methods grant applications in the health sciences: An analysis of reviewer comments

Timothy C. Guetterman, Rae V. Sakakibara, Vicki L. Plano Clark, Mark Luborsky, Sarah M. Murray, Felipe González Castro, John W. Creswell, Charles Deutsch, Joseph J. Gallo

Research output: Contribution to journalReview article

Abstract

Our aim was to understand how reviewers appraise mixed methods research by analyzing reviewer comments for grant applications submitted primarily to the National Institutes of Health. We requested scholars and consultants in the Mixed Methods Research Training Program (MMRTP) for the Health Sciences to send us summary statements from their mixed methods grant applications and obtained 40 summary statements of funded (40%) and unfunded (60%) mixed methods grant applications. We conducted a document analysis using a coding rubric based on the NIH Best Practices for Mixed Methods Research in the Health Sciences and allowed inductive codes to emerge. Reviewers favorably appraised mixed methods applications demonstrating coherence among aims and research design elements, detailed methods, plans for mixed methods integration, and the use of theoretical models. Reviewers identified weaknesses in mixed methods applications that lacked methodological details or rationales, had a high participant burden, and failed to delineate investigator roles. Successful mixed methods applications convey assumptions behind the methods chosen to accomplish specific aims and clearly detail the procedures to be taken. Investigators planning to use mixed methods should remember that reviewers are looking for both points of view.

Original languageEnglish (US)
Article numbere0225308
JournalPloS one
Volume14
Issue number11
DOIs
StatePublished - Jan 1 2019

Fingerprint

Health
research methods
application methods
methodology
National Institutes of Health
consultants
education programs
planning
Research
Research Personnel
Planning
Organized Financing
National Institutes of Health (U.S.)
Consultants
Practice Guidelines
Research Design
Theoretical Models
Education

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)
  • General

Cite this

Guetterman, T. C., Sakakibara, R. V., Plano Clark, V. L., Luborsky, M., Murray, S. M., Castro, F. G., ... Gallo, J. J. (2019). Mixed methods grant applications in the health sciences: An analysis of reviewer comments. PloS one, 14(11), [e0225308]. https://doi.org/10.1371/journal.pone.0225308

Mixed methods grant applications in the health sciences : An analysis of reviewer comments. / Guetterman, Timothy C.; Sakakibara, Rae V.; Plano Clark, Vicki L.; Luborsky, Mark; Murray, Sarah M.; Castro, Felipe González; Creswell, John W.; Deutsch, Charles; Gallo, Joseph J.

In: PloS one, Vol. 14, No. 11, e0225308, 01.01.2019.

Research output: Contribution to journalReview article

Guetterman, TC, Sakakibara, RV, Plano Clark, VL, Luborsky, M, Murray, SM, Castro, FG, Creswell, JW, Deutsch, C & Gallo, JJ 2019, 'Mixed methods grant applications in the health sciences: An analysis of reviewer comments', PloS one, vol. 14, no. 11, e0225308. https://doi.org/10.1371/journal.pone.0225308
Guetterman TC, Sakakibara RV, Plano Clark VL, Luborsky M, Murray SM, Castro FG et al. Mixed methods grant applications in the health sciences: An analysis of reviewer comments. PloS one. 2019 Jan 1;14(11). e0225308. https://doi.org/10.1371/journal.pone.0225308
Guetterman, Timothy C. ; Sakakibara, Rae V. ; Plano Clark, Vicki L. ; Luborsky, Mark ; Murray, Sarah M. ; Castro, Felipe González ; Creswell, John W. ; Deutsch, Charles ; Gallo, Joseph J. / Mixed methods grant applications in the health sciences : An analysis of reviewer comments. In: PloS one. 2019 ; Vol. 14, No. 11.
@article{070b2a7de23c4c2f822ea1a9c3d46424,
title = "Mixed methods grant applications in the health sciences: An analysis of reviewer comments",
abstract = "Our aim was to understand how reviewers appraise mixed methods research by analyzing reviewer comments for grant applications submitted primarily to the National Institutes of Health. We requested scholars and consultants in the Mixed Methods Research Training Program (MMRTP) for the Health Sciences to send us summary statements from their mixed methods grant applications and obtained 40 summary statements of funded (40{\%}) and unfunded (60{\%}) mixed methods grant applications. We conducted a document analysis using a coding rubric based on the NIH Best Practices for Mixed Methods Research in the Health Sciences and allowed inductive codes to emerge. Reviewers favorably appraised mixed methods applications demonstrating coherence among aims and research design elements, detailed methods, plans for mixed methods integration, and the use of theoretical models. Reviewers identified weaknesses in mixed methods applications that lacked methodological details or rationales, had a high participant burden, and failed to delineate investigator roles. Successful mixed methods applications convey assumptions behind the methods chosen to accomplish specific aims and clearly detail the procedures to be taken. Investigators planning to use mixed methods should remember that reviewers are looking for both points of view.",
author = "Guetterman, {Timothy C.} and Sakakibara, {Rae V.} and {Plano Clark}, {Vicki L.} and Mark Luborsky and Murray, {Sarah M.} and Castro, {Felipe Gonz{\'a}lez} and Creswell, {John W.} and Charles Deutsch and Gallo, {Joseph J.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1371/journal.pone.0225308",
language = "English (US)",
volume = "14",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "11",

}

TY - JOUR

T1 - Mixed methods grant applications in the health sciences

T2 - An analysis of reviewer comments

AU - Guetterman, Timothy C.

AU - Sakakibara, Rae V.

AU - Plano Clark, Vicki L.

AU - Luborsky, Mark

AU - Murray, Sarah M.

AU - Castro, Felipe González

AU - Creswell, John W.

AU - Deutsch, Charles

AU - Gallo, Joseph J.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Our aim was to understand how reviewers appraise mixed methods research by analyzing reviewer comments for grant applications submitted primarily to the National Institutes of Health. We requested scholars and consultants in the Mixed Methods Research Training Program (MMRTP) for the Health Sciences to send us summary statements from their mixed methods grant applications and obtained 40 summary statements of funded (40%) and unfunded (60%) mixed methods grant applications. We conducted a document analysis using a coding rubric based on the NIH Best Practices for Mixed Methods Research in the Health Sciences and allowed inductive codes to emerge. Reviewers favorably appraised mixed methods applications demonstrating coherence among aims and research design elements, detailed methods, plans for mixed methods integration, and the use of theoretical models. Reviewers identified weaknesses in mixed methods applications that lacked methodological details or rationales, had a high participant burden, and failed to delineate investigator roles. Successful mixed methods applications convey assumptions behind the methods chosen to accomplish specific aims and clearly detail the procedures to be taken. Investigators planning to use mixed methods should remember that reviewers are looking for both points of view.

AB - Our aim was to understand how reviewers appraise mixed methods research by analyzing reviewer comments for grant applications submitted primarily to the National Institutes of Health. We requested scholars and consultants in the Mixed Methods Research Training Program (MMRTP) for the Health Sciences to send us summary statements from their mixed methods grant applications and obtained 40 summary statements of funded (40%) and unfunded (60%) mixed methods grant applications. We conducted a document analysis using a coding rubric based on the NIH Best Practices for Mixed Methods Research in the Health Sciences and allowed inductive codes to emerge. Reviewers favorably appraised mixed methods applications demonstrating coherence among aims and research design elements, detailed methods, plans for mixed methods integration, and the use of theoretical models. Reviewers identified weaknesses in mixed methods applications that lacked methodological details or rationales, had a high participant burden, and failed to delineate investigator roles. Successful mixed methods applications convey assumptions behind the methods chosen to accomplish specific aims and clearly detail the procedures to be taken. Investigators planning to use mixed methods should remember that reviewers are looking for both points of view.

UR - http://www.scopus.com/inward/record.url?scp=85075042151&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075042151&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0225308

DO - 10.1371/journal.pone.0225308

M3 - Review article

C2 - 31730660

AN - SCOPUS:85075042151

VL - 14

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 11

M1 - e0225308

ER -