Assessing the internal and external validity of mobile health physical activity promotion interventions: A systematic literaturereview using the RE-AIM framework

Kacie C.A. Blackman, Jamie Zoellner, Leanna M. Berrey, Ramine Alexander, Jason Fanning, Jennie L. Hill, Paul A. Estabrooks

Research output: Contribution to journalReview article

36 Citations (Scopus)

Abstract

Background: Mobile health (mHealth) interventions are effective in promoting physical activity (PA); however, the degree to which external validity indicators are reported is unclear. Objective: The purpose of this systematic review was to use the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to determine the extent to which mHealth intervention research for promoting PA reports on factors that inform generalizability across settings and populations and to provide recommendations for investigators planning to conduct this type of research. Methods: Twenty articles reflecting 15 trials published between 2000 and 2012 were identified through a systematic review process (ie, queries of three online databases and reference lists of eligible articles) and met inclusion criteria (ie, implementation of mobile technologies, target physical activity, and provide original data). Two researchers coded each article using a validated RE-AIM data extraction tool (reach, efficacy/effectiveness, adoption, implementation, maintenance). Two members of the study team independently abstracted information from each article (inter-rater reliability >90%) and group meetings were used to gain consensus on discrepancies. Results: The majority of studies were randomized controlled trials (n=14). The average reporting across RE-AIM indicators varied by dimension (reach=53.3%, 2.67/5; effectiveness/efficacy=60.0%, 2.4/4; adoption=11.1%, 0.7/6; implementation=24.4%, 0.7/3; maintenance=0%, 0/3). While most studies described changes in the primary outcome (effectiveness), few addressed the representativeness of participants (reach) or settings (adoption) and few reported on issues related to maintenance and degree of implementation fidelity. Conclusions: This review suggests that more focus is needed on research designs that highlight and report on both internal and external validity indicators. Specific recommendations are provided to encourage future mHealth interventionists and investigators to report on representativeness, settings, delivery agents for planned interventions, the extent to which protocol is delivered as intended, and maintenance of effects at the individual or organizational level.

Original languageEnglish (US)
Article number2745
JournalJournal of Medical Internet Research
Volume15
Issue number10
DOIs
StatePublished - Oct 1 2013

Fingerprint

Telemedicine
Maintenance
Research Personnel
Group Processes
Research
Consensus
Research Design
Randomized Controlled Trials
Databases
Technology
Population

Keywords

  • Generalizability
  • Mobile technology
  • Physical activity
  • Review

ASJC Scopus subject areas

  • Health Informatics

Cite this

Assessing the internal and external validity of mobile health physical activity promotion interventions : A systematic literaturereview using the RE-AIM framework. / Blackman, Kacie C.A.; Zoellner, Jamie; Berrey, Leanna M.; Alexander, Ramine; Fanning, Jason; Hill, Jennie L.; Estabrooks, Paul A.

In: Journal of Medical Internet Research, Vol. 15, No. 10, 2745, 01.10.2013.

Research output: Contribution to journalReview article

@article{1e903684579d4d79b0f2b9af9a1d1624,
title = "Assessing the internal and external validity of mobile health physical activity promotion interventions: A systematic literaturereview using the RE-AIM framework",
abstract = "Background: Mobile health (mHealth) interventions are effective in promoting physical activity (PA); however, the degree to which external validity indicators are reported is unclear. Objective: The purpose of this systematic review was to use the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to determine the extent to which mHealth intervention research for promoting PA reports on factors that inform generalizability across settings and populations and to provide recommendations for investigators planning to conduct this type of research. Methods: Twenty articles reflecting 15 trials published between 2000 and 2012 were identified through a systematic review process (ie, queries of three online databases and reference lists of eligible articles) and met inclusion criteria (ie, implementation of mobile technologies, target physical activity, and provide original data). Two researchers coded each article using a validated RE-AIM data extraction tool (reach, efficacy/effectiveness, adoption, implementation, maintenance). Two members of the study team independently abstracted information from each article (inter-rater reliability >90{\%}) and group meetings were used to gain consensus on discrepancies. Results: The majority of studies were randomized controlled trials (n=14). The average reporting across RE-AIM indicators varied by dimension (reach=53.3{\%}, 2.67/5; effectiveness/efficacy=60.0{\%}, 2.4/4; adoption=11.1{\%}, 0.7/6; implementation=24.4{\%}, 0.7/3; maintenance=0{\%}, 0/3). While most studies described changes in the primary outcome (effectiveness), few addressed the representativeness of participants (reach) or settings (adoption) and few reported on issues related to maintenance and degree of implementation fidelity. Conclusions: This review suggests that more focus is needed on research designs that highlight and report on both internal and external validity indicators. Specific recommendations are provided to encourage future mHealth interventionists and investigators to report on representativeness, settings, delivery agents for planned interventions, the extent to which protocol is delivered as intended, and maintenance of effects at the individual or organizational level.",
keywords = "Generalizability, Mobile technology, Physical activity, Review",
author = "Blackman, {Kacie C.A.} and Jamie Zoellner and Berrey, {Leanna M.} and Ramine Alexander and Jason Fanning and Hill, {Jennie L.} and Estabrooks, {Paul A.}",
year = "2013",
month = "10",
day = "1",
doi = "10.2196/jmir.2745",
language = "English (US)",
volume = "15",
journal = "Journal of Medical Internet Research",
issn = "1439-4456",
publisher = "Journal of medical Internet Research",
number = "10",

}

TY - JOUR

T1 - Assessing the internal and external validity of mobile health physical activity promotion interventions

T2 - A systematic literaturereview using the RE-AIM framework

AU - Blackman, Kacie C.A.

AU - Zoellner, Jamie

AU - Berrey, Leanna M.

AU - Alexander, Ramine

AU - Fanning, Jason

AU - Hill, Jennie L.

AU - Estabrooks, Paul A.

PY - 2013/10/1

Y1 - 2013/10/1

N2 - Background: Mobile health (mHealth) interventions are effective in promoting physical activity (PA); however, the degree to which external validity indicators are reported is unclear. Objective: The purpose of this systematic review was to use the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to determine the extent to which mHealth intervention research for promoting PA reports on factors that inform generalizability across settings and populations and to provide recommendations for investigators planning to conduct this type of research. Methods: Twenty articles reflecting 15 trials published between 2000 and 2012 were identified through a systematic review process (ie, queries of three online databases and reference lists of eligible articles) and met inclusion criteria (ie, implementation of mobile technologies, target physical activity, and provide original data). Two researchers coded each article using a validated RE-AIM data extraction tool (reach, efficacy/effectiveness, adoption, implementation, maintenance). Two members of the study team independently abstracted information from each article (inter-rater reliability >90%) and group meetings were used to gain consensus on discrepancies. Results: The majority of studies were randomized controlled trials (n=14). The average reporting across RE-AIM indicators varied by dimension (reach=53.3%, 2.67/5; effectiveness/efficacy=60.0%, 2.4/4; adoption=11.1%, 0.7/6; implementation=24.4%, 0.7/3; maintenance=0%, 0/3). While most studies described changes in the primary outcome (effectiveness), few addressed the representativeness of participants (reach) or settings (adoption) and few reported on issues related to maintenance and degree of implementation fidelity. Conclusions: This review suggests that more focus is needed on research designs that highlight and report on both internal and external validity indicators. Specific recommendations are provided to encourage future mHealth interventionists and investigators to report on representativeness, settings, delivery agents for planned interventions, the extent to which protocol is delivered as intended, and maintenance of effects at the individual or organizational level.

AB - Background: Mobile health (mHealth) interventions are effective in promoting physical activity (PA); however, the degree to which external validity indicators are reported is unclear. Objective: The purpose of this systematic review was to use the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to determine the extent to which mHealth intervention research for promoting PA reports on factors that inform generalizability across settings and populations and to provide recommendations for investigators planning to conduct this type of research. Methods: Twenty articles reflecting 15 trials published between 2000 and 2012 were identified through a systematic review process (ie, queries of three online databases and reference lists of eligible articles) and met inclusion criteria (ie, implementation of mobile technologies, target physical activity, and provide original data). Two researchers coded each article using a validated RE-AIM data extraction tool (reach, efficacy/effectiveness, adoption, implementation, maintenance). Two members of the study team independently abstracted information from each article (inter-rater reliability >90%) and group meetings were used to gain consensus on discrepancies. Results: The majority of studies were randomized controlled trials (n=14). The average reporting across RE-AIM indicators varied by dimension (reach=53.3%, 2.67/5; effectiveness/efficacy=60.0%, 2.4/4; adoption=11.1%, 0.7/6; implementation=24.4%, 0.7/3; maintenance=0%, 0/3). While most studies described changes in the primary outcome (effectiveness), few addressed the representativeness of participants (reach) or settings (adoption) and few reported on issues related to maintenance and degree of implementation fidelity. Conclusions: This review suggests that more focus is needed on research designs that highlight and report on both internal and external validity indicators. Specific recommendations are provided to encourage future mHealth interventionists and investigators to report on representativeness, settings, delivery agents for planned interventions, the extent to which protocol is delivered as intended, and maintenance of effects at the individual or organizational level.

KW - Generalizability

KW - Mobile technology

KW - Physical activity

KW - Review

UR - http://www.scopus.com/inward/record.url?scp=84887877073&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84887877073&partnerID=8YFLogxK

U2 - 10.2196/jmir.2745

DO - 10.2196/jmir.2745

M3 - Review article

C2 - 24095951

AN - SCOPUS:84887877073

VL - 15

JO - Journal of Medical Internet Research

JF - Journal of Medical Internet Research

SN - 1439-4456

IS - 10

M1 - 2745

ER -