How best to measure implementation of school health curricula

A comparison of three measures

Ken Resnicow, Marsha Davis, Matt Smith, Amy L Yaroch, Tom Baranowski, Janice Baranowski, Colleen Doyle, Dongqing Terry Wang

Research output: Contribution to journalArticle

85 Citations (Scopus)

Abstract

The impact of school health education programs is often attenuated by inadequate teacher implementation. Using data from a school-based nutrition education program delivered in a sample of fifth graders, this study examines the discriminant and predictive validity of three measures of curriculum implementation: classroom observation of fidelity, and two measures of completeness, teacher self-report questionnaire and post-implementation interview. A fourth measure, obtained during teacher observations, that assessed student and teacher interaction and student receptivity to the curriculum (labeled Rapport) was also obtained. Predictive validity was determined by examining the association of implementation measures with three study outcomes; health knowledge, asking behaviors related to fruit and vegetables, and fruit and vegetable intake, assessed by 7-day diary. Of the 37 teachers observed, 21 were observed for two sessions and 16 were observed once. Implementation measures were moderately correlated, an indication of discriminant validity. Predictive validity analyses indicated that the observed fidelity, Rapport and interview measures were significantly correlated with post-test student knowledge. The association between health knowledge and observed fidelity (based on dual observation only), Rapport and interview measures remained significant after adjustment for pre-test knowledge values. None of the implementation variables were significantly associated with student fruit and vegetable intake or asking behaviors controlling for pre-test values. These results indicate that the teacher self-report questionnaire was not a valid measure of implementation completeness in this study. Post-implementation completeness interviews and dual observations of fidelity and Rapport appear to be more valid, and largely independent methods of implementation assessment.

Original languageEnglish (US)
Pages (from-to)239-250
Number of pages12
JournalHealth Education Research
Volume13
Issue number2
DOIs
StatePublished - Jun 26 1998

Fingerprint

School Health Services
Curriculum
Vegetables
Interviews
Students
Fruit
curriculum
health
Self Report
school
Observation
teacher
vegetables
Health
Health Education
interview
Outcome Assessment (Health Care)
student
Education
nutrition education

ASJC Scopus subject areas

  • Education
  • Public Health, Environmental and Occupational Health

Cite this

Resnicow, K., Davis, M., Smith, M., Yaroch, A. L., Baranowski, T., Baranowski, J., ... Wang, D. T. (1998). How best to measure implementation of school health curricula: A comparison of three measures. Health Education Research, 13(2), 239-250. https://doi.org/10.1093/her/13.2.239

How best to measure implementation of school health curricula : A comparison of three measures. / Resnicow, Ken; Davis, Marsha; Smith, Matt; Yaroch, Amy L; Baranowski, Tom; Baranowski, Janice; Doyle, Colleen; Wang, Dongqing Terry.

In: Health Education Research, Vol. 13, No. 2, 26.06.1998, p. 239-250.

Research output: Contribution to journalArticle

Resnicow, K, Davis, M, Smith, M, Yaroch, AL, Baranowski, T, Baranowski, J, Doyle, C & Wang, DT 1998, 'How best to measure implementation of school health curricula: A comparison of three measures', Health Education Research, vol. 13, no. 2, pp. 239-250. https://doi.org/10.1093/her/13.2.239
Resnicow K, Davis M, Smith M, Yaroch AL, Baranowski T, Baranowski J et al. How best to measure implementation of school health curricula: A comparison of three measures. Health Education Research. 1998 Jun 26;13(2):239-250. https://doi.org/10.1093/her/13.2.239
Resnicow, Ken ; Davis, Marsha ; Smith, Matt ; Yaroch, Amy L ; Baranowski, Tom ; Baranowski, Janice ; Doyle, Colleen ; Wang, Dongqing Terry. / How best to measure implementation of school health curricula : A comparison of three measures. In: Health Education Research. 1998 ; Vol. 13, No. 2. pp. 239-250.
@article{2c6a09bcbc934d12b9292b296bf5467a,
title = "How best to measure implementation of school health curricula: A comparison of three measures",
abstract = "The impact of school health education programs is often attenuated by inadequate teacher implementation. Using data from a school-based nutrition education program delivered in a sample of fifth graders, this study examines the discriminant and predictive validity of three measures of curriculum implementation: classroom observation of fidelity, and two measures of completeness, teacher self-report questionnaire and post-implementation interview. A fourth measure, obtained during teacher observations, that assessed student and teacher interaction and student receptivity to the curriculum (labeled Rapport) was also obtained. Predictive validity was determined by examining the association of implementation measures with three study outcomes; health knowledge, asking behaviors related to fruit and vegetables, and fruit and vegetable intake, assessed by 7-day diary. Of the 37 teachers observed, 21 were observed for two sessions and 16 were observed once. Implementation measures were moderately correlated, an indication of discriminant validity. Predictive validity analyses indicated that the observed fidelity, Rapport and interview measures were significantly correlated with post-test student knowledge. The association between health knowledge and observed fidelity (based on dual observation only), Rapport and interview measures remained significant after adjustment for pre-test knowledge values. None of the implementation variables were significantly associated with student fruit and vegetable intake or asking behaviors controlling for pre-test values. These results indicate that the teacher self-report questionnaire was not a valid measure of implementation completeness in this study. Post-implementation completeness interviews and dual observations of fidelity and Rapport appear to be more valid, and largely independent methods of implementation assessment.",
author = "Ken Resnicow and Marsha Davis and Matt Smith and Yaroch, {Amy L} and Tom Baranowski and Janice Baranowski and Colleen Doyle and Wang, {Dongqing Terry}",
year = "1998",
month = "6",
day = "26",
doi = "10.1093/her/13.2.239",
language = "English (US)",
volume = "13",
pages = "239--250",
journal = "Health Education Research",
issn = "0268-1153",
publisher = "Oxford University Press",
number = "2",

}

TY - JOUR

T1 - How best to measure implementation of school health curricula

T2 - A comparison of three measures

AU - Resnicow, Ken

AU - Davis, Marsha

AU - Smith, Matt

AU - Yaroch, Amy L

AU - Baranowski, Tom

AU - Baranowski, Janice

AU - Doyle, Colleen

AU - Wang, Dongqing Terry

PY - 1998/6/26

Y1 - 1998/6/26

N2 - The impact of school health education programs is often attenuated by inadequate teacher implementation. Using data from a school-based nutrition education program delivered in a sample of fifth graders, this study examines the discriminant and predictive validity of three measures of curriculum implementation: classroom observation of fidelity, and two measures of completeness, teacher self-report questionnaire and post-implementation interview. A fourth measure, obtained during teacher observations, that assessed student and teacher interaction and student receptivity to the curriculum (labeled Rapport) was also obtained. Predictive validity was determined by examining the association of implementation measures with three study outcomes; health knowledge, asking behaviors related to fruit and vegetables, and fruit and vegetable intake, assessed by 7-day diary. Of the 37 teachers observed, 21 were observed for two sessions and 16 were observed once. Implementation measures were moderately correlated, an indication of discriminant validity. Predictive validity analyses indicated that the observed fidelity, Rapport and interview measures were significantly correlated with post-test student knowledge. The association between health knowledge and observed fidelity (based on dual observation only), Rapport and interview measures remained significant after adjustment for pre-test knowledge values. None of the implementation variables were significantly associated with student fruit and vegetable intake or asking behaviors controlling for pre-test values. These results indicate that the teacher self-report questionnaire was not a valid measure of implementation completeness in this study. Post-implementation completeness interviews and dual observations of fidelity and Rapport appear to be more valid, and largely independent methods of implementation assessment.

AB - The impact of school health education programs is often attenuated by inadequate teacher implementation. Using data from a school-based nutrition education program delivered in a sample of fifth graders, this study examines the discriminant and predictive validity of three measures of curriculum implementation: classroom observation of fidelity, and two measures of completeness, teacher self-report questionnaire and post-implementation interview. A fourth measure, obtained during teacher observations, that assessed student and teacher interaction and student receptivity to the curriculum (labeled Rapport) was also obtained. Predictive validity was determined by examining the association of implementation measures with three study outcomes; health knowledge, asking behaviors related to fruit and vegetables, and fruit and vegetable intake, assessed by 7-day diary. Of the 37 teachers observed, 21 were observed for two sessions and 16 were observed once. Implementation measures were moderately correlated, an indication of discriminant validity. Predictive validity analyses indicated that the observed fidelity, Rapport and interview measures were significantly correlated with post-test student knowledge. The association between health knowledge and observed fidelity (based on dual observation only), Rapport and interview measures remained significant after adjustment for pre-test knowledge values. None of the implementation variables were significantly associated with student fruit and vegetable intake or asking behaviors controlling for pre-test values. These results indicate that the teacher self-report questionnaire was not a valid measure of implementation completeness in this study. Post-implementation completeness interviews and dual observations of fidelity and Rapport appear to be more valid, and largely independent methods of implementation assessment.

UR - http://www.scopus.com/inward/record.url?scp=0031861231&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0031861231&partnerID=8YFLogxK

U2 - 10.1093/her/13.2.239

DO - 10.1093/her/13.2.239

M3 - Article

VL - 13

SP - 239

EP - 250

JO - Health Education Research

JF - Health Education Research

SN - 0268-1153

IS - 2

ER -