The Hawthorne effect, sponsored trials, and the overestimation of treatment effectiveness

Frederick Wolfe, Kaleb D Michaud

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

Objective. To determine if the results of rheumatoid arthritis (RA) clinical trials are upwardly biased by the Hawthorne effect. Methods. We studied 264 patients with RA who completed a commercially sponsored 3-month, open-label, phase 4 trial of a US Food and Drug Administration approved RA treatment. We evaluated changes in the Health Assessment Questionnaire disability index (HAQ) and visual analog scales for pain, patient global, and fatigue during 3 periods: pretreatment in the trial, on treatment at the close of the trial, and by a trial-unrelated survey 8 months after the close of the trial, but while the patients were receiving the same treatment. Results. The HAQ score (0-3) improved by 41.3% during the trial, but only by 16.5% when the end-point was the post-trial result. Similar results for the other variables were patient global (0-10) 51.9% and 34.6%, pain (0-10) 51.7% and 39.7%, fatigue (0-10) 45.6% and 24.6%. Worsening between the trial end and the first survey assessment was HAQ 0.29 units, pain 0.8 units, patient global 0.8 units, and fatigue 1.1 units. Conclusion. Almost half the improvement noted in the clinical trial HAQ score disappeared on entry to a non-sponsored followup study, and from 23% to 44% of improvements in pain, patient global, and fatigue also disappeared. These changes can be attributed to the Hawthorne effect. Based on these data, we hypothesize that the absolute values of RA outcome variables in clinical trials are upwardly biased, and that the treatment effect is less than observed. The Journal of Rheumatology

Original languageEnglish (US)
Pages (from-to)2216-2220
Number of pages5
JournalJournal of Rheumatology
Volume37
Issue number11
DOIs
StatePublished - Nov 1 2010

Fingerprint

ametantrone
Epidemiologic Effect Modifiers
Fatigue
Rheumatoid Arthritis
Clinical Trials
Pain
Rheumatology
Pain Measurement
United States Food and Drug Administration
Therapeutics
Health
Surveys and Questionnaires

Keywords

  • Clinical trials
  • Effectiveness
  • Hawthorne effect
  • Rheumatoid arthritis

ASJC Scopus subject areas

  • Rheumatology
  • Immunology and Allergy
  • Immunology

Cite this

The Hawthorne effect, sponsored trials, and the overestimation of treatment effectiveness. / Wolfe, Frederick; Michaud, Kaleb D.

In: Journal of Rheumatology, Vol. 37, No. 11, 01.11.2010, p. 2216-2220.

Research output: Contribution to journalArticle

@article{cda3420e44d74e409cd032b2268d0af0,
title = "The Hawthorne effect, sponsored trials, and the overestimation of treatment effectiveness",
abstract = "Objective. To determine if the results of rheumatoid arthritis (RA) clinical trials are upwardly biased by the Hawthorne effect. Methods. We studied 264 patients with RA who completed a commercially sponsored 3-month, open-label, phase 4 trial of a US Food and Drug Administration approved RA treatment. We evaluated changes in the Health Assessment Questionnaire disability index (HAQ) and visual analog scales for pain, patient global, and fatigue during 3 periods: pretreatment in the trial, on treatment at the close of the trial, and by a trial-unrelated survey 8 months after the close of the trial, but while the patients were receiving the same treatment. Results. The HAQ score (0-3) improved by 41.3{\%} during the trial, but only by 16.5{\%} when the end-point was the post-trial result. Similar results for the other variables were patient global (0-10) 51.9{\%} and 34.6{\%}, pain (0-10) 51.7{\%} and 39.7{\%}, fatigue (0-10) 45.6{\%} and 24.6{\%}. Worsening between the trial end and the first survey assessment was HAQ 0.29 units, pain 0.8 units, patient global 0.8 units, and fatigue 1.1 units. Conclusion. Almost half the improvement noted in the clinical trial HAQ score disappeared on entry to a non-sponsored followup study, and from 23{\%} to 44{\%} of improvements in pain, patient global, and fatigue also disappeared. These changes can be attributed to the Hawthorne effect. Based on these data, we hypothesize that the absolute values of RA outcome variables in clinical trials are upwardly biased, and that the treatment effect is less than observed. The Journal of Rheumatology",
keywords = "Clinical trials, Effectiveness, Hawthorne effect, Rheumatoid arthritis",
author = "Frederick Wolfe and Michaud, {Kaleb D}",
year = "2010",
month = "11",
day = "1",
doi = "10.3899/jrheum.100497",
language = "English (US)",
volume = "37",
pages = "2216--2220",
journal = "Journal of Rheumatology",
issn = "0315-162X",
publisher = "Journal of Rheumatology",
number = "11",

}

TY - JOUR

T1 - The Hawthorne effect, sponsored trials, and the overestimation of treatment effectiveness

AU - Wolfe, Frederick

AU - Michaud, Kaleb D

PY - 2010/11/1

Y1 - 2010/11/1

N2 - Objective. To determine if the results of rheumatoid arthritis (RA) clinical trials are upwardly biased by the Hawthorne effect. Methods. We studied 264 patients with RA who completed a commercially sponsored 3-month, open-label, phase 4 trial of a US Food and Drug Administration approved RA treatment. We evaluated changes in the Health Assessment Questionnaire disability index (HAQ) and visual analog scales for pain, patient global, and fatigue during 3 periods: pretreatment in the trial, on treatment at the close of the trial, and by a trial-unrelated survey 8 months after the close of the trial, but while the patients were receiving the same treatment. Results. The HAQ score (0-3) improved by 41.3% during the trial, but only by 16.5% when the end-point was the post-trial result. Similar results for the other variables were patient global (0-10) 51.9% and 34.6%, pain (0-10) 51.7% and 39.7%, fatigue (0-10) 45.6% and 24.6%. Worsening between the trial end and the first survey assessment was HAQ 0.29 units, pain 0.8 units, patient global 0.8 units, and fatigue 1.1 units. Conclusion. Almost half the improvement noted in the clinical trial HAQ score disappeared on entry to a non-sponsored followup study, and from 23% to 44% of improvements in pain, patient global, and fatigue also disappeared. These changes can be attributed to the Hawthorne effect. Based on these data, we hypothesize that the absolute values of RA outcome variables in clinical trials are upwardly biased, and that the treatment effect is less than observed. The Journal of Rheumatology

AB - Objective. To determine if the results of rheumatoid arthritis (RA) clinical trials are upwardly biased by the Hawthorne effect. Methods. We studied 264 patients with RA who completed a commercially sponsored 3-month, open-label, phase 4 trial of a US Food and Drug Administration approved RA treatment. We evaluated changes in the Health Assessment Questionnaire disability index (HAQ) and visual analog scales for pain, patient global, and fatigue during 3 periods: pretreatment in the trial, on treatment at the close of the trial, and by a trial-unrelated survey 8 months after the close of the trial, but while the patients were receiving the same treatment. Results. The HAQ score (0-3) improved by 41.3% during the trial, but only by 16.5% when the end-point was the post-trial result. Similar results for the other variables were patient global (0-10) 51.9% and 34.6%, pain (0-10) 51.7% and 39.7%, fatigue (0-10) 45.6% and 24.6%. Worsening between the trial end and the first survey assessment was HAQ 0.29 units, pain 0.8 units, patient global 0.8 units, and fatigue 1.1 units. Conclusion. Almost half the improvement noted in the clinical trial HAQ score disappeared on entry to a non-sponsored followup study, and from 23% to 44% of improvements in pain, patient global, and fatigue also disappeared. These changes can be attributed to the Hawthorne effect. Based on these data, we hypothesize that the absolute values of RA outcome variables in clinical trials are upwardly biased, and that the treatment effect is less than observed. The Journal of Rheumatology

KW - Clinical trials

KW - Effectiveness

KW - Hawthorne effect

KW - Rheumatoid arthritis

UR - http://www.scopus.com/inward/record.url?scp=78149254491&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78149254491&partnerID=8YFLogxK

U2 - 10.3899/jrheum.100497

DO - 10.3899/jrheum.100497

M3 - Article

VL - 37

SP - 2216

EP - 2220

JO - Journal of Rheumatology

JF - Journal of Rheumatology

SN - 0315-162X

IS - 11

ER -