Toward the development of structured criteria for interpretation of functional analysis data

Louis P. Hagopian, Wayne W Fisher, Rachel H. Thompson, Jamie Owen-Deschryver, Brian A. Iwata, David P. Wacker

Research output: Contribution to journalArticle

93 Citations (Scopus)

Abstract

Using functional analysis results to prescribe treatments is the preferred method for developing behavioral interventions. Little is known, however, about the reliability and validity of visual inspection for the interpretation of functional analysis data. The purpose of this investigation was to develop a set of structured criteria for visual inspection of multielement functional analyses that, when applied correctly, would increase interrater agreement and agreement with interpretations reached by expert consensus. In Study 1, 3 predoctoral interns interpreted functional analysis graphs, and interrater agreement was low (M = .46). In Study 2, 64 functional analysis graphs were interpreted by a panel of experts, and then a set of structured criteria were developed that yielded interpretive results similar to those of the panel (exact agreement = .94). In Study 3, the 3 predoctoral interns from Study 1 were trained to use the structured criteria, and the mean interrater agreement coefficient increased to .81. The results suggest that (a) the interpretation of functional analysis data may be less reliable than is generally assumed, (b) decision-making rules used by experts in the interpretation of functional analysis data can be operationalized, and (c) individuals can be trained to apply these rules accurately to increase interrater agreement. Potential uses of the criteria are discussed.

Original languageEnglish (US)
Pages (from-to)313-326
Number of pages14
JournalJournal of applied behavior analysis
Volume30
Issue number2
DOIs
StatePublished - Jan 1 1997

Fingerprint

functional analysis
interpretation
expert
Reproducibility of Results
Consensus
Decision Making
Functional Analysis
decision making
Interrater Agreement

Keywords

  • Assessment
  • Functional analysis
  • Interrater agreement
  • Visual inspection

ASJC Scopus subject areas

  • Applied Psychology
  • Sociology and Political Science
  • Philosophy

Cite this

Toward the development of structured criteria for interpretation of functional analysis data. / Hagopian, Louis P.; Fisher, Wayne W; Thompson, Rachel H.; Owen-Deschryver, Jamie; Iwata, Brian A.; Wacker, David P.

In: Journal of applied behavior analysis, Vol. 30, No. 2, 01.01.1997, p. 313-326.

Research output: Contribution to journalArticle

Hagopian, Louis P. ; Fisher, Wayne W ; Thompson, Rachel H. ; Owen-Deschryver, Jamie ; Iwata, Brian A. ; Wacker, David P. / Toward the development of structured criteria for interpretation of functional analysis data. In: Journal of applied behavior analysis. 1997 ; Vol. 30, No. 2. pp. 313-326.
@article{79e39196f9f14facb9bae90b10bfdc1a,
title = "Toward the development of structured criteria for interpretation of functional analysis data",
abstract = "Using functional analysis results to prescribe treatments is the preferred method for developing behavioral interventions. Little is known, however, about the reliability and validity of visual inspection for the interpretation of functional analysis data. The purpose of this investigation was to develop a set of structured criteria for visual inspection of multielement functional analyses that, when applied correctly, would increase interrater agreement and agreement with interpretations reached by expert consensus. In Study 1, 3 predoctoral interns interpreted functional analysis graphs, and interrater agreement was low (M = .46). In Study 2, 64 functional analysis graphs were interpreted by a panel of experts, and then a set of structured criteria were developed that yielded interpretive results similar to those of the panel (exact agreement = .94). In Study 3, the 3 predoctoral interns from Study 1 were trained to use the structured criteria, and the mean interrater agreement coefficient increased to .81. The results suggest that (a) the interpretation of functional analysis data may be less reliable than is generally assumed, (b) decision-making rules used by experts in the interpretation of functional analysis data can be operationalized, and (c) individuals can be trained to apply these rules accurately to increase interrater agreement. Potential uses of the criteria are discussed.",
keywords = "Assessment, Functional analysis, Interrater agreement, Visual inspection",
author = "Hagopian, {Louis P.} and Fisher, {Wayne W} and Thompson, {Rachel H.} and Jamie Owen-Deschryver and Iwata, {Brian A.} and Wacker, {David P.}",
year = "1997",
month = "1",
day = "1",
doi = "10.1901/jaba.1997.30-313",
language = "English (US)",
volume = "30",
pages = "313--326",
journal = "Journal of Applied Behavior Analysis",
issn = "0021-8855",
publisher = "Society for the Experimental Analysis of Behavior Inc.",
number = "2",

}

TY - JOUR

T1 - Toward the development of structured criteria for interpretation of functional analysis data

AU - Hagopian, Louis P.

AU - Fisher, Wayne W

AU - Thompson, Rachel H.

AU - Owen-Deschryver, Jamie

AU - Iwata, Brian A.

AU - Wacker, David P.

PY - 1997/1/1

Y1 - 1997/1/1

N2 - Using functional analysis results to prescribe treatments is the preferred method for developing behavioral interventions. Little is known, however, about the reliability and validity of visual inspection for the interpretation of functional analysis data. The purpose of this investigation was to develop a set of structured criteria for visual inspection of multielement functional analyses that, when applied correctly, would increase interrater agreement and agreement with interpretations reached by expert consensus. In Study 1, 3 predoctoral interns interpreted functional analysis graphs, and interrater agreement was low (M = .46). In Study 2, 64 functional analysis graphs were interpreted by a panel of experts, and then a set of structured criteria were developed that yielded interpretive results similar to those of the panel (exact agreement = .94). In Study 3, the 3 predoctoral interns from Study 1 were trained to use the structured criteria, and the mean interrater agreement coefficient increased to .81. The results suggest that (a) the interpretation of functional analysis data may be less reliable than is generally assumed, (b) decision-making rules used by experts in the interpretation of functional analysis data can be operationalized, and (c) individuals can be trained to apply these rules accurately to increase interrater agreement. Potential uses of the criteria are discussed.

AB - Using functional analysis results to prescribe treatments is the preferred method for developing behavioral interventions. Little is known, however, about the reliability and validity of visual inspection for the interpretation of functional analysis data. The purpose of this investigation was to develop a set of structured criteria for visual inspection of multielement functional analyses that, when applied correctly, would increase interrater agreement and agreement with interpretations reached by expert consensus. In Study 1, 3 predoctoral interns interpreted functional analysis graphs, and interrater agreement was low (M = .46). In Study 2, 64 functional analysis graphs were interpreted by a panel of experts, and then a set of structured criteria were developed that yielded interpretive results similar to those of the panel (exact agreement = .94). In Study 3, the 3 predoctoral interns from Study 1 were trained to use the structured criteria, and the mean interrater agreement coefficient increased to .81. The results suggest that (a) the interpretation of functional analysis data may be less reliable than is generally assumed, (b) decision-making rules used by experts in the interpretation of functional analysis data can be operationalized, and (c) individuals can be trained to apply these rules accurately to increase interrater agreement. Potential uses of the criteria are discussed.

KW - Assessment

KW - Functional analysis

KW - Interrater agreement

KW - Visual inspection

UR - http://www.scopus.com/inward/record.url?scp=0031160711&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0031160711&partnerID=8YFLogxK

U2 - 10.1901/jaba.1997.30-313

DO - 10.1901/jaba.1997.30-313

M3 - Article

VL - 30

SP - 313

EP - 326

JO - Journal of Applied Behavior Analysis

JF - Journal of Applied Behavior Analysis

SN - 0021-8855

IS - 2

ER -