Validation of visual statistical inference, applied to linear models

Mahbubul Majumder, Heike Hofmann, Dianne Cook

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

Statistical graphics play a crucial role in exploratory data analysis, model checking, and diagnosis. The lineup protocol enables statistical significance testing of visual findings, bridging the gulf between exploratory and inferential statistics. In this article, inferential methods for statistical graphics are developed further by refining the terminology of visual inference and framing the lineup protocol in a context that allows direct comparison with conventional tests in scenarios when a conventional test exists. This framework is used to compare the performance of the lineup protocol against conventional statistical testing in the scenario of fitting linear models.Ahuman subjects experiment is conducted using simulated data to provide controlled conditions. Results suggest that the lineup protocol performs comparably with the conventional tests, and expectedly outperforms them when data are contaminated, a scenario where assumptions required for performing a conventional test are violated. Surprisingly, visual tests have higher power than the conventional tests when the effect size is large. And, interestingly, there may be some super-visual individuals who yield better performance and power than the conventional test even in the most difficult tasks. Supplementary materials for this article are available online.

Original languageEnglish (US)
Pages (from-to)942-956
Number of pages15
JournalJournal of the American Statistical Association
Volume108
Issue number503
DOIs
StatePublished - Dec 16 2013

Fingerprint

Statistical Inference
Linear Model
Statistical Graphics
Scenarios
Exploratory Data Analysis
Testing
Effect Size
Statistical Significance
Vision
Statistical inference
High Power
Model Checking
Statistics
Experiment

Keywords

  • Data mining
  • Effect size
  • Exploratory data analysis
  • Lineup
  • Nonparametric test
  • Practical significance
  • Statistical graphics
  • Visualization

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Validation of visual statistical inference, applied to linear models. / Majumder, Mahbubul; Hofmann, Heike; Cook, Dianne.

In: Journal of the American Statistical Association, Vol. 108, No. 503, 16.12.2013, p. 942-956.

Research output: Contribution to journalArticle

@article{27b9440d616945aca63d420892254c58,
title = "Validation of visual statistical inference, applied to linear models",
abstract = "Statistical graphics play a crucial role in exploratory data analysis, model checking, and diagnosis. The lineup protocol enables statistical significance testing of visual findings, bridging the gulf between exploratory and inferential statistics. In this article, inferential methods for statistical graphics are developed further by refining the terminology of visual inference and framing the lineup protocol in a context that allows direct comparison with conventional tests in scenarios when a conventional test exists. This framework is used to compare the performance of the lineup protocol against conventional statistical testing in the scenario of fitting linear models.Ahuman subjects experiment is conducted using simulated data to provide controlled conditions. Results suggest that the lineup protocol performs comparably with the conventional tests, and expectedly outperforms them when data are contaminated, a scenario where assumptions required for performing a conventional test are violated. Surprisingly, visual tests have higher power than the conventional tests when the effect size is large. And, interestingly, there may be some super-visual individuals who yield better performance and power than the conventional test even in the most difficult tasks. Supplementary materials for this article are available online.",
keywords = "Data mining, Effect size, Exploratory data analysis, Lineup, Nonparametric test, Practical significance, Statistical graphics, Visualization",
author = "Mahbubul Majumder and Heike Hofmann and Dianne Cook",
year = "2013",
month = "12",
day = "16",
doi = "10.1080/01621459.2013.808157",
language = "English (US)",
volume = "108",
pages = "942--956",
journal = "Journal of the American Statistical Association",
issn = "0162-1459",
publisher = "Taylor and Francis Ltd.",
number = "503",

}

TY - JOUR

T1 - Validation of visual statistical inference, applied to linear models

AU - Majumder, Mahbubul

AU - Hofmann, Heike

AU - Cook, Dianne

PY - 2013/12/16

Y1 - 2013/12/16

N2 - Statistical graphics play a crucial role in exploratory data analysis, model checking, and diagnosis. The lineup protocol enables statistical significance testing of visual findings, bridging the gulf between exploratory and inferential statistics. In this article, inferential methods for statistical graphics are developed further by refining the terminology of visual inference and framing the lineup protocol in a context that allows direct comparison with conventional tests in scenarios when a conventional test exists. This framework is used to compare the performance of the lineup protocol against conventional statistical testing in the scenario of fitting linear models.Ahuman subjects experiment is conducted using simulated data to provide controlled conditions. Results suggest that the lineup protocol performs comparably with the conventional tests, and expectedly outperforms them when data are contaminated, a scenario where assumptions required for performing a conventional test are violated. Surprisingly, visual tests have higher power than the conventional tests when the effect size is large. And, interestingly, there may be some super-visual individuals who yield better performance and power than the conventional test even in the most difficult tasks. Supplementary materials for this article are available online.

AB - Statistical graphics play a crucial role in exploratory data analysis, model checking, and diagnosis. The lineup protocol enables statistical significance testing of visual findings, bridging the gulf between exploratory and inferential statistics. In this article, inferential methods for statistical graphics are developed further by refining the terminology of visual inference and framing the lineup protocol in a context that allows direct comparison with conventional tests in scenarios when a conventional test exists. This framework is used to compare the performance of the lineup protocol against conventional statistical testing in the scenario of fitting linear models.Ahuman subjects experiment is conducted using simulated data to provide controlled conditions. Results suggest that the lineup protocol performs comparably with the conventional tests, and expectedly outperforms them when data are contaminated, a scenario where assumptions required for performing a conventional test are violated. Surprisingly, visual tests have higher power than the conventional tests when the effect size is large. And, interestingly, there may be some super-visual individuals who yield better performance and power than the conventional test even in the most difficult tasks. Supplementary materials for this article are available online.

KW - Data mining

KW - Effect size

KW - Exploratory data analysis

KW - Lineup

KW - Nonparametric test

KW - Practical significance

KW - Statistical graphics

KW - Visualization

UR - http://www.scopus.com/inward/record.url?scp=84890085949&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84890085949&partnerID=8YFLogxK

U2 - 10.1080/01621459.2013.808157

DO - 10.1080/01621459.2013.808157

M3 - Article

AN - SCOPUS:84890085949

VL - 108

SP - 942

EP - 956

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

SN - 0162-1459

IS - 503

ER -