The effects of nonnormality on the analysis of supersaturated designs: A comparison of stepwise, SCAD and permutation test methods

Woon Yuen Koh, Kent M. Eskridge, Dong Wang

Research output: Contribution to journalArticle

Abstract

Supersaturated designs (SSDs) are useful in examining many factors with a restricted number of experimental units. Many analysis methods have been proposed to analyse data from SSDs, with some methods performing better than others when data are normally distributed. It is possible that data sets violate assumptions of standard analysis methods used to analyse data from SSDs, and to date the performance of these analysis methods have not been evaluated using nonnormally distributed data sets. We conducted a simulation study with normally and nonnormally distributed data sets to compare the identification rates, power and coverage of the true models using a permutation test, the stepwise procedure and the smoothly clipped absolute deviation (SCAD) method. Results showed that at the level of significance α = 0.01, the identification rates of the true models of the three methods were comparable; however at α = 0.05, both the permutation test and stepwise procedures had considerably lower identification rates than SCAD. For most cases, the three methods produced high power and coverage. The experimentwise error rates (EER) were close to the nominal level (11.36%) for the stepwise method, while they were somewhat higher for the permutation test. The EER for the SCAD method were extremely high (84–87%) for the normal and t-distributions, as well as for data with outlier.

Original languageEnglish (US)
Article numberA011
Pages (from-to)158-166
Number of pages9
JournalJournal of Statistical Computation and Simulation
Volume83
Issue number1
DOIs
StatePublished - Jan 1 2013

Fingerprint

Supersaturated Design
Permutation Test
Non-normality
Deviation
Identification (control systems)
Error Rate
Coverage
Test methods
t-distribution
Violate
High Power
Outlier
Categorical or nominal
Gaussian distribution
Simulation Study

Keywords

  • Factor screening
  • Least squares
  • Nonparametric method
  • Screening design

ASJC Scopus subject areas

  • Statistics and Probability
  • Modeling and Simulation
  • Statistics, Probability and Uncertainty
  • Applied Mathematics

Cite this

The effects of nonnormality on the analysis of supersaturated designs : A comparison of stepwise, SCAD and permutation test methods. / Koh, Woon Yuen; Eskridge, Kent M.; Wang, Dong.

In: Journal of Statistical Computation and Simulation, Vol. 83, No. 1, A011, 01.01.2013, p. 158-166.

Research output: Contribution to journalArticle

@article{2c84f642a6934a198e53060d484eb462,
title = "The effects of nonnormality on the analysis of supersaturated designs: A comparison of stepwise, SCAD and permutation test methods",
abstract = "Supersaturated designs (SSDs) are useful in examining many factors with a restricted number of experimental units. Many analysis methods have been proposed to analyse data from SSDs, with some methods performing better than others when data are normally distributed. It is possible that data sets violate assumptions of standard analysis methods used to analyse data from SSDs, and to date the performance of these analysis methods have not been evaluated using nonnormally distributed data sets. We conducted a simulation study with normally and nonnormally distributed data sets to compare the identification rates, power and coverage of the true models using a permutation test, the stepwise procedure and the smoothly clipped absolute deviation (SCAD) method. Results showed that at the level of significance α = 0.01, the identification rates of the true models of the three methods were comparable; however at α = 0.05, both the permutation test and stepwise procedures had considerably lower identification rates than SCAD. For most cases, the three methods produced high power and coverage. The experimentwise error rates (EER) were close to the nominal level (11.36{\%}) for the stepwise method, while they were somewhat higher for the permutation test. The EER for the SCAD method were extremely high (84–87{\%}) for the normal and t-distributions, as well as for data with outlier.",
keywords = "Factor screening, Least squares, Nonparametric method, Screening design",
author = "Koh, {Woon Yuen} and Eskridge, {Kent M.} and Dong Wang",
year = "2013",
month = "1",
day = "1",
doi = "10.1080/00949655.2011.621953",
language = "English (US)",
volume = "83",
pages = "158--166",
journal = "Journal of Statistical Computation and Simulation",
issn = "0094-9655",
publisher = "Taylor and Francis Ltd.",
number = "1",

}

TY - JOUR

T1 - The effects of nonnormality on the analysis of supersaturated designs

T2 - A comparison of stepwise, SCAD and permutation test methods

AU - Koh, Woon Yuen

AU - Eskridge, Kent M.

AU - Wang, Dong

PY - 2013/1/1

Y1 - 2013/1/1

N2 - Supersaturated designs (SSDs) are useful in examining many factors with a restricted number of experimental units. Many analysis methods have been proposed to analyse data from SSDs, with some methods performing better than others when data are normally distributed. It is possible that data sets violate assumptions of standard analysis methods used to analyse data from SSDs, and to date the performance of these analysis methods have not been evaluated using nonnormally distributed data sets. We conducted a simulation study with normally and nonnormally distributed data sets to compare the identification rates, power and coverage of the true models using a permutation test, the stepwise procedure and the smoothly clipped absolute deviation (SCAD) method. Results showed that at the level of significance α = 0.01, the identification rates of the true models of the three methods were comparable; however at α = 0.05, both the permutation test and stepwise procedures had considerably lower identification rates than SCAD. For most cases, the three methods produced high power and coverage. The experimentwise error rates (EER) were close to the nominal level (11.36%) for the stepwise method, while they were somewhat higher for the permutation test. The EER for the SCAD method were extremely high (84–87%) for the normal and t-distributions, as well as for data with outlier.

AB - Supersaturated designs (SSDs) are useful in examining many factors with a restricted number of experimental units. Many analysis methods have been proposed to analyse data from SSDs, with some methods performing better than others when data are normally distributed. It is possible that data sets violate assumptions of standard analysis methods used to analyse data from SSDs, and to date the performance of these analysis methods have not been evaluated using nonnormally distributed data sets. We conducted a simulation study with normally and nonnormally distributed data sets to compare the identification rates, power and coverage of the true models using a permutation test, the stepwise procedure and the smoothly clipped absolute deviation (SCAD) method. Results showed that at the level of significance α = 0.01, the identification rates of the true models of the three methods were comparable; however at α = 0.05, both the permutation test and stepwise procedures had considerably lower identification rates than SCAD. For most cases, the three methods produced high power and coverage. The experimentwise error rates (EER) were close to the nominal level (11.36%) for the stepwise method, while they were somewhat higher for the permutation test. The EER for the SCAD method were extremely high (84–87%) for the normal and t-distributions, as well as for data with outlier.

KW - Factor screening

KW - Least squares

KW - Nonparametric method

KW - Screening design

UR - http://www.scopus.com/inward/record.url?scp=84946070683&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84946070683&partnerID=8YFLogxK

U2 - 10.1080/00949655.2011.621953

DO - 10.1080/00949655.2011.621953

M3 - Article

AN - SCOPUS:84946070683

VL - 83

SP - 158

EP - 166

JO - Journal of Statistical Computation and Simulation

JF - Journal of Statistical Computation and Simulation

SN - 0094-9655

IS - 1

M1 - A011

ER -