Prediction in M-complete problems with limited sample size

Jennifer Lynn Clarke, Bertrand Clarke, Chi Wai Yu

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

We define a new Bayesian predictor called the posterior weighted median (PWM) and compare its performance to several other predictors includ- ing the Bayes model average under squared error loss, the Barbieri-Berger me- dian model predictor, the stacking predictor, and the model average predictor based on Akaike's information criterion. We argue that PWM generally gives better performance than other predictors over a range of M-complete problems. This range is between the M-closed-M-complete boundary and the M-complete- M-open boundary. Indeed, as a problem gets closer to M-open, it seems that M-complete predictive methods begin to break down. Our comparisons rest on extensive simulations and real data examples. As a separate issue, we introduce the concepts of the 'Bail out effect' and the 'Bail in effect'. These occur when a predictor gives not just poor results but defaults to the simplest model ('bails out') or to the most complex model ('bails in') on the model list. Either can occur inM-complete problems when the complexity of the data generator is too high for the predictor scheme to accommodate.

Original languageEnglish (US)
Pages (from-to)647-690
Number of pages44
JournalBayesian Analysis
Volume8
Issue number3
DOIs
StatePublished - Oct 2 2013

Fingerprint

Predictors
Sample Size
Prediction
Model
Squared Error Loss
Akaike Information Criterion
Stacking
Bayes
Range of data
Breakdown
Generator
Closed
Simulation

Keywords

  • Basis selection
  • Ensemble methods
  • M-complete
  • Model list selection
  • Model selection
  • Prediction

ASJC Scopus subject areas

  • Statistics and Probability
  • Applied Mathematics

Cite this

Prediction in M-complete problems with limited sample size. / Clarke, Jennifer Lynn; Clarke, Bertrand; Yu, Chi Wai.

In: Bayesian Analysis, Vol. 8, No. 3, 02.10.2013, p. 647-690.

Research output: Contribution to journalArticle

Clarke, Jennifer Lynn ; Clarke, Bertrand ; Yu, Chi Wai. / Prediction in M-complete problems with limited sample size. In: Bayesian Analysis. 2013 ; Vol. 8, No. 3. pp. 647-690.
@article{aac444a6b26d4160a47ba10698465e32,
title = "Prediction in M-complete problems with limited sample size",
abstract = "We define a new Bayesian predictor called the posterior weighted median (PWM) and compare its performance to several other predictors includ- ing the Bayes model average under squared error loss, the Barbieri-Berger me- dian model predictor, the stacking predictor, and the model average predictor based on Akaike's information criterion. We argue that PWM generally gives better performance than other predictors over a range of M-complete problems. This range is between the M-closed-M-complete boundary and the M-complete- M-open boundary. Indeed, as a problem gets closer to M-open, it seems that M-complete predictive methods begin to break down. Our comparisons rest on extensive simulations and real data examples. As a separate issue, we introduce the concepts of the 'Bail out effect' and the 'Bail in effect'. These occur when a predictor gives not just poor results but defaults to the simplest model ('bails out') or to the most complex model ('bails in') on the model list. Either can occur inM-complete problems when the complexity of the data generator is too high for the predictor scheme to accommodate.",
keywords = "Basis selection, Ensemble methods, M-complete, Model list selection, Model selection, Prediction",
author = "Clarke, {Jennifer Lynn} and Bertrand Clarke and Yu, {Chi Wai}",
year = "2013",
month = "10",
day = "2",
doi = "10.1214/13-BA826",
language = "English (US)",
volume = "8",
pages = "647--690",
journal = "Bayesian Analysis",
issn = "1936-0975",
publisher = "Carnegie Mellon University",
number = "3",

}

TY - JOUR

T1 - Prediction in M-complete problems with limited sample size

AU - Clarke, Jennifer Lynn

AU - Clarke, Bertrand

AU - Yu, Chi Wai

PY - 2013/10/2

Y1 - 2013/10/2

N2 - We define a new Bayesian predictor called the posterior weighted median (PWM) and compare its performance to several other predictors includ- ing the Bayes model average under squared error loss, the Barbieri-Berger me- dian model predictor, the stacking predictor, and the model average predictor based on Akaike's information criterion. We argue that PWM generally gives better performance than other predictors over a range of M-complete problems. This range is between the M-closed-M-complete boundary and the M-complete- M-open boundary. Indeed, as a problem gets closer to M-open, it seems that M-complete predictive methods begin to break down. Our comparisons rest on extensive simulations and real data examples. As a separate issue, we introduce the concepts of the 'Bail out effect' and the 'Bail in effect'. These occur when a predictor gives not just poor results but defaults to the simplest model ('bails out') or to the most complex model ('bails in') on the model list. Either can occur inM-complete problems when the complexity of the data generator is too high for the predictor scheme to accommodate.

AB - We define a new Bayesian predictor called the posterior weighted median (PWM) and compare its performance to several other predictors includ- ing the Bayes model average under squared error loss, the Barbieri-Berger me- dian model predictor, the stacking predictor, and the model average predictor based on Akaike's information criterion. We argue that PWM generally gives better performance than other predictors over a range of M-complete problems. This range is between the M-closed-M-complete boundary and the M-complete- M-open boundary. Indeed, as a problem gets closer to M-open, it seems that M-complete predictive methods begin to break down. Our comparisons rest on extensive simulations and real data examples. As a separate issue, we introduce the concepts of the 'Bail out effect' and the 'Bail in effect'. These occur when a predictor gives not just poor results but defaults to the simplest model ('bails out') or to the most complex model ('bails in') on the model list. Either can occur inM-complete problems when the complexity of the data generator is too high for the predictor scheme to accommodate.

KW - Basis selection

KW - Ensemble methods

KW - M-complete

KW - Model list selection

KW - Model selection

KW - Prediction

UR - http://www.scopus.com/inward/record.url?scp=84884695160&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84884695160&partnerID=8YFLogxK

U2 - 10.1214/13-BA826

DO - 10.1214/13-BA826

M3 - Article

AN - SCOPUS:84884695160

VL - 8

SP - 647

EP - 690

JO - Bayesian Analysis

JF - Bayesian Analysis

SN - 1936-0975

IS - 3

ER -