Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction

Nicholas G. Polson, James G. Scott, Bertrand Clarke, C. Severinski

Research output: Chapter in Book/Report/Conference proceedingChapter

10 Citations (Scopus)

Abstract

We study the classic problem of choosing a prior distribution for a location parameter β = (β 1,..., β p) as p grows large. First, we study the standard "global-local shrinkage" approach, based on scale mixtures of normals. Two theorems are presented which characterize certain desirable properties of shrinkage priors for sparse problems. Next, we review some recent results showing how Lévy processes can be used to generate infinite-dimensional versions of standard normal scale-mixture priors, along with new priors that have yet to be seriously studied in the literature. This approach provides an intuitive framework both for generating new regularization penalties and shrinkage rules, and for performing asymptotic analysis on existing models.

Original languageEnglish (US)
Title of host publicationBayesian Statistics 9
PublisherOxford University Press
Volume9780199694587
ISBN (Electronic)9780191731921
ISBN (Print)9780199694587
DOIs
StatePublished - Jan 19 2012

Fingerprint

Shrinkage
Scale Mixture
Regularization
Prediction
Normal Mixture
Location Parameter
Prior distribution
Asymptotic Analysis
Penalty
Intuitive
Theorem
Standards
Model

Keywords

  • Lévy processes
  • Shrinkage
  • Sparsity

ASJC Scopus subject areas

  • Mathematics(all)

Cite this

Polson, N. G., Scott, J. G., Clarke, B., & Severinski, C. (2012). Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction. In Bayesian Statistics 9 (Vol. 9780199694587). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199694587.003.0017

Shrink Globally, Act Locally : Sparse Bayesian Regularization and Prediction. / Polson, Nicholas G.; Scott, James G.; Clarke, Bertrand; Severinski, C.

Bayesian Statistics 9. Vol. 9780199694587 Oxford University Press, 2012.

Research output: Chapter in Book/Report/Conference proceedingChapter

Polson, NG, Scott, JG, Clarke, B & Severinski, C 2012, Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction. in Bayesian Statistics 9. vol. 9780199694587, Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199694587.003.0017
Polson NG, Scott JG, Clarke B, Severinski C. Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction. In Bayesian Statistics 9. Vol. 9780199694587. Oxford University Press. 2012 https://doi.org/10.1093/acprof:oso/9780199694587.003.0017
Polson, Nicholas G. ; Scott, James G. ; Clarke, Bertrand ; Severinski, C. / Shrink Globally, Act Locally : Sparse Bayesian Regularization and Prediction. Bayesian Statistics 9. Vol. 9780199694587 Oxford University Press, 2012.
@inbook{9385f4ed2e2a47f3993c77b19b9fa53b,
title = "Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction",
abstract = "We study the classic problem of choosing a prior distribution for a location parameter β = (β 1,..., β p) as p grows large. First, we study the standard {"}global-local shrinkage{"} approach, based on scale mixtures of normals. Two theorems are presented which characterize certain desirable properties of shrinkage priors for sparse problems. Next, we review some recent results showing how L{\'e}vy processes can be used to generate infinite-dimensional versions of standard normal scale-mixture priors, along with new priors that have yet to be seriously studied in the literature. This approach provides an intuitive framework both for generating new regularization penalties and shrinkage rules, and for performing asymptotic analysis on existing models.",
keywords = "L{\'e}vy processes, Shrinkage, Sparsity",
author = "Polson, {Nicholas G.} and Scott, {James G.} and Bertrand Clarke and C. Severinski",
year = "2012",
month = "1",
day = "19",
doi = "10.1093/acprof:oso/9780199694587.003.0017",
language = "English (US)",
isbn = "9780199694587",
volume = "9780199694587",
booktitle = "Bayesian Statistics 9",
publisher = "Oxford University Press",
address = "United States",

}

TY - CHAP

T1 - Shrink Globally, Act Locally

T2 - Sparse Bayesian Regularization and Prediction

AU - Polson, Nicholas G.

AU - Scott, James G.

AU - Clarke, Bertrand

AU - Severinski, C.

PY - 2012/1/19

Y1 - 2012/1/19

N2 - We study the classic problem of choosing a prior distribution for a location parameter β = (β 1,..., β p) as p grows large. First, we study the standard "global-local shrinkage" approach, based on scale mixtures of normals. Two theorems are presented which characterize certain desirable properties of shrinkage priors for sparse problems. Next, we review some recent results showing how Lévy processes can be used to generate infinite-dimensional versions of standard normal scale-mixture priors, along with new priors that have yet to be seriously studied in the literature. This approach provides an intuitive framework both for generating new regularization penalties and shrinkage rules, and for performing asymptotic analysis on existing models.

AB - We study the classic problem of choosing a prior distribution for a location parameter β = (β 1,..., β p) as p grows large. First, we study the standard "global-local shrinkage" approach, based on scale mixtures of normals. Two theorems are presented which characterize certain desirable properties of shrinkage priors for sparse problems. Next, we review some recent results showing how Lévy processes can be used to generate infinite-dimensional versions of standard normal scale-mixture priors, along with new priors that have yet to be seriously studied in the literature. This approach provides an intuitive framework both for generating new regularization penalties and shrinkage rules, and for performing asymptotic analysis on existing models.

KW - Lévy processes

KW - Shrinkage

KW - Sparsity

UR - http://www.scopus.com/inward/record.url?scp=84933474694&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84933474694&partnerID=8YFLogxK

U2 - 10.1093/acprof:oso/9780199694587.003.0017

DO - 10.1093/acprof:oso/9780199694587.003.0017

M3 - Chapter

AN - SCOPUS:84933474694

SN - 9780199694587

VL - 9780199694587

BT - Bayesian Statistics 9

PB - Oxford University Press

ER -