Information tradeoff

L. Wasserman, Bertrand S Clarke

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

A prior may be noninformative for one parameter at the cost of being informative for another parameter. This leads to the idea of tradeoff priors: priors that give up noninformativity for some parameters to achieve noninformativity for others. We propose a general framework where priors are selected by optimizing a functional with two components. The first component formalizes the requirement that the optimal prior be noninformative for the parameter of interest. The second component is a penalty term that forces the optimizing prior to be close to some target prior. Optimizing such a functional results in a parameterized family of priors from which a specific prior may be selected as the tradeoff prior. An important particular example of such functionals is provided by choosing the first term to be the marginal missing information for the parameter of interest (generalizing Bernardo's notion of missing information) and the second term to be the relative entropy between the unknown prior and the Jeffreys prior. In this case we find a closed form expression for the tradeoff prior and we make explicit connections with the Berger-Bernardo prior. In particular, we show that under certain conditions, the Berger-Bernardo prior and the Jeffreys prior are special cases of the tradeoff prior. We consider several examples.

Original languageEnglish (US)
Pages (from-to)19-38
Number of pages20
JournalTest
Volume4
Issue number1
DOIs
StatePublished - Jun 1 1995

Fingerprint

Trade-offs
Jeffreys Prior
Term
Relative Entropy
Penalty
Closed-form
Unknown
Target
Requirements
Missing information

Keywords

  • Asymptotic Information
  • Noninformative Priors
  • Nuisance Parameters

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Wasserman, L., & Clarke, B. S. (1995). Information tradeoff. Test, 4(1), 19-38. https://doi.org/10.1007/BF02563101

Information tradeoff. / Wasserman, L.; Clarke, Bertrand S.

In: Test, Vol. 4, No. 1, 01.06.1995, p. 19-38.

Research output: Contribution to journalArticle

Wasserman, L & Clarke, BS 1995, 'Information tradeoff', Test, vol. 4, no. 1, pp. 19-38. https://doi.org/10.1007/BF02563101
Wasserman L, Clarke BS. Information tradeoff. Test. 1995 Jun 1;4(1):19-38. https://doi.org/10.1007/BF02563101
Wasserman, L. ; Clarke, Bertrand S. / Information tradeoff. In: Test. 1995 ; Vol. 4, No. 1. pp. 19-38.
@article{92263cc75de04deba34384c1112f0102,
title = "Information tradeoff",
abstract = "A prior may be noninformative for one parameter at the cost of being informative for another parameter. This leads to the idea of tradeoff priors: priors that give up noninformativity for some parameters to achieve noninformativity for others. We propose a general framework where priors are selected by optimizing a functional with two components. The first component formalizes the requirement that the optimal prior be noninformative for the parameter of interest. The second component is a penalty term that forces the optimizing prior to be close to some target prior. Optimizing such a functional results in a parameterized family of priors from which a specific prior may be selected as the tradeoff prior. An important particular example of such functionals is provided by choosing the first term to be the marginal missing information for the parameter of interest (generalizing Bernardo's notion of missing information) and the second term to be the relative entropy between the unknown prior and the Jeffreys prior. In this case we find a closed form expression for the tradeoff prior and we make explicit connections with the Berger-Bernardo prior. In particular, we show that under certain conditions, the Berger-Bernardo prior and the Jeffreys prior are special cases of the tradeoff prior. We consider several examples.",
keywords = "Asymptotic Information, Noninformative Priors, Nuisance Parameters",
author = "L. Wasserman and Clarke, {Bertrand S}",
year = "1995",
month = "6",
day = "1",
doi = "10.1007/BF02563101",
language = "English (US)",
volume = "4",
pages = "19--38",
journal = "Trabajos de Estadistica Y de Investigacion Operativa",
issn = "0041-0241",
publisher = "Springer New York",
number = "1",

}

TY - JOUR

T1 - Information tradeoff

AU - Wasserman, L.

AU - Clarke, Bertrand S

PY - 1995/6/1

Y1 - 1995/6/1

N2 - A prior may be noninformative for one parameter at the cost of being informative for another parameter. This leads to the idea of tradeoff priors: priors that give up noninformativity for some parameters to achieve noninformativity for others. We propose a general framework where priors are selected by optimizing a functional with two components. The first component formalizes the requirement that the optimal prior be noninformative for the parameter of interest. The second component is a penalty term that forces the optimizing prior to be close to some target prior. Optimizing such a functional results in a parameterized family of priors from which a specific prior may be selected as the tradeoff prior. An important particular example of such functionals is provided by choosing the first term to be the marginal missing information for the parameter of interest (generalizing Bernardo's notion of missing information) and the second term to be the relative entropy between the unknown prior and the Jeffreys prior. In this case we find a closed form expression for the tradeoff prior and we make explicit connections with the Berger-Bernardo prior. In particular, we show that under certain conditions, the Berger-Bernardo prior and the Jeffreys prior are special cases of the tradeoff prior. We consider several examples.

AB - A prior may be noninformative for one parameter at the cost of being informative for another parameter. This leads to the idea of tradeoff priors: priors that give up noninformativity for some parameters to achieve noninformativity for others. We propose a general framework where priors are selected by optimizing a functional with two components. The first component formalizes the requirement that the optimal prior be noninformative for the parameter of interest. The second component is a penalty term that forces the optimizing prior to be close to some target prior. Optimizing such a functional results in a parameterized family of priors from which a specific prior may be selected as the tradeoff prior. An important particular example of such functionals is provided by choosing the first term to be the marginal missing information for the parameter of interest (generalizing Bernardo's notion of missing information) and the second term to be the relative entropy between the unknown prior and the Jeffreys prior. In this case we find a closed form expression for the tradeoff prior and we make explicit connections with the Berger-Bernardo prior. In particular, we show that under certain conditions, the Berger-Bernardo prior and the Jeffreys prior are special cases of the tradeoff prior. We consider several examples.

KW - Asymptotic Information

KW - Noninformative Priors

KW - Nuisance Parameters

UR - http://www.scopus.com/inward/record.url?scp=51249164161&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=51249164161&partnerID=8YFLogxK

U2 - 10.1007/BF02563101

DO - 10.1007/BF02563101

M3 - Article

VL - 4

SP - 19

EP - 38

JO - Trabajos de Estadistica Y de Investigacion Operativa

JF - Trabajos de Estadistica Y de Investigacion Operativa

SN - 0041-0241

IS - 1

ER -