An information criterion for likelihood selection

A. Yuan, B. Clarke

Research output: Contribution to journalArticle

11 Scopus citations

Abstract

For a given source distribution, we establish properties of the conditional density achieving the rate distortion function lower bound as the distortion parameter varies. In the limit as the distortion tolerated goes to zero, the conditional density achieving the rate distortion function lower bound becomes degenerate in the sense that the channel it defines becomes error-free. As the permitted distortion increases to its limit, the conditional density achieving the rate distortion function lower bound defines a channel which no longer depends on the source distribution. In addition to the data compression motivation, we establish two results-one asymptotic, one nonasymptotic-showing that the the conditional densities achieving the rate distortion function lower bound make relatively weak assumptions on the dependence between the source and its representation. This corresponds, in Bayes estimation, to choosing a likelihood which makes relatively weak assumptions on the data generating mechanism if the source is regarded as a prior. Taken together, these results suggest one can use the conditional density obtained from the rate distortion function in data analysis. That is, when it is impossible to identify a "true" parametric family on the basis of physical modeling, our results provide both data compression and channel coding justification for using the conditional density achieving the rate distortion function lower bound as a likelihood.

Original languageEnglish (US)
Pages (from-to)562-571
Number of pages10
JournalIEEE Transactions on Information Theory
Volume45
Issue number2
DOIs
Publication statusPublished - Dec 1 1999

    Fingerprint

Keywords

  • Likelihood selection
  • Mutual information
  • Rate distortion

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this