### Abstract

For a given source distribution, we establish properties of the conditional density achieving the rate distortion function lower bound as the distortion parameter varies. In the limit as the distortion tolerated goes to zero, the conditional density achieving the rate distortion function lower bound becomes degenerate in the sense that the channel it defines becomes error-free. As the permitted distortion increases to its limit, the conditional density achieving the rate distortion function lower bound defines a channel which no longer depends on the source distribution. In addition to the data compression motivation, we establish two results-one asymptotic, one nonasymptotic-showing that the the conditional densities achieving the rate distortion function lower bound make relatively weak assumptions on the dependence between the source and its representation. This corresponds, in Bayes estimation, to choosing a likelihood which makes relatively weak assumptions on the data generating mechanism if the source is regarded as a prior. Taken together, these results suggest one can use the conditional density obtained from the rate distortion function in data analysis. That is, when it is impossible to identify a "true" parametric family on the basis of physical modeling, our results provide both data compression and channel coding justification for using the conditional density achieving the rate distortion function lower bound as a likelihood.

Original language | English (US) |
---|---|

Pages (from-to) | 562-571 |

Number of pages | 10 |

Journal | IEEE Transactions on Information Theory |

Volume | 45 |

Issue number | 2 |

DOIs | |

State | Published - Dec 1 1999 |

### Fingerprint

### Keywords

- Likelihood selection
- Mutual information
- Rate distortion

### ASJC Scopus subject areas

- Information Systems
- Computer Science Applications
- Library and Information Sciences

### Cite this

*IEEE Transactions on Information Theory*,

*45*(2), 562-571. https://doi.org/10.1109/18.749003

**An information criterion for likelihood selection.** / Yuan, A.; Clarke, B.

Research output: Contribution to journal › Article

*IEEE Transactions on Information Theory*, vol. 45, no. 2, pp. 562-571. https://doi.org/10.1109/18.749003

}

TY - JOUR

T1 - An information criterion for likelihood selection

AU - Yuan, A.

AU - Clarke, B.

PY - 1999/12/1

Y1 - 1999/12/1

N2 - For a given source distribution, we establish properties of the conditional density achieving the rate distortion function lower bound as the distortion parameter varies. In the limit as the distortion tolerated goes to zero, the conditional density achieving the rate distortion function lower bound becomes degenerate in the sense that the channel it defines becomes error-free. As the permitted distortion increases to its limit, the conditional density achieving the rate distortion function lower bound defines a channel which no longer depends on the source distribution. In addition to the data compression motivation, we establish two results-one asymptotic, one nonasymptotic-showing that the the conditional densities achieving the rate distortion function lower bound make relatively weak assumptions on the dependence between the source and its representation. This corresponds, in Bayes estimation, to choosing a likelihood which makes relatively weak assumptions on the data generating mechanism if the source is regarded as a prior. Taken together, these results suggest one can use the conditional density obtained from the rate distortion function in data analysis. That is, when it is impossible to identify a "true" parametric family on the basis of physical modeling, our results provide both data compression and channel coding justification for using the conditional density achieving the rate distortion function lower bound as a likelihood.

AB - For a given source distribution, we establish properties of the conditional density achieving the rate distortion function lower bound as the distortion parameter varies. In the limit as the distortion tolerated goes to zero, the conditional density achieving the rate distortion function lower bound becomes degenerate in the sense that the channel it defines becomes error-free. As the permitted distortion increases to its limit, the conditional density achieving the rate distortion function lower bound defines a channel which no longer depends on the source distribution. In addition to the data compression motivation, we establish two results-one asymptotic, one nonasymptotic-showing that the the conditional densities achieving the rate distortion function lower bound make relatively weak assumptions on the dependence between the source and its representation. This corresponds, in Bayes estimation, to choosing a likelihood which makes relatively weak assumptions on the data generating mechanism if the source is regarded as a prior. Taken together, these results suggest one can use the conditional density obtained from the rate distortion function in data analysis. That is, when it is impossible to identify a "true" parametric family on the basis of physical modeling, our results provide both data compression and channel coding justification for using the conditional density achieving the rate distortion function lower bound as a likelihood.

KW - Likelihood selection

KW - Mutual information

KW - Rate distortion

UR - http://www.scopus.com/inward/record.url?scp=0033098617&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0033098617&partnerID=8YFLogxK

U2 - 10.1109/18.749003

DO - 10.1109/18.749003

M3 - Article

AN - SCOPUS:0033098617

VL - 45

SP - 562

EP - 571

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

SN - 0018-9448

IS - 2

ER -