Kernel estimation in transect sampling without the shoulder condition

Y. P. Mack, Pham X. Quang, Shunpu Zhang

Research output: Contribution to journalArticle

10 Scopus citations

Abstract

We consider the estimation of wildlife population density based on line transect data. Nonparametric kernel method is employed, without the usual assumption that the detection curve has a shoulder at distance zero, with the help of a special class of kernels called boundary kernels. Asymptotic distribution results are included. It is pointed out that the boundary kernel of Zhang and Karunamuni (1998) (see also Müller and Wang (1994)) performs better (for asymptotic mean square error consideration) than that of the boundary kernel of Müller (1991). But both of these kernels are clearly superior to the half-normal and one-sided Epanechnikov kernel when the shoulder condition fails to hold. In practice, however, for small to moderate sample sizes, caution should be exercised in using boundary kernels in that the density estimate might become negative. A Monte Carlo study is also presented, comparing the performance of four kernels applied to detection data, with and without the shoulder condition. Two boundary kernels for derivatives arc also included for the point transect case.

Original languageEnglish (US)
Pages (from-to)2277-2296
Number of pages20
JournalCommunications in Statistics - Theory and Methods
Volume28
Issue number10
DOIs
Publication statusPublished - Jan 1 1999

    Fingerprint

ASJC Scopus subject areas

  • Statistics and Probability

Cite this