A pseudo-nearest-neighbor approach for missing data recovery on Gaussian random data sets

Xiaolu Huang, Qiuming Zhu

Research output: Contribution to journalArticle

45 Citations (Scopus)

Abstract

Missing data handling is an important preparation step for most data discrimination or mining tasks. Inappropriate treatment of missing data may cause large errors or false results. In this paper, we study the effect of a missing data recovery method, namely the pseudo-nearest-neighbor substitution approach, on Gaussian distributed data sets that represent typical cases in data discrimination and data mining applications. The error rate of the proposed recovery method is evaluated by comparing the clustering results of the recovered data sets to the clustering results obtained on the originally complete data sets. The results are also compared with that obtained by applying two other missing data handling methods, the constant default value substitution and the missing data ignorance (non-substitution) methods. The experiment results provided a valuable insight to the improvement of the accuracy for data discrimination and knowledge discovery on large data sets containing missing values.

Original languageEnglish (US)
Pages (from-to)1613-1622
Number of pages10
JournalPattern Recognition Letters
Volume23
Issue number13
DOIs
StatePublished - Nov 1 2002

Fingerprint

Data handling
Data mining
Substitution reactions
Recovery
Experiments

Keywords

  • Data clustering
  • Data imputation
  • Data mining
  • Gaussian data distribution
  • Missing data
  • Missing data recovery

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

A pseudo-nearest-neighbor approach for missing data recovery on Gaussian random data sets. / Huang, Xiaolu; Zhu, Qiuming.

In: Pattern Recognition Letters, Vol. 23, No. 13, 01.11.2002, p. 1613-1622.

Research output: Contribution to journalArticle

@article{5eaabab84d0749f0afa70497097196a6,
title = "A pseudo-nearest-neighbor approach for missing data recovery on Gaussian random data sets",
abstract = "Missing data handling is an important preparation step for most data discrimination or mining tasks. Inappropriate treatment of missing data may cause large errors or false results. In this paper, we study the effect of a missing data recovery method, namely the pseudo-nearest-neighbor substitution approach, on Gaussian distributed data sets that represent typical cases in data discrimination and data mining applications. The error rate of the proposed recovery method is evaluated by comparing the clustering results of the recovered data sets to the clustering results obtained on the originally complete data sets. The results are also compared with that obtained by applying two other missing data handling methods, the constant default value substitution and the missing data ignorance (non-substitution) methods. The experiment results provided a valuable insight to the improvement of the accuracy for data discrimination and knowledge discovery on large data sets containing missing values.",
keywords = "Data clustering, Data imputation, Data mining, Gaussian data distribution, Missing data, Missing data recovery",
author = "Xiaolu Huang and Qiuming Zhu",
year = "2002",
month = "11",
day = "1",
doi = "10.1016/S0167-8655(02)00125-3",
language = "English (US)",
volume = "23",
pages = "1613--1622",
journal = "Pattern Recognition Letters",
issn = "0167-8655",
publisher = "Elsevier",
number = "13",

}

TY - JOUR

T1 - A pseudo-nearest-neighbor approach for missing data recovery on Gaussian random data sets

AU - Huang, Xiaolu

AU - Zhu, Qiuming

PY - 2002/11/1

Y1 - 2002/11/1

N2 - Missing data handling is an important preparation step for most data discrimination or mining tasks. Inappropriate treatment of missing data may cause large errors or false results. In this paper, we study the effect of a missing data recovery method, namely the pseudo-nearest-neighbor substitution approach, on Gaussian distributed data sets that represent typical cases in data discrimination and data mining applications. The error rate of the proposed recovery method is evaluated by comparing the clustering results of the recovered data sets to the clustering results obtained on the originally complete data sets. The results are also compared with that obtained by applying two other missing data handling methods, the constant default value substitution and the missing data ignorance (non-substitution) methods. The experiment results provided a valuable insight to the improvement of the accuracy for data discrimination and knowledge discovery on large data sets containing missing values.

AB - Missing data handling is an important preparation step for most data discrimination or mining tasks. Inappropriate treatment of missing data may cause large errors or false results. In this paper, we study the effect of a missing data recovery method, namely the pseudo-nearest-neighbor substitution approach, on Gaussian distributed data sets that represent typical cases in data discrimination and data mining applications. The error rate of the proposed recovery method is evaluated by comparing the clustering results of the recovered data sets to the clustering results obtained on the originally complete data sets. The results are also compared with that obtained by applying two other missing data handling methods, the constant default value substitution and the missing data ignorance (non-substitution) methods. The experiment results provided a valuable insight to the improvement of the accuracy for data discrimination and knowledge discovery on large data sets containing missing values.

KW - Data clustering

KW - Data imputation

KW - Data mining

KW - Gaussian data distribution

KW - Missing data

KW - Missing data recovery

UR - http://www.scopus.com/inward/record.url?scp=0036832985&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036832985&partnerID=8YFLogxK

U2 - 10.1016/S0167-8655(02)00125-3

DO - 10.1016/S0167-8655(02)00125-3

M3 - Article

VL - 23

SP - 1613

EP - 1622

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

SN - 0167-8655

IS - 13

ER -