Creating diverse nearest-neighbour ensembles using simultaneous metaheuristic feature selection

Tahir, Muhammad and Smith, Jim (2010) Creating diverse nearest-neighbour ensembles using simultaneous metaheuristic feature selection. Pattern Recognition Letters, 31 (11). pp. 1470-1480. ISSN 0167-8655

Full text not available from this repository. (Request a copy)

Abstract

The nearest-neighbour (1NN) classifier has long been used in pattern recognition, exploratory data analysis, and data mining problems. A vital consideration in obtaining good results with this technique is the choice of distance function, and correspondingly which features to consider when computing distances between samples. In recent years there has been an increasing interest in creating ensembles of classifiers in order to improve classification accuracy. This paper proposes a new ensemble technique which combines multiple 1NN classifiers, each using a different distance function, and potentially a different set of features (feature vector).

These feature vectors are determined for each distance metric simultaneously using Tabu Search to minimise the ensemble error rate. We show that this approach implicitly selects for a diverse set of classifiers, and by doing so achieves greater performance improvements than can be achieved by treating the classifiers independently, or using a single feature set. Naturally, optimising the level of ensembles necessitates a much larger solution space, to make this approach tractable, we show how Tabu Search at the ensemble level can be hybridised with local search at the level of individual classifiers. The proposed ensemble classifier with different distance metrics and different feature vectors is evaluated using various benchmark datasets from UCI Machine Learning Repository and a real-world machine-vision application. Results have indicated a significant increase in the performance when compared with various well-known classifiers. Furthermore, the proposed ensemble method is also compared with ensemble classifier using different distance metrics but with same feature vector (with or without feature selection (FS)).

Item Type:	Article
Uncontrolled Keywords:	Tabu Search, 1NN classifier, ensemble classifiers
Subjects:	G900 Others in Mathematical and Computing Sciences
Department:	Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User:	Ay Okpokam
Date Deposited:	24 Jan 2012 16:48
Last Modified:	13 Oct 2019 00:30
URI:	http://nrl.northumbria.ac.uk/id/eprint/5092

Downloads