Random subspace method for multivariate feature selection

@article{lai06random, abstract = {In a growing number of domains data captured encapsulates as many features as possible. This poses a challenge to classical pattern recognition techniques, since the number of samples often still is limited with respect to the number of features. Classical pattern recognition methods suffer from the small sample size, and robust classification techniques are needed. In order to reduce the dimensionality of the feature space, the selection of informative features becomes an essential step towards the classification task. The relevance of the features can be evaluated either individually (univariate approaches), or in a multivariate manner. Univariate approaches are simple and fast, therefore appealing. However, possible correlation and dependencies between the features are not considered. Therefore, multivariate search techniques may be helpful. Several limitations restrict the use of multivariate searches. First, they are prone to overtraining, especially in p >> n (many features and few samples) settings. Secondly, they can be computationally too expensive when dealing with a large feature space. We introduce a new multivariate search technique, that is less sensitive to the noise in the data and computationally feasible as well. We compare our approach with several multivariate and univariate feature selection techniques, on an artificial dataset which provides us with ground truth information, and on a real dataset. The results show the importance of multivariate search techniques and the robustness and reliability of our novel multivariate feature selection method.}, author = {Lai, Carmen and Reinders, Marcel J. and Wessels, Lodewyk }, citeulike-article-id = {3017356}, doi = {http://dx.doi.org/10.1016/j.patrec.2005.12.018}, journal = {Pattern Recognition Letters}, keywords = {boosting, feature-ranking, feature-selection, high-dimensions, random-subspace-method, small-sample-size-problem, weighting-feature-selection}, month = {July}, number = {10}, pages = {1067--1076}, posted-at = {2008-07-18 13:43:26}, priority = {4}, title = {Random subspace method for multivariate feature selection}, url = {http://dx.doi.org/10.1016/j.patrec.2005.12.018}, volume = {27}, year = {2006} }

TY - JOUR ID - lai06random L3 - citeulike-article-id:3017356 TI - Random subspace method for multivariate feature selection JF - Pattern Recognition Letters VL - 27 IS - 10 SP - 1067 EP - 1076 N2 - In a growing number of domains data captured encapsulates as many features as possible. This poses a challenge to classical pattern recognition techniques, since the number of samples often still is limited with respect to the number of features. Classical pattern recognition methods suffer from the small sample size, and robust classification techniques are needed. In order to reduce the dimensionality of the feature space, the selection of informative features becomes an essential step towards the classification task. The relevance of the features can be evaluated either individually (univariate approaches), or in a multivariate manner. Univariate approaches are simple and fast, therefore appealing. However, possible correlation and dependencies between the features are not considered. Therefore, multivariate search techniques may be helpful. Several limitations restrict the use of multivariate searches. First, they are prone to overtraining, especially in p >> n (many features and few samples) settings. Secondly, they can be computationally too expensive when dealing with a large feature space. We introduce a new multivariate search technique, that is less sensitive to the noise in the data and computationally feasible as well. We compare our approach with several multivariate and univariate feature selection techniques, on an artificial dataset which provides us with ground truth information, and on a real dataset. The results show the importance of multivariate search techniques and the robustness and reliability of our novel multivariate feature selection method. KW - boosting KW - feature-ranking KW - feature-selection KW - high-dimensions KW - random-subspace-method KW - small-sample-size-problem KW - weighting-feature-selection AU - Lai, Carmen AU - Reinders, Marcel AU - Wessels, Lodewyk PY - 2006/07/15/ UR - http://dx.doi.org/10.1016/j.patrec.2005.12.018 ER -