The random subspace method for constructing decision forests

@article{ho98random, abstract = {Much of previous attention on decision trees focuses on the splitting criteria and optimization of tree sizes. The dilemma between overfitting and achieving maximum accuracy is seldom resolved. A method to construct a decision tree based classifier is proposed that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity. The classifier consists of multiple trees constructed systematically by pseudorandomly selecting subsets of components of the feature vector, that is, trees constructed in randomly chosen subspaces. The subspace method is compared to single-tree classifiers and other forest construction methods by experiments on publicly available datasets, where the method's superiority is demonstrated. We also discuss independence between trees in a forest and relate that to the combined classification accuracy}, author = {Ho, Tin K. }, citeulike-article-id = {1223894}, doi = {http://dx.doi.org/10.1109/34.709601}, journal = {Pattern Analysis and Machine Intelligence, IEEE Transactions on}, keywords = {decision-trees, feature-selection, mixture-of-experts, random-subspace-method}, number = {8}, pages = {832--844}, posted-at = {2008-07-18 17:23:56}, priority = {4}, title = {The random subspace method for constructing decision forests}, url = {http://dx.doi.org/10.1109/34.709601}, volume = {20}, year = {1998} }

TY - JOUR ID - ho98random L3 - citeulike-article-id:1223894 TI - The random subspace method for constructing decision forests JF - Pattern Analysis and Machine Intelligence, IEEE Transactions on VL - 20 IS - 8 SP - 832 EP - 844 N2 - Much of previous attention on decision trees focuses on the splitting criteria and optimization of tree sizes. The dilemma between overfitting and achieving maximum accuracy is seldom resolved. A method to construct a decision tree based classifier is proposed that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity. The classifier consists of multiple trees constructed systematically by pseudorandomly selecting subsets of components of the feature vector, that is, trees constructed in randomly chosen subspaces. The subspace method is compared to single-tree classifiers and other forest construction methods by experiments on publicly available datasets, where the method's superiority is demonstrated. We also discuss independence between trees in a forest and relate that to the combined classification accuracy KW - decision-trees KW - feature-selection KW - mixture-of-experts KW - random-subspace-method AU - Ho, Tin PY - 1998/// UR - http://dx.doi.org/10.1109/34.709601 ER -