Mon, 07 Jul 2008 16:42:29 BST CiteULike: neteler boosting http://www.citeulike.org/user/neteler/tag/boosting CiteULike.org en-gb Copyright © 2004-2008 citeulike.org http://www.citeulike.org/user/neteler/article/167644 <i>(1997), pp. 322-330.</i><br /><br />One of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated hypothesis usually does not increase as its size becomes very large, and often is observed to decrease even after the training error reaches zero. In this paper, we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule, where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label. We show that techniques used in the analysis of Vapnik’s support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. We also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples. We argue that this explanation is better than explanations based on the bias-variance decomposition and provides new insight to the behaviour of boosting as well as other voting schemes, such as bagging and ECOC. Boosting the margin: a new explanation for the effectiveness of voting methods Robert Schapire Yoav Freund Peter Bartlett Wee Lee (1997), pp. 322-330. 2005-04-22T18:54:02-00:00 1997 322 330 Morgan Kaufmann boosting ensemble machine-learning http://www.citeulike.org/user/neteler/article/784728 <i>(2000), pp. 220-229.</i><br /><br />Boosting of tree-based classifiers has been interfaced to the Geographical Information System (GIS) GRASS to create predictive classification models from digital maps. On a risk management problem in landscape ecology, the performance of the boosted tree model is better than either with a single classifier or with bagging. This results in an improved digital map of the risk of human exposure to tick-borne diseases in Trentino (Italian Alps) given sampling on 388 sites and the use of several overlaying georeferenced data bases. Margin distributions are compared for bagging and boosting. Boosting is confirmed to give the most accurate model on two additional and independent test sets of reported cases of bites on humans and of infestation measured on roe deer. An interesting feature of combining classification models within a GIS is the visualization through maps of the single elements of the combination: each boosting step map focuses on different details of data distribution. In this problem, the best performance is obtained without controlling tree sizes, which indicates that there is a strong interaction between input variables. Boosting of Tree-based Classifiers for Predictive Risk Modeling in GIS C Furlanello S Merler (2000), pp. 220-229. 2006-08-03T16:11:22-00:00 2000 220 229 Springer boosting disease ixodes machine-learning risk http://www.citeulike.org/user/neteler/article/165116 <i>Machine Learning, Vol. 45, No. 1. (2001), pp. 5-32.</i><br /><br />Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of ... Random Forests Leo Breiman doi:10.1023/A:1010933404324 Machine Learning, Vol. 45, No. 1. (2001), pp. 5-32. 2005-04-19T18:57:17-00:00 2001 Machine Learning 45 1 5 32 Kluwer Academic Publishers, Boston boosting cart classification ensemble machine-learning