Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations

by: Peter J Bickel, Elizaveta Levina

Bernoulli, Vol. 10, No. 6. (2004), pp. 989-1010.

View FullText article

http://projecteuclid.org/DPubS?service=UI…

Reviews [Write a review of this article]

There are no reviews of this article

Find related articles from these CiteULike users

bpacker

Find related articles with these CiteULike tags

regularization, statistics

Abstract

We show that the `naive Bayes' classifier which assumes independent covariates greatly outperforms the Fisher linear discriminant rule under broad conditions when the number of variables grows faster than the number of observations, in the classical problem of discriminating between two normal populations. We also introduce a class of rules spanning the range between independence and arbitrary dependence. These rules are shown to achieve Bayes consistency for the Gaussian `coloured noise' model and to adapt to a spectrum of convergence rates, which we conjecture to be minimax.

@article{citeulike:2086477, abstract = {We show that the `naive Bayes' classifier which assumes independent covariates greatly outperforms the Fisher linear discriminant rule under broad conditions when the number of variables grows faster than the number of observations, in the classical problem of discriminating between two normal populations. We also introduce a class of rules spanning the range between independence and arbitrary dependence. These rules are shown to achieve Bayes consistency for the Gaussian `coloured noise' model and to adapt to a spectrum of convergence rates, which we conjecture to be minimax.}, author = {Bickel, Peter J. and Levina, Elizaveta }, citeulike-article-id = {2086477}, journal = {Bernoulli}, keywords = {regularization, statistics}, number = {6}, pages = {989--1010}, posted-at = {2007-12-10 18:48:07}, priority = {3}, title = {Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations}, url = {http://projecteuclid.org/DPubS?service=UI\&\#38;version=1.0\&\#38;verb=Display\&\#38;handle=euclid.bj/1106314847}, volume = {10}, year = {2004} }

RIS record

TY - JOUR ID - citeulike:2086477 L3 - citeulike-article-id:2086477 TI - Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations JF - Bernoulli VL - 10 IS - 6 SP - 989 EP - 1010 N2 - We show that the `naive Bayes' classifier which assumes independent covariates greatly outperforms the Fisher linear discriminant rule under broad conditions when the number of variables grows faster than the number of observations, in the classical problem of discriminating between two normal populations. We also introduce a class of rules spanning the range between independence and arbitrary dependence. These rules are shown to achieve Bayes consistency for the Gaussian `coloured noise' model and to adapt to a spectrum of convergence rates, which we conjecture to be minimax. KW - regularization KW - statistics AU - Bickel, Peter AU - Levina, Elizaveta PY - 2004/// UR - http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.bj/1106314847 ER -

RIS BibTeX