It is important to have an understanding of some of the more traditional approaches to function estimation and classification before delving into the trendier topics of neural networks and decision trees. Many of these methods build on an understanding of each other and thus to truly be a MACHINE LEARNING MASTER, we’ve got to pay our dues. We will therefore start with the slightly less sexy topic of kernel density estimation.
Let be a random variable with a continuous distribution function (CDF) and probability density function (PDF)
Our goal is to estimate from a random sample . Estimation of has a number of applications including construction of the popular Naive Bayes classifier,