Nearest Neighbor Algorithm

Overview

The nearest neighbor algorithm is an algorithm designed to estimate a datapoints properties, including possibly the category to which the point belongs, by examining the points near to the given point. For example, one might look at the nearest k neighbors to the point in question, and then assign a category to the point based on which category appears the most in the examined neighbors.

The algorithm depends on having a function, called a metric, which measures the "distance" between points in order to determine which points are closest to the point in question.

Algorithms

• K Nearest Neighbor - interpolates the value of a given point from the values of k points nearest it in the dataset
• Kernel Density - Extending the method a little further, we get a technique called kernel density estimation (or previously, parzen windows).