K Nearest Neighbours

• non-parametric classifier
• Simply look at the K points in the training set that are closest to the input, counts how many members of each class are in this set, and returns the probability

Here

$N_k(x, D)$ are the indices of the K nearest points to $x$ in $D$.

$I(e)$ is the indicator function defined as followes:

• Generally works quite well if the distance metric is good and has enough labeled training data.
• The main problem is that they do not work well with high dimensional inputs due to the need for dense datasets and the Curse of Dimensionality.

---

Backlinks

• Curse of Dimensionality
• Titanic