Overview
Classification is the process of assigning one of a finite set of labels to a point (or points) in a dataset.Types
The types of classification algorithms can roughly be divided into two categories.- Linear - models were the first and easiest to implement. They laid the foundation for machine learning in general. Even though linear patterns are somewhat rare in real life, the linear based algorithms can generally be made to recognize non-linear patterns using kernel methods or feature extraction.
- Non Linear
There are multiple algorithms that can be used to learn classification problems.
Response Function
The response function the case of classification maps each input to a finite set of outputs, often labels, as in the following.
let labels = ['apple', 'orange', 'cherry'];
How the classes are encoded is often dependent on the type of algorithm. For instance, the previous set of three labels could also be encoded as column vectors.
{%
apple = \begin{bmatrix}
1 \\
0 \\
0 \\
\end{bmatrix}
%}
{%
orange = \begin{bmatrix}
0 \\
1 \\
0 \\
\end{bmatrix}
%}
{%
cherry = \begin{bmatrix}
0 \\
0 \\
1 \\
\end{bmatrix}
%}