Overview
Feature extraction is the process of taking a dataset and computing features derived from the values in the dataset. Feature extraction is often used to overcome limitions of the learning algorithm, for instance, if the learning algorithm learns only linear patterns, but the patterns in the data set are non-linear.
Features are generally extracted into a vector (typically, a column vector ), before being sent to the model.
Reasons for Feature Extraction
- The Model has Limitations in the Types of Patterns it can Recognize - the typical example of this is the perceptron or the support vector machine, both of which can only recognize linear patterns. The kernel trick is used to project the data into a space where the pattern is linear and a perceptron or svg can be used.
- Prior Domain Knowledge - having knowledge about the domain and using that knowledge to build a feature can increase the models ability to recognize the pattern in question, or at least to fit the data in less time.
- Dimensions - a common method of feature extraction is to increase or reduce the number of the dimensions in the data.
Topics
The following are some common feature extraction techniques.
- Time Series Feature Extraction
- Normalization : is the process of scaling all the values of a dataset. Typically this means scaling the numeric value from its measured value to the number of standard deviations it is away from the mean of the dataset at hand.
- Kernel Methods are a set of methods that can convert an algorithm that
recognizes linear patterns to one that can recognize non-linear patterns.
Example algorithms that can be extended using kernel methods:
- perceptron
- linear regression
- principal component analysis
- clustering