Overview
The bias variance tradeoff discusses how the accuracy and error of a model is related to the model complexity.
Variations of the bias variance tradeoff can be found within mathematical statistics, see bias variance in stastistical inference for example.
The discussion here follows the logic in Abu Mostafa.
Definitions
The bias variance tradeoff is defined in terms of the two following definitions.
{% bias(x) = (\bar{g}(x) - f(x))^2 %}
{% var(x) = \mathbb{E}_{\mathcal{D}} (g^{\mathcal{D}}(x) - \bar{g}(x))^2 %}
Bias Variance Tradeoff
The bias variance tradeoff is stated as
{% \mathbb{E}_D [\mathbb{E}_{out}(g^D)] = \mathbb{E}_x[bias(x)+var(x)] %}
derivation