Variance

Overview


The second moments of a distribution is a measure of the spread of the disribution about the mean (first moment).

  • Covariance - the extension of the concept of variance to measure how two random variables co-vary

Variance


The variance is similar to the second moment, it is the expectation of the variable squared, but this time with the average subtracted out.

{% Var(X) = \mathbb{E}[(X - \mu)^2] %}

Two Normal Distributions with Differing Variance

copy

Standard Deviation


The standard deviation is defined to be the square root of the variance.
{% Std \; Dev(X) = \sqrt{Variance(X)} %}

Sample Moments


The calculation of the variance brings forth an important issue in calculating moments on sample data. Suppose we calculate the sample variance as follows:

{% \mu = \sum_i^n X_i / n %}
and then the variance is calculated as
{% Var(X) = 1/n \times \sum_i^n(X_i - \mu)^2 %}
The problem is that the sample mean is random, different sample datasets will produce different estimates for the mean. This means that the mean itself has a variance, and this will contribute to the calculated sample variance.

To account for this variance, the estimated variance should be calculated as follows

{% Var(X) = 1/(n-1) \times \sum_i^n(X_i - \mu)^2 %}
This formula can be shown to be non biased in its estimate of variance. The varaince function listed above in the moments library is the sample variance that includes this correction. Any moment calculation that subtracts out the mean as we did for variance needs to make corresponding corrections.

Moments API



let mt = await import('/lib/statistics/moments/v1.0.0/moments.mjs');
let numbers = [1,2,3,4,5];
let variance = mt.variance(numbers);
Try it!

Variance Properties


The following relations hold for the variance of a sum. Here {% X %} and {% Y %} are random variables and {% a %} and {% b %} are constants.

  • {% Var(X+a) = Var(X) %}
  • {% Var(aX) = a^2Var(X) %}
  • {% Var(aX + bY) = a^2Var(X) + b^2 Var(Y) + 2abCov(X,Y) %}
  • {% Var(\sum a_iX_i) = \sum_{i,j=1} ^n Covar(a_iX_i, a_jX_j) %}

Variance and the Second Moment


The variance and the second moment are related by:
{% \mathbb{E}(X^2) = (\mathbb{E}(X))^2 + Var(X) %}