OLS Regression - Classical Model

Overview


The classical model specifies a set of assumptions that were originally developed to justify the use of the regression algorithm. Additional sets of assumptions have been developed to deal with situations where the classical model does not hold.

  • Linearity: {% \vec{y} = X \vec{\beta} + \vec{\epsilon} %} where {% \epsilon %} satisfies the additional properties below.
  • Strict Exogeneity : {% \mathbb{E}[\vec{\epsilon}|X] = 0 %}
  • No Multicollinearity : The rank of the n:k matrix X is k with probability 1.
  • Spherical Errors : {% \mathbb{E}(\vec{\epsilon} \vec{\epsilon} ^T | X) = \sigma ^2 I %}


where {% X %} is a matrix, with each row being a single observation.

Finite Sample Properties


When a model satifies the classical model assumptions above, the following properties can be shown.

  1. {% \mathbb{E}(\vec{b}|X) = \vec{\beta} %} :
    derivation
  2. {% Var(\vec{b}|X) = \sigma^2 (X^T X)^{-1} %}
    derivation
  3. {% Cov(\vec{b},\vec{y}-X \vec{b}|X) = 0 %}

Contents