Overview
Given a random variable {% y(\vec{x}) %}, the value of the variable can be expressed as sum of the expected value of the variable plus noise.
{% y(\vec{x}) = \mathbb{E}(y(\vec{x})) + noise %}
The expecation itself is a function of the underlying
{% f(\vec{x}) = \mathbb{E}(y(\vec{x})) %}
Often times, an analyst wants to approximate the expectation function, given a set of observed data points.
This makes the task more complex, as any given data point includes noise in its value, so that the analyst
cannot merely fit a function to the data, rather, she must use a procedure such as
maximum likelihood
in addition to
approximation techniques
to deliver a result.
Techniques
- OLS Regression - the most prominent method of approximating an expectation function is to assume a functional form, and then use a squared error loss function to find the least square error function. Assuming that the assumed form of the function is accurate, it can be shown that this results in an unbiased estimate.