Hypothesis Testing
Overview
Hypothesis testing is a statistical method used to determine whether a given Hypothesis is true.
Typically the hypothesis to be tested is a statement about a given distribution from which the
tester has a sample.
Testing Distribution Parameters
The standard case for hypothesis testing is to test whether an unknown distribution parameter is a given value or not.
As an example, if the researcher knows that the distribution in question is a
normal distribution with a known standard deviation,
but is uncertain as to the mean of the distribution, she may decide to construct a hypothesis test to
test whether the mean is a given value.
Formulation
The researcher starts with a sample from a distribution for which a given paremeter {% \theta %}
is unknown. (That is, the distribution can be stated as a function of the parameter {% F(\theta) %}).
Next the researcher states the Hypothesis, denoted {% H_0 %}. Typically this will take the form of
a statement of teh value of {% \theta %}, for example:
{% H_0 : \theta = 0 %}
The statement of the hypothesis then also implies the alternative hypothesis, {% H_1 %}, which is the
case when {% H_0 %} does not hold. For the example above
{% H_1 : \theta \neq 0 %}
Testing the Hypothesis
The standard way to test a hypothesis is to calculate some aggregate value from the collected sample, and then compare the
calculate value against the distribution that would occur if the hypothesis were true.
As an example, suppose the researche has a sample drawn from a normal distribution with known standard deviation, but
unknown mean. The researcher hypothesizes that the mean is zero, thereby stating the hypothesis as
{% H_0 : \mu = 0 %}
Next, the researcher calculates the average of the values in the sample
Because the distribution of a
sum of noramls
is known, the researcher can
Two Sided Hypothesis
After plotting the distribution of the aggregate value given the hypothesis, the researcher rejects the hypothesis if the true
sampled value lies somewhere in the tails of the distribution.
One Sided Hypothesis
In a one sided test, the researcher rejects the hypothesis only when the true value lies in only one of the tails.
Error Types
A hypothesis test is subject to two possible errors, given the following names.
- Type I - rejecting the hypothesis {% H_0 %} when the hypothesis is true
- Type II - rejecting the alternative hypothesis {% H_1 %} when the althernative hypothesis is true
Using Simulations
As mentioned above, monte carlo simulations is often an effective way for calculating the solution to inference problems
when you dont have an inverse cumulative function. For information about simulations, please see
simulations
