To understand hypothesis testing we start with an example. Given a coin, consider a test where we have to determine weather the coin is biased towards head or not. Now there are two situations:
- Situation 1: The coin is biased towards head i.e P(H)>0.5
- Situation 2: The coin is not biased towards head i.e. P(H) = 0.5
To solve the above problem, we have to design an experiment. Let the experiment be as follows.
Experiment: Flip a coin 5 times. Let number of heads that appear in all be X. Here X is a random variable. (In theory of hypothesis testing, X is called a test-statistics.)
Now, we try to determine the probability that X = 5 given that coin is not biased towards head. This is nothing but conditional probability. The same can be written as:
P(X=5 | Coin is not biased towards head) = ?
If coin is not biased towards head, that would mean, P(H) = 1/2 = 0.5. This means in each of the flip of the coin, the probability of getting head is 1/2. So. .
P(X=5 | Coin is not biased towards head) = 1/25
There is a 3% chance of getting 5 heads in 5 flips given that the coin is not biased towards head.
Thus hypothesis testing is the probability of observation by experiment if the assumption is true. In our case:
observation by experiment is (X=5) , that is, number of heads that appear, which is 5 in our case. AND
assumption is that the coin is not biased towards head. This is the null hypothesis.
P(Observation by Expt. | Null hypothesis) = 3% = 1/25
This probability value is called P-Value in hypothesis testing. P-Value is considered small if it is less then 5%. (This is a thumb rule). Now:
if P(Observation | Null Hypothesis) < 5% then the hypothesis may be incorrect and it is rejected. We cannot reject the observation as it is result of an experiment that we performed which can never be wrong. The observation was made by us after the experiment.
Therefore we reject the idea that the coin is not biased towards head. H0 is rejected.
Null hypothesis is generally referred as H0