Suppose a revolutionary hypothesis has been proposed by the famous Professor Brainstorm, and yet the truth were known, it really wouldn't fit facts. In the formal language of hypothesis testing, it is the null hypothesis that is true, not Brainstorm's hypothesis. (The professor interpreted his results at the 5 percent level). His research was published, and he won accolades for his scientific breakthrough. What does this story say about hypothesis testing and research? What is the likelihood of this happening? There are a number of safeguards in place so this should not happen in the scientific world. What are some of these?
This story reflects a common concern about classical hypothesis testing, which often uses 0.05 significance level. Researchers who use classical hypothesis testing approach set the significance level (i.e., critical range for rejecting the null hypothesis) even before they collect any data. And very often the significance level is set as 0.05. Subsequently researchers collect data and calculate the statistics of interest. If the observed statistics fall in the rejection region, the null hypothesis is rejected. If not, the null hypothesis is failed to be rejected.
Keep in mind decisions based on hypothesis testing rely on a sample. Let us say the null hypothesis is rejected, which is the case of Professor Brainstorm, there are two possibilities:
1. The null hypothesis is not correct; that is, the population parameters set in the null hypothesis are not true for the population. In this case, our decision of rejecting the null is correct.
2. The null hypothesis is indeed correct; that is, the population parameters set in the null hypothesis are true for the population. In this case, our decision of rejecting the null is false.
Indeed, in every ...
The solution provides a detailed explanation on the rejection of a null hypothesis when it is true.