ethical issues we encounter in using hypothesis testing

Solution Preview

One may be faced with the problem of making a definite decision with respect to an uncertain hypothesis which is known only through its observable consequences. A statistical hypothesis test, or more briefly, hypothesis test, is an algorithm to state the alternative (for or against the hypothesis) which minimizes certain risks.

This information describes the commonly used frequentist treatment of hypothesis testing. From the Bayesian point of view, it is appropriate to treat hypothesis testing as a special case of normative decision theory (specifically a model selection problem) and it is possible to accumulate evidence in favor of (or against) a hypothesis using concepts such as likelihood ratios known as Bayes factors.

There are several preparations we make before we observe the data.

The hypothesis must be stated in mathematical/statistical terms that make it possible to calculate the probability of possible samples assuming the hypothesis is correct. For example: The mean response to treatment being tested is equal to the mean response to the placebo in the control group. Both responses have the normal distribution with this unknown mean and the same known standard deviation ... (value).
A test statistic must be chosen that will summarize the information in the sample that is relevant to the hypothesis. Such a statistic is known as a sufficient statistic. A sufficient statistic for a parameter of a distribution exists if and only if the distribution forms an exponential family. In the example given above, it might be the numerical difference between the two sample means, m1 − m2.
The distribution of the test statistic is used to calculate the probability sets of possible values (usually an interval or union of intervals).
Among all the sets of possible values, we must choose one that we think ...