Confidence intervals are inextricably linked to the practice of hypothesis testing because they represent the possible range of limits for which the true mean value of a population may lie for a given proportion of samples. To properly understand the concept of confidence intervals, it is critical to have a firm understanding of what a “p-value” is.
Before any test statistics are computed for a hypothesis test, it is imperative that a significance level is set. Typically, the significance level is set at 5%, which means that 95% of the time, the sample mean obtained will fall within a range where the true population mean lies. Remember that the null hypothesis, which states that there is no difference between two observations, is believed to be true. Thus, a p-value of 0.05 or greater represents the probability of obtaining study results which allow for the null hypothesis not to be rejected.
A p-value which is small is indicative of the results obtained being unlikely when the null hypothesis is true and conversely, a large p-value states that the results are likely to be obtained when the null is true1. However, a p-value may have some error associated with it, and that is why a confidence interval is utilized. The confidence interval represents the certainty associated with the p-value. The 95% confidence interval is the standard1.
Formula: sample mean (xbar) – 1.96 x s/√n to sample mean (xbar) + 1.96 x s/√n
s = sample standard deviation
n = number of samples
This formula is based off of the normal distribution and is representative of a two-sided test using a significance level of 5%.
1. Freeman, J.V. (2009). Confidence Intervals. In Encyclopedia of Medical Decision Making (pages 164-168). Retrieved from: http://knowledge.sagepub.com.proxy.queensu.ca/view/medical/n48.xml