Hypothesis Testing and Type 1/Type 2 Errors

1. "Our sample size was sufficient to detect a ... 10 point difference in [severity] scores between the groups with over 80% power. However, over 150 subjects would be required to detect a difference between groups of 20% in the percentage of cases with weight loss with 80% power." These authors did not find a difference in [severity] score or weight loss percentage between groups. They had 52 subjects. [Modified from Margolis et al, Am. Rev. Resp. Dis. V 142, 1990.] Assume that a 10-point and a 20% difference are both clinically significant.

The authors are making what argument in this quote? (Pick all that apply).
a. They had an adequate sample size for both of these questions.
b. They had an adequate sample size for detecting a severity difference but not for a difference in patients with weight loss.
c. They needed a larger N for both differences.
d. The study supports the absence of a population difference on either parameter.
e. The study supports the absence of a difference in severity score, but not in percent with weight loss.
f. No evidence is provided against a population difference on either parameter.
g. Power is not relevant to whether or not the study showed the absence of a difference.

2.A researcher reports "The mean + S on parameter X for the two groups was 100 + 20 and 103 + 20, p > 0.10, two-tailed. There were 30 subjects in each group." You are reviewing this report. You would consider a difference of 4 to be important on parameter X. Is this result good evidence that there is no important mean difference between the groups on parameter X?

a. Yes, because the p value is non-significant.
b. No, because the p-value is significant.
c. No, because the power is less than 25%.
d.Yes, because the power is less than 25%.
e. No, because the power is over 70%.
f. Yes, because the power is over 70%.

3.A researcher estimates that with 50 subjects per group, power will be 90% for a theoretically important difference. Nevertheless, although 50 per group were used, the result is not significant. This indicates that:
a. The researcher may have committed a type I error.
b. The researcher may have committed a type II error.
c. These results provide reasonable evidence that a theoretically important difference does NOT exist.
d. No evidence against a theoretically important difference is provided.
e. a and c
f. a and d
g. b and c
h. b and d

4. You are planning a survey of students on a scale that measures stress. You will be interested in comparing first year and second year students on this stress scale, which has a SD of 10. You are interested in fairly small differences, such as a mean difference of 4. Compute the sample size needed for a two tailed unpaired t-test to compare the classes. You want at least 70% power. (1 point).

5.. You are reviewing a research proposal for your institution. The researcher is comparing two groups on BUN (a measure of kidney function -- higher is bad), and is expecting a SD of about 3 for the kind of patients studied. The previous research that is cited suggests that the groups are likely to differ by a mean of about 1.5. The researcher is proposing 10 subjects per group. Roughly estimate the power of the study, for a two-tailed test. What comment would you make to the researcher? (2 points)

6. A certain study is repeated 100 times, each time comparing a particular (always the same) treatment and control condition. 10 of the repeats lead to a rejection of the null hypothesis. The other 90 times the null is NOT rejected.
a. If there is, in truth, no difference between treatment and control, identify the results which constitute type I errors and type II errors, if any. Estimate alpha and beta, if possible. (2 points)
b. If there is a true difference between treatment and control, answer the same questions. (2 points)

7. A researcher would like to demonstrate the absence of a difference between two treatments. Which combination of the following (choose 1 from each row) would most convincingly accomplish this?
Row 1: p = 0.01 p = 0.05 p = 0.60
Row 2: power = 90% power = 20%

8. A researcher reports that the type II error probability was set to 0.3, and alpha was set to 0.02. (1 point for each of the following, 4 total):
a. What is the power of this study?
b. If the null hypothesis is true, how likely are the researchers to obtain a significant result?
c. If the null hypothesis is false, how likely are the researchers to obtain a significant result?
d. What is the largest p-value that would be statistically significant.