Graduate Level: Please answer the question and provide an example. Thanks.
If you have a significant difference (ie. a main effect of treatment) but more than two treatment groups, all you know is that there is a significant difference somewhere between your groups. You do not yet know where that difference is coming from.
Let's look at an example.
Imagine you're comparing doctors, lawyers, and nurses on their ability to diagnose various diseases. Each group is given 30 'medical cases' and asked to provide a diagnosis for each case. Your dependent variable can be the number of cases correctly identified. Let's say the mean number correct for each group is as follows:
Now, if we were to run an ANOVA on this, our null hypothesis would be:
and our alternative hypothesis would be that one or more of these equalities is incorrect. The important thing to remember is this: ONE OR MORE IS INCORRECT. So if we get a significant p-value for this ANOVA, we only know that:
maybe doctors and nurses and lawyers are all different from one another
OR maybe doctors differ from nurses but not from lawyers
OR maybe doctors differ from lawyers but ...