Reliability and Validity and Measurement Scores

identify and describe reliability studies that must be conducted by the researcher in each of the mini-cases shown below. This is a good way to learn about the information each reliability method provides, as well as sources of error examined by each method.

Read and study each of the following mini-cases:

Case A
A research group was asked to provide a test publisher with the internal consistency reliability coefficient for a 100-item, dichotomously-scored (correct/incorrect) measure of intelligence they created.

Case B
Dr. Bates is using a published behavioral rating scale in her research on children's social aggression on the playground. She has four research assistants who will spend approximately 50 hours each observing and recording the children's behavior, using the rating scale. Dr. Bates wants to make sure the research assistants are recording and scoring the children's behavior in the same way.

Case C
Researcher who developed a measure of optimism wants to show that their test measures a stable personality characteristic that changes very little over time.

Case D
Dr. Ludlow is interested in gathering reliability data for a creativity test he recently developed. Items on the test were scored from 1 to 5, with 5 indicating a very creative response and 1 indicating a non-creative response. Dr. Ludlow's primary interest is whether all the items on the test measure the same thing.

Case E
Dr. Gladeau is approached by a teacher in the local public school system who asks him to help her with a problem. Specifically the teacher states that she teaches two different third-grade math classes and uses different unit tests in each. "I need to provide evidence that my two math tests are measuring the same thing - that even though the items themselves are different, the tests are functionally the same."

Case F
Dr. Veverka is interested in whether the items on her 50-item multiple-choice exam are of similar difficulty, or whether they increase in difficulty.

For each mini-case, answer the following questions and enter your responses on the answer sheet provided.
1. Identify the method of estimating reliability the researcher should use.
2. Describe the actions that must be taken to find the reliability coefficient.
3. Describe the information that is provided by the reliability coefficient, including sources of error.