Psychometrics: Reliability and Validity Terms

Solution Preview

Please see response attached, which is presented below as well. I hope this helps and take care.

1. Explain each of the following in own words. If appropriate, give an example. Each should

These definitions are measures of reliability or terms related to reliability.

Definition: Reliability is the consistency of your measurement, or the degree to which an instrument measures the same way each time it is used under the same condition with the same subjects. In short, it is the repeatability of your measurement. A measure is considered reliable if a person's score on the same test given twice is similar. It is important to remember that reliability is not measured, it is estimated. There are two ways that reliability is usually estimated: test/retest (discussed below) and internal consistency. (1)

Estimates of Reliability: It is impossible to calculate a reliability coefficient that conforms to the theoretical definition. Recall, the theoretical definition depends on knowing the degree to which a population of examinees vary in their true achievement (or whatever the test measures). But if we knew that, then we wouldn't need the test! Instead, there are several statistics (coefficients) commonly used to estimate the stability of a set of test scores for a group of examinees: test-retest reliability, split-half reliability, measures of internal consistency, and alternate form reliability are the most common. (5)

a) Internal consistency, homogeneity, and alpha coefficient

Internal consistency estimates reliability by grouping questions in a questionnaire that measure the same concept. For example, you could write two sets of three questions that measure the same concept (say class participation) and after collecting the responses, run a correlation between those two groups of three questions to determine if your instrument is reliably measuring that concept.

The test assume homogeneity of construct e.g. test is measuring a single construct - unidimensional construct
a. There is homogeneity in aspect of the construct you are trying to measure.
b. Example: Construct = achievement
Focus of test instrument is motivation, but not innate ability (13).

One common way of computing correlation values among the questions on your instruments is by using Cronbach's Alpha. In short, Cronbach's alpha splits all the questions on your instrument every possible way and computes correlation values for them all (we use a computer program for this part). In the end, your computer output ...