Validity and Reliablity

Solution Preview

RESPONSE:

RELIABILITY:

An example of the importance of reliability is the use of measuring devices in Olympic track and field events. For the vast majority of people, ordinary measuring rulers and their degree of accuracy are reliable enough. However, for an Olympic event, such as the discus throw, the slightest variation in a measuring device -- whether it is a tape, clock, or other device -- could mean the difference between the gold and silver medals. Additionally, it could mean the difference between a new world record and outright failure to qualify for an event. Olympic measuring devices, then, must be reliable from one throw or race to another and from one competition to another. They must also be reliable when used in different parts of the world, as temperature, air pressure, humidity, interpretation, or other variables might affect their readings. Quixotic reliability refers to the situation where a single manner of observation consistently, yet erroneously, yields the same result.

1. Test - Internal Consistency Reliability

Application-- Used to assess the consistency of results across items within a test. In internal consistency reliability estimation we use our single measurement instrument administered to a group of people on one occasion to estimate reliability. In effect we judge the reliability of the instrument by estimating how well the items that reflect the same construct yield similar results. We are looking at how consistent the results are for different items for the same construct within the measure. There are a wide variety of internal consistency measures that can be used (e.g. split-half, test/retest, parallel forms). For example, the split-halves method is often used to measure the internal consistency reliability of survey instruments. (http://www.socialresearchmethods.net/kb/reltypes.php)

Strength: There is wide variety of measures to use based on needs of the researcher. Theory based and experts determine internal consistency providing multiple items to test internal consistency of items.

Weakness: One major constraint of all internal consistency measures-- you have to have multiple items designed to measure the same construct. This is relatively easy to achieve in certain contexts like achievement testing (it's easy, for instance, to construct lots of similar addition problems for a math test), but for more complex or subjective constructs this can be a real challenge. (http://www.socialresearchmethods.net/kb/reltypes.php)

2. Test: Split-half Reliability

Application: It a type of internal reliability. In split-half reliability we randomly divide all items that purport to measure the same construct into two sets. We administer the entire instrument to a sample of people and calculate the total score for each randomly divided half. The split-half reliability estimate, as shown in the figure, is simply the correlation between these two total scores. In the example it is .87. (1) (http://www.socialresearchmethods.net/kb/reltypes.php)

Strength: It is a fairly easy measure and estimate to calculate--the correlation coefficient between the two sets of measurements is often used as a quantitative measure of the internal consistency reliability of a survey instrument. (http://www.statistics.com/resources/glossary/s/splithalves.php).

Weakness: As mentioned above, you have to have multiple items designed to measure the same construct, which is a challenge for some complex constructs (e..g, subjective measures versus mathematics) (http://www.socialresearchmethods.net/kb/reltypes.php)

3. Test:Test / Retest Reliability

Application: We estimate test-retest reliability when we administer the same test to the same sample on two different occasions. The test-retest estimator is especially feasible in most experimental and quasi-experimental designs that use a no-treatment control group. In these designs you always have a control group that is measured on two occasions (pretest and posttest). This approach assumes that there is no substantial change in the construct being measured between the two occasions. The amount of time allowed between measures is critical. We know that if we measure the same thing twice that the correlation between the two observations will depend in ...