What are the special challenges in Criterion Deficiency and Criterion Contamination and what would be the impacts on employees and employers if the challenge were not addressed.
1. What are the special challenges in Criterion Deficiency and Criterion Contamination?
Let’s look at the special challenges in Criterion Deficiency and Criterion Contamination by example. Briefly, Criterion Shaped Behavior (CSB), is defined as any change in behavior that results from efforts to increase one's score on an Actual Criterion Measure of Performance. Probability of performance of such CSBs will be a function of the Motivational, Ability, and Constraint variables of the Universal Theory of Performance. http://employees.oneonta.edu/downinll/CSB-2.htm
Example: Implication from the Theory of Criterion Shaped Behavior
The designer of a performance evaluation system is encouraged in developing Actual Criterion Measures to take into account potentially undesirable consequences, Contamination CSBs and Deficiency CSBs, as well as potentially desirable Relevant CSBs. This should be done for any performance evaluation instrument, whether its "intended" use is for feedback, for assessment, or for shaping of desirable behaviors. The Universal Theory of Performance addressed the issue of whether or not an individual is motivated to engage in behaviors directed toward increasing scores on the Actual Criterion Measure. If we assume that the basic requirements for such motivation have been met, then we should address the issue of what specific changes in behavior are likely to result from motivation to increase one's score. http://employees.oneonta.edu/downinll/CSB-4.htm
CSB Theory describes three categories of such behavior, desirable Relevant CSBs, and undesirable Contamination and Deficiency CSBs. A designer of a new evaluation and compensation system wants to increase only the desirable behaviors, and should be sensitive to system characteristics that will promote or interfere with that objective. http://employees.oneonta.edu/downinll/CSB-4.htm
Validity of the Actual Criterion Measure is the primary consideration for implementation of a new evaluation system. A perfectly valid measure would assess all factors of the Ultimate Criterion, and no other factors. Invalidity relates to problems of Deficiency and of Contamination. In other words, Criterion deficiency or Criterion Deficiency might impact employees negatively, because the validity of the performance appraisals has been challenged. In other words, the employee’s true score is not realized and they may suffer negative consequences (or perhaps undeserved positive consequences) due to an invalid measure of performance appraisal. Overall, though, the organization will suffer also as all performance appraisals are linked to organizations goals and objectives as well. So Contamination and Deficiency CSBs impact the employee through invalid measures of performance, which in turn impacts the organization. http://employees.oneonta.edu/downinll/CSB-4.htm
Figure 1 represents these Ultimate and Actual Criterion Measures as two overlapping circles, and identifies the three resulting sectors as Criterion Contamination, Criterion Relevance, and Criterion Deficiency.
Figure 1: Overlap of the Ultimate and the Actual Criterion Measures http://employees.oneonta.edu/downinll/CSB-2.htm
Criterion Relevant Measures and Factors are those Actual Criterion Measures and Factors that overlap Ultimate Criterion Measures and Factors. Criterion Relevance is, essentially, the validity of the Actual Criterion Measure.
Criterion Contamination Measures and Factors refer to that portion of the Actual Criterion Measures and Factors that does not overlap with Ultimate Criterion Measures and Factors. Criterion Contamination is one source of invalidity of the Actual Criterion Measure.
Criterion Deficiency Measures and Factors refer to that portion of the Ultimate Criterion Measures and Factors that does not overlap with Actual Criterion Measures and Factors. Criterion Deficiency is one source of invalidity of the Actual Criterion Measure.
For purposes of assessment validity, some theorists argue that failure of a measure, the Actual Criterion, to fully capture the Ultimate Criterion, i.e. Criterion Deficiency, may not be very important (Landy & Trumbo, 1978, p. 138). If a measure is assumed to be non-reactive, then it is true that measuring only a part of the Ultimate Criterion is sufficient, as long as that part is highly correlated with the Ultimate Criterion as a whole. Once a measure becomes reactive, however, this approach is doomed to failure, for not only will its use lead to undesirable Deficiency CSBs, to be described below, but as a result of this effect the correlation of Actual Criterion Measures scores with The Ultimate Criterion will steadily decrease. Koretz (1989) has convincingly argued, without the aid of a systematic theory of Criterion Shaped Behavior, that the previously mentioned NAEP assessment test will be invalidated by attempts to broaden its use by rewarding states or school systems whose students ...
Through examples and discussion, this solution discusses the special challenges in Criterion Deficiency and Criterion Contamination and what the impacts would be on employees and employers if the challenges were not addressed.