Share
Explore BrainMass

EVALUATION INSTRUMENT, validity, reliability

The following are questions from the Handbook of Training Evaluation and Measurement Methods by Jack J. Phillips. We are to consider some thoughts regarding these questions. I would appreciate some input!

BEFORE SELECTING AN EVALUATION INSTRUMENT, WHY IS IT IMPORTANT TO ANALYZE THE TYPES OF DATA NEEDED AND EXAMINE HOW THE DATA WILL BE USED IN THE EVALUATION?

CAN ALL DATA BE GROUPED INTO EITHER HARD TO SOFT DATA? EXPLAIN.

WHAT PROBLEMS OCCUR IN COLLECTING FEEDBACK FROM GROUPS OTHER THAN PARTICIPANTS' SUPERVISORS?

WHAT ARE THE ADVANTAGES AND DISADVANTAGES OF USING STANDARD INSTRUMENTS VERSUS CUSTOMIZED INSTRUMENTS?

WHAT TYPES OF VALIDITY WOULD BE OF MOST CONCERN FOR HRD PROGRAM EVALUATORS? WHY?

WHAT REPRESENTS A GOOD VALIDITY COEFFICIENT? IS THE DEVELOPMENT OF A VALIDITY COEFFICIENT NECESSARY IN PROGRAM EVALUATION? EXPLAIN.

HOW DOES RELIABILITY DIFFER FROM VALIDITY? WHAT REPRESENTS A GOOD RELIABILITY COEFFICIENT? IS IT NECESSARY TO DEVELOP A RELIABILITY COEFFICIENT FOR AN INSTRUMENT USED IN EVALUATION?

Solution Preview

Hi,

Here is a start.
Please look at the references as well.

Best of luck on your dissertation.

BEFORE SELECTING AN EVALUATION INSTRUMENT, WHY IS IT IMPORTANT TO ANALYZE THE TYPES OF DATA NEEDED AND EXAMINE HOW THE DATA WILL BE USED IN THE EVALUATION?

Data suits different types of statistical analysis. So, if a group comparison is needed, two sets of data will be required, and in turn means comparison will be needed, for which a t-test comparison will be best suited. In contrast, when the evaluation is still quantitative, but based on the scores on a performance based survey with 7 or 5- point likert-scale responses and corresponding numerical values assigned for each one of the response, then the best way of statistical analysis will be to include a summary statistics from specific analysis created for scale responses item-analysis for reliability and validity of the survey as a composite set of question statement, related to a given topic for assessment and performance evaluation. SPSS AND SAS are two soft-wares which can handle data analysis for several kinds of analysis like the ones described above (i.e. t-text comparisons and scale analysis).

CAN ALL DATA BE GROUPED INTO EITHER HARD TO SOFT DATA? EXPLAIN.
Hard data refer to the raw data and soft data refer to the break down of the data into categories and classes for the variables (i.e. independent or dependent variables or dummy variables). Large set of data with thousands of input items and or sample size can be further analyzed fro interesting themes or findings in data trends by shrinking techniques in which sub-clusters of problem statements or hypotheses are determined by dividing the hard date into a) more specific sub-set of data and b) consequently more manageable subset of data. Hard data is discrete data that can be counted and legitimately expressed as ratios, where as soft data is data, that consists of things like opinions, attitudes, satisfaction etc.

In IO this method is extremely useful, since even on MMPI and aptitude or performance tests for various types of job functions, or in the case of preliminary job assessment for matching the candidate with the job functions, one will need to divide the hard date into soft data or create more qualitative assessments based upon responses to interviewing themes and techniques or open ended questions as evaluation by a panel of hiring committee. It also has to be acknowledged that a lot of research done in I/O is based on the evidence- based practices of constant improvement in the running of the organizational ...

Solution Summary

EVALUATION INSTRUMENT, validity, reliability, research study design

$2.19