Evaluation of Investment In Workforce Education & Development
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Evaluation of Investment InWorkforce Education & Development
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
Decision types
ActualIntended
Ends
Means
PLANNING RECYCLING
STRUCTURING IMPLEMENTING
To determineobjectives
To judge and react to attainments
To design proceduresTo utilize, control, and refine procedures
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
Describe use of criteria in evaluation
Criteria
A decision requires judging among two or more alternatives.
Criteria are rules or tests for judging. Criteria are yardsticks for values.
Ö Values = estimated or assessed worth.Ö Values result from systems of beliefs, perceptions,
interpretations of facts, judgments of needs. Ö Values are shared and/or idiosyncratic.
Decision-makers apply criteria.Ö They must “own” criteria.Ö Often need assistance in identifying and finding criteria.
Four kinds of values that impinge on establishment of criteria
Instiutional valuesÖ E.g., “What does the Board of Trustees expect our College to
do?”
Values external to the institutionÖ E.g., “What does the State Legislature believe the College
should produce?”
Subsystem maintenance valuesÖ E.g., “What do my College colleagues believe we should do?”
Private valuesÖ E.g, “What do I think our College should produce?”
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
Delineate criteria for evaluation
Criteria for evaluation
ScientificÖ Internal and external validity, reliability, objectivity
PracticalÖ Relevance, importance, scope, credibility, timeliness,
pervasiveness
PrudentialÖ Efficiency
Criteria for evaluation:Scientific
Internal validityÖ No alternative explanations are possible for results obtained.
External validityÖ Findings are generalizable across people, places, and time.
ReliabilityÖ Accuracy of findings; reproducibility.
ObjectivityÖ Intersubjectivity; lack of bias.
Criteria for evaluation:Practical
RelevanceÖ Information serves decisions.
ImportanceÖ “Best” information culled to serve decisions.
ScopeÖ All necessary information is provided.
CredibilityÖ Clients/users believe information.
TimelinessÖ Information provided when needed.
PervasivenessÖ Information provided to all clients/users.
Criteria for evaluation:Efficiency
Conducted with the “best” use of resources.
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
Suggest a meta-methodology for evaluation
Work breakdown of evaluation design
Delineation of information needs
Delineation of information needs
Delineation ofinformation needs
Delineation ofinformation needs
Work breakdown of evaluation design
Plan forobtaining information
Plan forobtaining information
Plan forobtaining information
Work breakdown of evaluation design
Plan forproviding information
Plan forproviding information
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
Specify four types of evaluation
Evaluation &decision-making
Evaluation types
ActualIntended
Ends
Means
CONTEXT PRODUCT
INPUT PROCESS
For planningdecisions
For recyclingdecisions
For structuringdecisions
For implementingdecisions
Evaluation types:Context
Defines relevant environment. Identifies unmet needs and unused opportunities. Diagnoses problems that prevent needs from being
met and opportunities from being used. Is macro-analytic and systematic. Involves contingency and congruence studies to set
direction.Ö Contingency - searches for opportunities and pressures outside
system to promote improvement within.Ö Congruence - compares actual and intended system
performance to affirm objectives or consider new objectives.
Evaluation types:Input
Determines how to use resources to achieve aims.Ö Identifies relevant capabilities.Ö Specifies strategies for achieving goals.Ö Fabricates specific designs to accomplish aims.
End product of input evaluation is one or more procedural designs, with a lists of costs and benefits.Ö Assesses alternative designs in light of resource
requirements/limits.Ö Identifies procedural barriers, and risk/consequences of failure
to overcome.Ö Relevance of various designs to aims.
Build or make? Project proposals embody the results of input
evaluation.
Evaluation types:Input (cont.)
Although context evaluation is systematic and macro-analytic, input evaluation is ad hoc and micro-analytic.
Some questions asked:Ö Is need documented?Ö What strategies already exist?Ö What are costs and benefits of competing strategies?Ö What is track record for competing strategies?Ö Are designs logical? Based on sound theory? Assumptions met?
Legal? Moral? Ö Side effects? Ö Likely to be adopted?
Evaluation types:Process
Aims:Ö Detects or predicts defects in procedural design or in
implementation in early stages.Ö Provides information for programmed decisions.Ö Records implementation as it occurs.
Depends on context and input evaluation.Ö More adequate context and input evaluation, the less critical the
need for process evaluation.Ö Less adequate context and input evaluation, the more critical
process evaluation becomes.
More important than product evaluation in early development stages, but less important as more structure and knowledge is developed.
Evaluation types:Product
Interprets attainments at end of project cycle or as often as necessary during cycle for accountability or improvement.
Involves: Ö Devising operational definitions of objectives.Ö Obtaining criterion measures.Ö Comparing measures with some absolute or relative standards.Ö Making rational interpretations of outcomes using recorded
context, input, and process information.
Criteria are instrumental or consequential:Ö Instrumental -- related to intermediate accomplishments
prerequisite to some end.Ö Consequential -- pertain to fundamental, penultimate ends
sought.
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
Present a general evaluation model
Evaluation of Investment InWorkforce Education & Development
Related Documents
