Training Evaluation

Evaluation Of Training and Development

DEFINITIONEvaluation is a process to determine the relevance ,effectiveness, and impact of activities in light of their objectives.Training evaluation is the process of deep examination and analysis of Selecting measures Gathering information based on those measures Comparing what participant learn to some standard ,goal or expectation

DEFINITIONTraining evaluation is a systematic process of collecting information for and about a training activity which can then be used for guiding decision making and for assessing the relevance and effectiveness of various training components.

TYPES OF EVALUATION

Formative EvaluationSummative Evaluation

FORMATIVE E VALUATIONIt involves the collection of relevant and useful data while the training and development is being conducted.This information can identify drawbacks and unintended outcomes and is helpful in revising the plan and structure of training and development programmes to suit needs of the situation.

SUMMATIVE EVALUATIONIt is done at the end of the programme and makes an overall assessment of its effectiveness in relation to achieving the objectives and goals.

TYPES OF EVALUATIONEvaluation for Planning provides information with planning decision are made. Methods ,Materials ,Instructional aids, strategies are planned.Process Evaluation Conducted to detect defects in the procedural design of training activity during the implementation phase. Periodically conducted

TYPES OF EVALUATIONTerminal Evaluation Conducted to find out the effectiveness of training and development after it is completed. Impact Evaluation Assesses changes in on-the-job behaviour as a result of training and development efforts.

Evaluation StakeholdersTraining OrganiserTrainersParticipantsCurriculum developers

Training and DevelopmentIncreased skill

Increased Knowledge

Increased awareness

Improved workPerformance

Improved quality of Service

Increased RevenuesLeads toLeads toReactionLearningEvaluationApplicationCost benefitEvaluation

OBJECTIVES OF EVALUATIONTo validate training and development as a business tool.To justify the cost incurred in training and development.To help improve the design of training and development programmeTo help in selecting training and development methods

OBJECTIVES OF EVALUATIONTo determine strength and weakness of training and development.To determine success in accomplishing programme objectives.To test clarity and validity of methods used.To reinforce major points.To decide who should participate in future.

OBJECTIVES OF EVALUATIONTo determine whether the programme was the appropriate solution for the specified training.To establish a database that can assist management in making decisions.

PURPOSE OF EVALUATIONAt Macro level

At Micro Level

AT MACRO LEVELTo assess training and development programme's business impact as a whole.To facilitate selection of training and development programme's performance matrices related to organizations goals and objectives.To get at the business impact measure through direct correlation to organisational goals and objectives.

AT MICRO LEVELTo measure a specific training and development programmes business impact To combine this to overall programme business impact.

STRATEGIES TO EVALUATIONEvaluation design must be formed from strategic perspective.Designing a good evaluation effort involves when to collect evaluation measures and which group to collect them from.

STRATEGIES TO EVALUATIONCompare the trainees after the training with the way they were before the training.Compare the learning ,behaviour or result of the trained group with that has not been trained.

EVALUATION DESIGNOne-Shot Posttest Only DesignOne-Group Pretest - Posttest Design Multiple Baseline DesignPretest-Posttest Control Group Design

ONE-SHOT POSTTEST ONLY DESIGN

Training evaluation measures are collected only from the trained groupThere is no pertaining measure and no untrained group for the purpose of comparison.TrainMeasure

ONE-GROUP PRETEST - POSTTEST DESIGNTraining group is assessed both before and after the training programme.Helps to assess change from one time period to another.TrainMeasureMeasure

MULTIPLE BASELINE DESIGN

Trainer measures the group several times before and after the training.It helps to observe trends in performance and change in trendsThe design is the best one to use if all employees are to be trained simultaneously.

MeasureMeasureTrainMeasureMeasureMeasureMeasure

PRETEST-POSTTEST CONTROL GROUP DESIGN

A control group of employees who are very similar to the training group except that they do not receive the training for last one year.Both group are measured at least once before and after the training.It helps to analyse whether any change had occurred and whether the change had resulted from the training.

TrainMeasureMeasureMeasureMeasureNo TrainingGroup 1Group 2

Models of Training Evaluation

MODELSKirkpatrick Four Level ApproachKaufmans Five Level ApproachCIRO ApproachPhilips Five Level ROI Framework

KIRKPATRICK FOUR LEVEL APPROACH

Donald Kirkpatrick developed in 1959.Four Levels Level 1: Reaction - a measure of satisfaction Level 2 : Learning - a measure of learning Level 3 : a measure of behavioural change Level 4 : Results a measure of results

REACTIONSimply assess the participants reaction to training and programme.Did they like the programme, the trainer and course material.Did they feel that they had learned somethingApproach helps assesses the perceived value of participantsSurvey ,simple evaluation sheet (smile sheets, happy sheets)

LEARNINGLearning objectives are met or notMeasuring the extend which principles ,facts, techniques ,skills have been acquiredPretest and posttest and score compilation

BEHAVIOURNew knowledge and skill are retainedSkills and Knowledge are applied in job for improved behaviourDirect Observation at specific intervalsBehavioural scorecards

RESULTSBusiness ImpactPretest-Posttest Control Group Design

LevelMeasureKey questionIndicators/MethodologiesLimitationsReactionSatisfactionWhat was the participants reaction to the programme ?Programme evaluation sheets,interviews,questionnaire and general participants comments throughout the programmeCant measure what is learned,gurantee behavioural change, nor ensure quality result from learningLearningKnowledgeWhat did the participant learnPre-post testing,obseravtions,focus groups,interviews,survey people who observe the performer, questionnairesWill not measure the likeness of programme,if they will behave differently,and if results will be achieved

LevelMeasureKey questionIndicators/MethodologiesLimitationsBehaviourTransfer of learning and achievement performance objectivesDid the participants learning affect their behaviourPre-post testing,observations by trainers,managersand peers,team and self assessment,interviews and surveysCant determine if participants liked the training and if the behaviour accomplished resultsResultsTransfer or impact on workplaceDid participants behaviour changes affect the organisationProductivity ,profitability, sales and profit, decreased employee turnover and costCannot determine if participants like the training ,understood it or if it affected their previous behaviour

SOME EXAMPLESSales Training measure change in sales volume, customer retention, length of sales cycle, profitability on each sale after trainingTechnical training reduction in calls to help desk, reduced time to complete report, forms, or tasks or improved use of software or systems

SOME EXAMPLESQuality training reduction in number of defect itemsSafety training reduction in number or severity of accidents

KAUFMAN'S FIVE -LEVEL

LevelEvaluation Focus1b.ReactionMethods, means and process acceptability and efficiency1a.EnablingAvailability and quality of human, financial and physical resource input2.AcqusitionIndividual and small mastery and competency

KAUFMAN'S FIVE -LEVEL

LevelEvaluation Focus5.Societal outcomesSocietal and client responsiveness, consequences and payoffs.4.Organisational outputOrganisational contributions and payoffs3.ApplicationIndividual and small group (products) utilisation within the organisation

CIRO APPROACHDeveloped by Warr , Bird, RachamC- Context evaluationI Input evaluationR Reaction EvaluationO Outcome Evaluation

CONTEXT EVALUATION

involves obtaining and using information about the current operational situation or context to determine training needs and objectivesUltimate objectives ( particular deficiency in the organisation will eliminate or overcome)Intermediate objectives ( the changes in employees work behaviour that will be necessary for the ultimate objectives to be attained)Immediate objective ( the new skill ,knowledge or attitude that employees must acquire to change their behaviour and reach the immediate objective)

INPUT EVALUATIONObtaining and using information about possible training resources to choose between alternative inputs to training.Assessing the resource available

REACTION EVALUATIONParticipants reaction to improve the HRD process

OUTCOME EVALUATIONFour stages for outcome evaluation Defining trend objectives Selecting or constructing some measures of those objectivesMake the measurement at appropriate timeAssessing the results and using them to improve later.

PHILIPS FIVE LEVEL ROI FRAMEWORKReaction measuring the participants reaction to the programme and outlines specific plans for implementationLearning Measure skills, Knowledge ,attitude changesJob Applications Measuring the change in behaviour on the job and specific application of the training material

PHILIPS FIVE LEVEL ROI FRAMEWORKBusiness Results Output, quality ,costs, time and customer satisfactionROI Comparing the monetary benefits from the programme with its costs.

ROIReturn on investment is the order of the day.ROI is expressed as a percent of cost benefit ratio% ROI = Benefits/Costs * 100Another approach is pay back calculation how much time it will take before the benefits of the training match the costs and the training pay for itself.

ROIPay back Period = Costs/monthly benefits

TRAINING COSTSDesign and Development CostsPromotional costsAdministration costsFaculty costsMaterial costsCost of facilitiesParticipation costsEvaluation Costs

TRAINING BENEFITSLabour SavingsProductivity IncreasesOther cost savingsOther income generations

Duration of Training 33hrsEstimated number of Participants 750Period over which benefits are calculated 12 monthsCostsDesign and development Rs 40,930Promotion Rs 4,744Administration Rs 12,713Faculty Rs 86,250

Materials15,000Facilities40,500Participation553,156Evaluation872Total Costs754,165BenefitsLabour Savings241,071Productivity Increases675,000

Other cost savings161,250Other income generation0Total benefits1,077,321Return on Investment143%Payback period8 months

InputResourcesImmediate OutcomesParticipants SkillAnd Knowledge IncreaseIntermediate OutcomesParticipants Use their skillsAnd knowledge In their WorkplaceContextUltimate OutcomesParticipants areMore effective Or efficient In their jobActivitiesDeliver TrainingOutputTraining SessionsEffectiveness and Economy of Training

TRAINING DELIVERABILITYEconomyQualityConvenience

ECONOMY

Lower upfront implementation costsReduced content development costsLow managing costsReduce time away from jobReduce travel costs

QUALITYConsistencyBlended LearningEmployee empowermentQualification and standard basedEnhanced Loyalty and commitment.

CONVENIENCEAutomatic generation of Training recordsFlexibility, StabilityLearning fit with personal goalsPersonal supportTailored Training resources

CRITERIA OF SUCCESSNumbersDirect costsIndirect costsEfficiencyPerformance to scheduleIncome receivedThe extend to which trainees mix

CRITERIA OF SUCCESSReactionsLearningBehavioural changePerformance change