8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
1/43
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
2/43
MONITORING
AND
EVALUATION
OF
TRAINING PROGRAMMES
P.PRASHANTHPhD SCHOLAR
Dept. of Agricultural ExtensionI.D. No. RAD/11-10
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
3/43
2 WHAT IS MONITORING?
5 FOLLOW UP TRAINING PROGRAMME
MODELS, TYPES, LEVELS OF TRAINING EVALUATION4
IMPORTANT MONITORING TECHNIQUES 3
.
6 CONCLUSION
WHAT IS TRAINING NEED ANALYSIS?1
Presentation Outline
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
4/43
TRAINING
MANAGEMENTCYCLE
3Implementation
4Monitoring
1
Problems & Training needs analysis
5
Evaluating
2Planning, Designing
and Organising
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
5/43
MONITORING & EVALUATION
OF TRAINING PROGRAMME
Assessment and control of
the progress of trainingprogramme for its effective
organisation
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
6/43
A continuous or periodic review and surveillance
(overseeing) by management at every level of the
hierarchy of the implementation of an activity to
ensure that input deliveries, work schedules,targeted outputs and other required actions are
proceeding according to plan
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
7/43
Indicators are defined as specific (explicit) and objectivelyverifiable measures of changes or results brought out by anactivity. Monitoring indicators are required to be :
Valid-they should actually measure what they are supposed tomeasure;
Reliable--verifiable or objective-conclusions based on them
should be the same It measured by different people at different times and under
different circumstances;
Relevant-they should be relevant to project objectives
Sensitive-they should be sensitive to changes in the situationbeing observed;
Specific-they should be based on available data; Cost effective the results should be worth the time and money it
costs to them; and
Timely it should be possible to collect the data reasonablyquickly.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
8/43
Training performance has dimensions, namely
(i) Participation (commitment to training), (ii) Utility (used of training to trainees), and
(iii)Effectiveness (usefulness of training to ext
organisation)
Based on these three dimensions, the followingthree training performed indicators are suggested.
TPI
TUI
TEI
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
9/43
Assessment of training needs
Pre tests
Daily- tests
Alternate -day- tests
Weekly tests
Recall sessions
Fortnightly testsReviews
Back home application
IMPORTANT MONITORING TECHNIQUES
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
10/43
Evaluation is a process of investigating how far the objectives
of the training programme have been achieved.
It also involves analyzing the strengths and weakness of the
programme, reactions of the participants, and its over all impact
on behavioural change and job performance of the participants Attempt to obtain information (feedback) on the effects of a training
programme
To assess the value of the training in the light of that information
A systematic processes of collecting information for and about a
training activity which can then be used forguiding decision-
making and forassessing the relevance and effectiveness of
various training components.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
11/43
Formative training evaluation -Pertains to the
implementation stage of the training programme and helps
in day-to-day decision-making.
Summative training evaluation relates to final
evaluation of the objectives of the training programme
Ongoing training evaluation, as the term indicates, is the
evaluation undertaken during the course of a training
activity.
Ex post training evaluation is the evaluation undertaken
after completion of a training project, usually after full
project benefits and impact are expected to have been
realised.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
12/43
WHY DO YOU
EVALUATE ?
1. To see what has been achieved
2. To measure the progress
3. To improve
monitoring
4. To strengthen th
programme
5. To see whether
the effort is
effective6. To see cost
benefit ratio
7. To collect information
for planning and
managing programme
8. To share
experiences
9. To improve
effectiveness
to have more
impact
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
13/43
Assessment of training needs
Development of training objectives
Development of course design
Developing training methodology
Organizational climate for setting training
Actual conduct of training course
Monitoring & evaluation of training course
Follow -up of training programme
ESSENTIAL AND IMPORTANT COMPONENTS
OF TRAINING PROGRAMMES
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
14/43
Time of planning
Quality of training course
Coverage of the syllabus/content
Extent of use of teaching aids
Mid term reviews
Training methods Training skills
Field visits/study tours
Use of Audio-Visual equipment
ESSENTIAL ITEMS TO JUDGE THEEFFECTIVENESSOF A TRAINING
PROGRAMME
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
15/43
Training faculty/technicians
Availability of funds for training
Change in the behavior of trainees
Field applicability
Trainees opinion
Daily / alternate / weekly tests
Lodging and boarding facilities
Pre and post evaluation
ESSENTIAL ITEMS TO JUDGE THEEFFECTIVENESSOF A TRAINING
PROGRAMME
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
16/43
KNOWLEDGE SKILLS
ATTITUDE
FACTORS TO BE MEASURED IN
TRAINING EVALUATION
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
17/43
Objective-oriented Training Evaluation
Decision-oriented Training Evaluation
Transactional Training Evaluation
Goal-Free Training Evaluation
Adversary Training Evaluation
Illuminative Training Evaluation
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
18/43
REACTION LEARNING PERFORMANCE IMPACT
Purpose - To know training
participants attitude
towards the various
components of
training
To know the knowledge,
attitude and / or skills
gained by the participants
To know the trainees
performance in their
organisationsor place of
work
To assess the long term
effects of training on
the community or
society
Data Gathered Impression or attitude
toward the topics,
methods, aids
administrative support
Knowledge, attitude and
skills
Skills and outputs Demographic, socio-
economic Data
Sources of Data Trainees Trainees Superior and subordinate Members of the
trainees, community or
society
Tool for
gathering Data
Questionnaire Questionnaire Checklist and score card Interview schedule
Time to gather
Data
During and
immediately after
closing of training
Immediately after closing
of training or before and
after training
After trainees return to
work and / or during
training
Many years after
trainees have resumed
work or employment in
their community or
society
Person in-charge
of Data
gathering
Trainer Trainer Trainer or superior Independent evaluation
group
Use of Data Improving trainingenvironment
Improving trainingcontents, methods and
Improving skills andoutputs
Improving trainingstrategy
LEVELS OF TRAINING EVALUATION
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
19/43
The planning stage
The implementation stage
The stage immediately after
termination of training
The impact stage
TYPES OF TRAINING EVALUATION -
STAGES OF TRAINING PROCESS
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
20/43
Planning Evaluation Consists of two phases. In the first phase-TNA-
through one of the following four approaches:
Performance analysis Task analysis
Competency study
Training needs survey
The skill gap analysis
In the second phase, an assessment ofinstructional methods is done
The most appropriate training methodology andinstructional strategy is worked out.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
21/43
It helps in solving the problems arise in implementation ofprogramme
According to Raab et al. (1987) as reported by Misra (1990),
"process evaluation is performed to detect or predict defects in the
procedural design of a training activity during the implementation
phase".
Investigation is made on the following:
Monitoring of training to know whether it goes on as per the plan.
Appropriateness of training methods or audio visual aids used.
Effectiveness of delivery or presentation
Effectiveness with regard to proper mix of theory and practical
Changes in the knowledge, skill and attitude among participants
Satisfaction of trainees with boarding and lodging facilities
Effective utilization of finance
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
22/43
Concerned with learner performance
Analyzed by two approaches i.e.
Norm-referenced and criterion-referenced.
Under the norm- referenced evaluation,The gains in learning are determined based
on pre and post training scores.
The criterion referenced evaluation is
Related to investigation of what was taughtand what was learned.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
23/43
Refers to assessing the programme impact on the Jobperformance of the participants.
It attempts to find answer to the following questions:
What are the improvements in the performance of job asthe results of training
What are the benefits accrued to the participants andorganisations as the result of improved job performance?
What are the problems in applying the skills learned in realwork situation?
What can be done to enhance the application of newly
learnt knowledge and skill in the work place? It can be carried out at two levels
At the level of immediate objectives
At the level of long-range objectives
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
24/43
TRAINING EVALUATION ACCORDING TO STAGES OF TRAINING PROCESS
EVALUATION FOR
PLANNING
PROCESS EVALUATION TERMINAL
EVALUATION
IMPACT
EVALUATION
What is it? Instructional strategies
and means
Feedback from trainees and
others regarding training
Effectiveness of
training after its
completion
a) Immediate objectives
b) Long range
objectives
Who evaluates? Trainers Trainers, also external
evaluators
Trainers, also external
evaluators
a) Trainers
b) Specialized units ofthird party evaluators
When evaluated? Before and during
training
During training After completion of
training
After completion of
training
Methods of
evaluation?
1.Determining most
appropriate instructionalprocedures
2.Pilot testing
i) Pre-assessment
ii) Interim assessment
iii) Other sources of
assessment information
i) Measurement of
change intrainee(KSAOs)
ii) Measurement of
trainees perceptions
about training
i) Trainee follow ups
ii) ii) Employer
surveys
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
25/43
Decide on the method of evaluation
Decide on the type of evaluation
Plan the evaluation
Prepare the evaluation tools
Implement the evaluation
Report the evaluation results
Utilize the evaluation results
STEPS IN EVALUATION
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
26/43
Step 1: Identify and described the training to be evaluated. Step 2 : Identify and clarify the purposes for the evaluation.
Step 3: Identify the customers of training and evaluation and
clarify their needs for information.
Step 4 : Determine the key questions that the evaluation mustaddress.
Step 5 : Determine the over all design approach that will best
meet customer needs and answer the evaluation questions
identified.
Step 6 : Identify the best data collection sources and procedure
to address the evaluation questions.
Step 7 : Plan reporting and communication actions
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
27/43
(i)Constraints imposed by time dimensions as the sponsors arealways in a hurry to get the quick results for decision makingand other uses,
(ii) Constraints of ethical necessity for continuous feedback;which remains a costly affair,
(iii) Constraints imposed by disciplinary boundaries whichhave separate evaluation methodologies,
(iv) Constraints of openness of the social system which theproject is operating, thereby creating problem of control andmanipulation at will,
(v) Constraints of long-standing tensions between theprofessional personnel and evaluators who tend to view eachother with suspicion and even hostility. However, theseconstraints are showing signs of easing,
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
28/43
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
29/43
Activities related to finding out the utility of thelearning outcome of the programme in job
performance
Analysis of reasons for non-utilization of new
skills or knowledge acquired Providing help and reinforcement for application
of learning as well as assessing future training
requirements
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
30/43
To what extent the content covered /
taught during a particular traininghave been adopted by the trainees /
sponsoring institutions
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
31/43
To find out the impact on individual job performanceand the organisational environment is essential tojustify the continuance of the programme in future
Feedback about the utility of programme from the
trainees after they resume their job and theirexperimentation with new knowledge and skills learnt
is essential to validate the impact of the programme
The feedback from the participants will greatly help
in making suitable changes in the contents andmethodology of the programme
To tell us about the quality of training and the effect
that training has created on the learners
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
32/43
To know the extent of learning by the trainees
To know the application of the content covered
during the training in field situation
To find out reasons for poor
TOLearning
To be aware of the problems in adoption of the
technology in field situation
To know the impact of training on production level
To get the feedback about the quality of training
To know the future training requirements of clients
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
33/43
Evaluation of training Evaluation of behavioural change
Results of application
Further Personal Development
On the job Further training courses
Help in Practical Applications External services including consultancy
Help by superiors and colleagues
Liasion with former participants Personal contacts
Associations
Information, conferences and conventions
Alumnis help in training
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
34/43
Visit oftrainers to
work place
of trainees
Application
workshops
Follow up
through mailed
questionnaires
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
35/43
Decide the place of visit.
Make plan for follow-up.
Develop a interview schedule for follow-up.
Inform the trainees/sponsoring institution about
date of your visit to them.
Visit the trainees/sponsoring institution.
Observe the trainees while working.
Make notes on observation.
Prepare and submit report.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
36/43
Plan for the application workshop well in advance.
Decide date of application workshop.
Develop day-to-day programme/ design of
application workshop.
Inform/Invite trainees/sponsoring institutions well
in advance to participate in application workshop.
Conduct application workshop.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
37/43
Cheaper method of conducting the follow-
up of training Develop a detailed questionnaire based on the contents covered in a training
course.
Send the questionnaire to trainees/sponsoring institutions after 6 month/ one
year.
Write a letter to all the trainees/ sponsoring institutions to return thequestionnaire after filling it.
Send reminders to those, who have not returned the complete
questionnaires.
Compile all the questionnaires.
Process the data received.
Analyse the responses received from the trainees.
Prepare a detailed report.
Send the report to sponsoring institutions for information.
Take appropriate action on the suggestions given by the trainees for
improvement in the quality of training programmes.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
38/43
Systematic follow-up can point out weaknesses inapplication of learning so that they could be correct
and guided to apply the learning properly. It can also tell us the extent to which trainers have
transferred this learning to the work situation and theresults yielded there after.
Follow up during post training evaluation is done
during a period ranging from 3 months to 1 year afterthe training the ideal period is being six months;
In this period trainee would have got the opportunity, ifeverything including the organizational climate is inthe favour, to implement what is learned in the training
phase and Results to be visible to trainee.
To assess whether, trainees have applied their learningin the form of changed behaviour on the job.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
39/43
The percentage of getting the response of mailed
questionnaire is very less.
Some times no response is received even by
repeated reminders. Some times the questionnaire is not completely
filled up.
Some times the answer to some of the questions is
given out of concern. Some of the responses are received very late.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
40/43
Plan for Application workshop well in
advance
Decide date of Application workshop Develop day to day programme
Inform the trainees / sponsoring institute
about the Application workshop
Conduct Application workshop
Prepare and submit report
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
41/43
Develop a detailed questionnaire based on the contents coveredin a training
Send the questionnaire to trainees / sponsoring institutions after6 months / one year
Write a letter to all the trainees / sponsoring institutions to
return the questionnaire after filling it Send reminders, if necessary
Compile all the questionnaires
Process the data received
Analyze the responses received from the trainees Prepare a detailed report
Send the report to sponsoring institution for information
Take appropriate action on the suggestions
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
42/43
1. An analysis of utility of the programme.
2. Finding out reasons for non-adoption of newly acquired skills and knowledge.
3. Reinforcing the motivation and skills of the participants through:
(i) Post - training contact through regular Newsletter or Journal.
(ii) Organizing refresher courses.
(iii) Personal contact.4. Providing help for adoption of newly acquired skills through:
(i) Clarification of doubts and solving technical problems arising out of transfer of
learning.
(ii) Making arrangements for availability of needed equipments materials needed for
learning.
(iii) Providing positive feedback.
5. Encouraging communication among the trainees through:
(i) Formation of Alumni association.
(ii) Development of e-mailing list, for net work among the trainees.
(iii)Informal meetings.
8/3/2019 Monitoring and Evaluation and Followup of Training Programmes Prashanth
43/43
Thank You forur kind attention