Research to show results! Wendi Tatum MEd. Kentucky Farm Bureau Mutual Insurance Companies
Research to show results!
Wendi Tatum MEd. Kentucky Farm Bureau Mutual Insurance Companies
� What is your company doing for research in the training process?
� How do you evaluate the impacts of a training method? Do you conduct evaluations for all programs?
� Who sees that information? What is it used for?
� How do you know that you have made a positive change in the learners’ behavior or knowledge base?
� Jay needs training for
some Customer Service
Representatives. He
comes to you for training.
You have an idea about
what will help. You want to
be certain that this will
work…
EvaluateTest
HypothesisGather Feedback
and Post-test
Form
Hypothesis Pre-test
Form
Hypothesis
Turn your hypothesis into a question
� Find data sources to collect.
� Identify your sample (control vs. test
group)
Pre-test
�Method 1 : Pre- and Post- tests with
one group of learners.▪ Administer pre-test to the group
�Method 2: Two groups of learners, one
control group and one group with the
training method
▪ Administer pre-test to control group
▪ Administer pre-test to the test group
Test
Hypothesis
Kirkpatrick Review Evaluate
1. Conduct and
document a needs
assessment.
�2. Develop learning
objectives for each
training topic.
4. Develop evaluation
criteria.
�5. Set targets for
evaluation types. Some
examples: 100% L1, 80%
L2, 25% L3, 5-10% L4
�7. Track the data
Conduct Cost/ Benefit ROI
8. Communicate the
process and
incorporate into the
management system.
Steps to create an
effective
evaluation process
� Action Research
� Does _______ solve the problem or
validate a theory (educated guess) or
hypothesis?
� T-test
� The t-test is probably the most commonly
used procedure for hypothesis testing.
� Actually, there are several kinds of t-tests, but
the most common is the "two-sample t-test“ or
“students t-test” or the "independent samples t-
test".
� http://web.mst.edu/~psyworld/texample.ht
m
� http://ccnmtl.columbia.edu/projects/qmss/t
he_ttest/twosample_ttest.html
� http://www.markettruths.com/process/exa
mples/ttest.asp
� Problem: Freshman students use of grammar and writing conventions.
� Rationale: 2003-2004 ISTEP+ results below average in writing.
� Plan: Integrate rubrics into the ninth grade OLC curriculum.
� Question: Will rubrics improve students post-test scores and correct poor grammar?
� Data Collection: Writing Samples, Rubric Assignments, pre- and post- test data (4 week period)
http://www.graphpad.com/quickcalcs/ttest1.cfm
� Isolate one item.
� Anticipate and plan for
glitches. They will
happen.
� Set targets for your
evaluations .
� Track your data.
� Communicate the
process and incorporate
into the management
system.
EvaluateGather Feedback
and Post-test
Form
Hypothesis
Test
Hypothesis
Pre-test
Empirical Analysis for Training
Think about a problem
related to technology or training that you are
trying to solve at your
company right now..
List your problem
______________
Identify the ProblemIdentify the ProblemIdentify the ProblemIdentify the Problem
Learning Experience
Evaluation
Empirical Research and
Training
Did the presentation meet your
expectations?
1. Yes, it exceeded my
expectations
2. No, it didn’t meet my
expectations
Did you learn new information
in this session?
1. Yes
2. No
I will go back and implement pre-
and post- tests.
Yes No
I am
alre
ady
doin
g th
is
100%
0%0%
1. Yes
2. No
3. I am already doing this
I will go back and implement a t-
test with my pre- and post- tests.
1. Yes
2. No
3. I am already doing this
The length of presentation
was..?
1. just right
2. too long
3. too short
The presenter was
knowledgeable, well prepared
and presented material clearly
and effectively?
1. Strongly agree
2. Agree
3. Disagree
4. Strongly Disagree
Please feel free to leave any other
comments on the card in front of
you.
Contact: