1 New CTE Follow-up New CTE Follow-up Survey Survey
Dec 18, 2015
22
Training TopicsTraining TopicsDiscuss PurposeDiscuss PurposeExplain Need for Good Response RateExplain Need for Good Response RateExplain Interviewer ResponsibilitiesExplain Interviewer ResponsibilitiesIntroduce the SurveyIntroduce the SurveyMock InterviewMock InterviewPractice (2 w & w/o stumbling blocks) Practice (2 w & w/o stumbling blocks) Discuss Problems Discuss Problems Consistency & ConfidentialityConsistency & ConfidentialityLessons on making calculationsLessons on making calculationsClosely MonitorClosely Monitor
33
Purpose & UsePurpose & UseProvide data to meet reporting needs.Provide data to meet reporting needs.Used forUsed for
p1p1
Perkins III Core Performance Indicators.Perkins III Core Performance Indicators.
Ranking programs for State Added Cost fundingRanking programs for State Added Cost funding
LEAs to use for program improvement & aiding students not placedLEAs to use for program improvement & aiding students not placed
Program reviewsProgram reviews
Reporting CTE program impact to publicReporting CTE program impact to public
44
The Interview Process The Interview Process Qualities of Good ImplementationQualities of Good Implementation
ConsistencyConsistency
Response RateResponse Rate
Interviewer ResponsibilitiesInterviewer Responsibilities
p6-8p6-8
55
ConsistencyConsistency
Goal =Goal = to collect data from all to collect data from all students students statewidestatewide
p6p6
in exactly the same wayin exactly the same way
at exactly the same timeat exactly the same time
Timing = 9 mo after June of senior yearTiming = 9 mo after June of senior year
66
Response RateResponse Rate
Acceptable RateAcceptable Rate
0102030405060708090
100
Followup
Good
Low
P6-8P6-8
90% - 100%90% - 100%
< 50%< 50%
Relationship to Placement RateRelationship to Placement Rate
Unacceptable RateUnacceptable Rate
77
Tips for Improving RRTips for Improving RR
Use a phone survey methodUse a phone survey method
A-CA-C
Call from the school phoneCall from the school phone
Leave messages on answering machinesLeave messages on answering machines
Make at least 6 attemptsMake at least 6 attemptsOther suggestions in Attachment COther suggestions in Attachment C
Call in evenings & on weekendsCall in evenings & on weekends
88
Interviewer ResponsibilitiesInterviewer Responsibilities
Be consistentBe consistent
BEFORE Hanging UpBEFORE Hanging Up
P9-10P9-10
Follow question wording exactlyFollow question wording exactlyBe objectiveBe objective
Follow the scriptFollow the script
Check your workCheck your work
Enjoy your workEnjoy your work
99
Follow Wording ExactlyFollow Wording Exactly
Would you climb a tree?Would you climb a tree?
p9p9
Should you climb a tree?Should you climb a tree?
Could you climb a tree?Could you climb a tree?
1010
Be ObjectiveBe Objective
WHY do you say that?WHY do you say that?
p10p10
Why do YOU say that?Why do YOU say that?
Why do you say THAT?Why do you say THAT?
1111
New CTE Follow-upNew CTE Follow-up Phone Survey Phone Survey
UpfrontUpfront OpeningOpening Student contactStudent contact Proxy contactProxy contact
Part A: Current StatusPart A: Current StatusPart B: School/TrainingPart B: School/TrainingPart C: EmploymentPart C: EmploymentPart D: Only if Not WorkingPart D: Only if Not WorkingClosingClosing
P11-22P11-22
1212
Opening – Student ContactOpening – Student Contact
Hello, this is _____ from _____. Is ____ there?Hello, this is _____ from _____. Is ____ there?
StudentStudent:: Hi _____. How are you?Hi _____. How are you?
Single Single program program completecompletedd
______, in _____ grade you ______, in _____ grade you completed a program in completed a program in ________. Do you have a few ________. Do you have a few minutes to answer some minutes to answer some questions about what you are questions about what you are doing now? All responses will doing now? All responses will be kept confidential.be kept confidential.
p11p11
1313
Student contactStudent contactMultiple Multiple programs programs completedcompleted
We are talking with students who completed We are talking with students who completed programs in high school. Do you have a few programs in high school. Do you have a few minutes to answer some questions about minutes to answer some questions about what you are doing now? All responses will what you are doing now? All responses will be kept confidential.be kept confidential.
Our records show that you completed the _____, Our records show that you completed the _____, _____, and _____ programs. Are you pursuing _____, and _____ programs. Are you pursuing one of these more than the other?one of these more than the other?
Which one? ______________________Which one? ______________________
This is the class I would like you to think about This is the class I would like you to think about during this interview.during this interview.
p11p11
If no: We can only follow up on one program. : We can only follow up on one program. Which would you like to think about during this Which would you like to think about during this interview? ____________________interview? ____________________
1414
Proxy contactProxy contact
ProxyProxy:: We are talking with students who We are talking with students who completed high school programs completed high school programs to see how they are doing. to see how they are doing. [Student name] was in the _____ [Student name] was in the _____ class. How could I reach class. How could I reach him/her?him/her?
If unreachableIf unreachable
You could probably answer some of You could probably answer some of the questions. It would only take a the questions. It would only take a couple minutes and all responses couple minutes and all responses will be kept confidential.will be kept confidential.
p11p11
1515
Part A Current StatusPart A Current Status
1. Lets start what you’re doing now. Are you going to school? 1. Lets start what you’re doing now. Are you going to school? Working?Working?
p12p12
Verify applicable part of answer with statement Verify applicable part of answer with statement before coding.before coding. YesYes No No
a. in a training program or attending school or college?a. in a training program or attending school or college?
b. working as an apprentice?b. working as an apprentice? ((If YesIf Yes: Q1c = Yes*) : Q1c = Yes*)
c. working?c. working? d. on full-time, active duty in the military?d. on full-time, active duty in the military?
((If YesIf Yes: Q1c = Yes & Q7 = 40 hrs/wk): Q1c = Yes & Q7 = 40 hrs/wk)
e. on part-time duty in the military? e. on part-time duty in the military?
((If YesIf Yes: Q1c = Yes & Q7 = 40 hrs/wk): Q1c = Yes & Q7 = 40 hrs/wk)f. Other: (specify code from list below)f. Other: (specify code from list below)
________1. unpaid leave of absence (disability, Family leave)1. unpaid leave of absence (disability, Family leave)2. on seasonal layoff2. on seasonal layoff
1616
Q1 - Skip OptionsQ1 - Skip Options
If working, but not in school or training, If working, but not in school or training, Skip to Part CSkip to Part C..
If ‘Other’ only,If ‘Other’ only,
Skip to Part DSkip to Part D..
p13p13
1717
Part B: School/TrainingPart B: School/Training
I have some questions about your schooling or training.I have some questions about your schooling or training.
2. In your major area of study or 2. In your major area of study or training, how much do you use the training, how much do you use the skills you learned in this program?skills you learned in this program? Would you say you use themWould you say you use them a lot, a lot, some, hardly ever, or not at all?some, hardly ever, or not at all?
If a 3 (Hardly ever) or 4 (Not at all): If a 3 (Hardly ever) or 4 (Not at all):
Hardly Hardly NotNot
A lotA lot SomeSome EverEver at at allall
p14p14
2a. Right now you don’t use your 2a. Right now you don’t use your training, but how much do you training, but how much do you anticipate using it in the futureanticipate using it in the future . . . . . . a lot, some, hardly ever, or not at a lot, some, hardly ever, or not at all?all?
1818
p15p15
3. Where are you going to school?3. Where are you going to school?
(Note: If enrolled in more than one, identify the (Note: If enrolled in more than one, identify the primaryprimary one.) one.)
business or trade school, or career business or trade school, or career centercenter community collegecommunity college
college/universitycollege/university
military military (National Guard, West Point, Annapolis, (National Guard, West Point, Annapolis, DLI, Merchant Marines, etc.)DLI, Merchant Marines, etc.)
other ___________________________other ___________________________
1919
p16p16
4. What type of program are you in?4. What type of program are you in?
[Read list][Read list]
certificatecertificate
associate degreeassociate degree
bachelor’s degreebachelor’s degree
other (specify) ___________________other (specify) ___________________
2020
Part B – Skip OptionsPart B – Skip Options
If No Current job, If No Current job, Skip to Part DSkip to Part D..
p16p16
2121
Part C: EmploymentPart C: Employment
Now, I’d like to talk about your job as it Now, I’d like to talk about your job as it relates to your relates to your (selected) (selected) program. program.
p17p17
5.5. On your job, how much would you say On your job, how much would you say you’re using the skills you were taught?you’re using the skills you were taught?
Would you say you use them . . . ?Would you say you use them . . . ?
A lot A lot Some Some Hardly ever Hardly ever Not at Not at allall
2222
p18p18
6. How strongly do you agree with the 6. How strongly do you agree with the statement, I am satisfied with my statement, I am satisfied with my present job? Do you . . . ?present job? Do you . . . ?
Strongly agreeStrongly agree
AgreeAgree
DisagreeDisagree
Strongly disagreeStrongly disagree
2323
p18p18
7.7. How many hours a week do you How many hours a week do you work?work?
______________ hrs/week ______________ hrs/week
7a.7a. If not specificIf not specific::
Would you say it’s 35 hrs a week or Would you say it’s 35 hrs a week or more?more?
Yes Yes NoNo
2424
p19p19
8.8. Including tips and commissions, how Including tips and commissions, how much do you make an hour?much do you make an hour? $ $ _________ / hr_________ / hr
Skip to CommentsSkip to Comments
2525
Part D: Only if Not WorkingPart D: Only if Not Working
9. Are you currently looking for a 9. Are you currently looking for a job? job?
YesYes No No
p22p22
2626
CommentsComments
Thank you. That’s all I needed.Thank you. That’s all I needed.
p22p22
Now, are there any comments Now, are there any comments you’d like to make to help us you’d like to make to help us improve our programs.improve our programs.
2828
Proxy SurveyProxy Survey
Part A: Current StatusPart A: Current Status1.1. Let’s start with what (she/he) is Let’s start with what (she/he) is
doing now.doing now.
pp
Part B: School/Training Part B: School/Training 3. Where is (he/she) going to school?3. Where is (he/she) going to school?
4.4. What type of program is (he/she) What type of program is (he/she) in?in?
2929
Part C: EmploymentPart C: Employment7. How many hours a week does
(he/she) work?
7a. 7a. If not specificIf not specific: Would you say it’s : Would you say it’s 35 35 hours a week or more?hours a week or more?
8.8. Including tips and commissions, Including tips and commissions, how how much does (he/she) make an much does (he/she) make an hour?hour?
Skip to CommentsSkip to Comments..
Part D: Only if Not Working Part D: Only if Not Working 9. Is (he/she) currently looking for a job?
pp
3030
ConsistencyConsistency
Goal =Goal = to collect data from all to collect data from all students students statewidestatewide
in exactly the same way, in exactly the same way,
at exactly the same time.at exactly the same time.
p6p6
3131
Prefaces to Avoid UsingPrefaces to Avoid Using
Apologetic – If you don’t mind my asking…Apologetic – If you don’t mind my asking…
Blameless – This is confidential, but the Blameless – This is confidential, but the state needs it.state needs it.
Estimates – Approximately how much … Estimates – Approximately how much …
On guard! – I have to ask this question.On guard! – I have to ask this question.
Proxy – You may not be able to answer…Proxy – You may not be able to answer…
Opt out – You don’t have to answer … Opt out – You don’t have to answer …
p20p20
3232
Role Play #1Role Play #1Practice Conducting the SurveyPractice Conducting the Survey
Choose a partner.Choose a partner.
Go to a room Go to a room
Practice both roles – the interviewer and Practice both roles – the interviewer and the student.the student.
Practice Practice at least three timesat least three times..
Switch roles with your partner.Switch roles with your partner.
Be KIND. Do not throw in stumbling Be KIND. Do not throw in stumbling blocks.blocks. Back in 15 minBack in 15 min
pp
3333
Potential Problem & Solution #1Potential Problem & Solution #1
Where are you going to school?Where are you going to school?
I take a class at the career center.I take a class at the career center.
?? Are you registered at the career center?Are you registered at the career center?
I’m registered at the community college, I’m registered at the community college, but they hold the class at the career but they hold the class at the career center.center.
3434
Potential Problem & Solution #2Potential Problem & Solution #2On your job, how much would you say On your job, how much would you say you’re using the skills you were taught?you’re using the skills you were taught?
Student from automechanics program Student from automechanics program working at a dealership says “not at all.”working at a dealership says “not at all.”
? ? What should the interviewer do?What should the interviewer do?
a. Mark the answer and go on.a. Mark the answer and go on.
b. Review skills they learned & ask again.b. Review skills they learned & ask again.
c. Explain why that’s not possible & ask again.c. Explain why that’s not possible & ask again.
3535
Potential Problem & Solution #3Potential Problem & Solution #3How strongly do you agree with the statement, “I How strongly do you agree with the statement, “I am satisfied with my present job”?am satisfied with my present job”? Student working at McDonald’sStudent working at McDonald’s
““Strongly Agrees”.Strongly Agrees”.
?? What should the interviewer do?What should the interviewer do?
a.a. Review potential for the program they took.Review potential for the program they took.
b.b. Talk about benefits & ask again.Talk about benefits & ask again.
c.c. Mark the answer and go on.Mark the answer and go on.
3636
Potential Problem & Solution #4Potential Problem & Solution #4
Including tips and commissions, how much Including tips and commissions, how much do you make an hour?do you make an hour?
The completer has more than 1 job.The completer has more than 1 job.
?? What should the interviewer do?What should the interviewer do?
Combine and estimate the overall Combine and estimate the overall hourly wage.hourly wage.
3737
Potential Problem & Solution #5Potential Problem & Solution #5
Including tips and commissions, how much do Including tips and commissions, how much do you make an hour?you make an hour?
The interviewer is uncomfortable about The interviewer is uncomfortable about asking the question.asking the question.
?? What should the interviewer do?What should the interviewer do? a.a. Apologize, say “I know this is sensitive, Apologize, say “I know this is sensitive, butbut
b.b. Ask for an estimate, “About how much Ask for an estimate, “About how much … …
c.c. Explain it’s optional.Explain it’s optional. Just ask the question.Just ask the question.
3838
Role Play #2Role Play #2Practice Conducting the SurveyPractice Conducting the Survey
Choose a partner/new partnerChoose a partner/new partner
Go to a roomGo to a room
Practice both roles (interviewer & Practice both roles (interviewer & responder)responder)
Practice both surveys (phone & proxy)Practice both surveys (phone & proxy)
Do the survey Do the survey twicetwice (2 each) (2 each)
Go ahead -- throw in Go ahead -- throw in stumbling blocksstumbling blocks
Back in 15 minBack in 15 min
3939
Handling Difficult CallsHandling Difficult Calls
TalkativeTalkative
A-CA-C
GrouchyGrouchy
IrateIrate
4141
ConfidentialityConfidentialityPart of the 2 Big CsPart of the 2 Big Cs
What was the first C? What was the first C?
MeaningMeaningA promise by interviewers who know A promise by interviewers who know the response of specific completers the response of specific completers not to reveal those responses to not to reveal those responses to anyoneanyone..
Keeping responses a secretKeeping responses a secret
ConsistencyConsistency
pp
4242
CalculationsCalculations
Weekly rate based on 40 hr/wkWeekly rate based on 40 hr/wk$400/wk $400/wk ÷ 40hr/wk = $10.00/hr÷ 40hr/wk = $10.00/hr
p19p19
Monthly rate based on 2080 hr/yrMonthly rate based on 2080 hr/yr$1200/mo x 12mo = $14,400/yr$1200/mo x 12mo = $14,400/yr$14,000/yr ÷ 2080 hr/yr = $6.92/hr$14,000/yr ÷ 2080 hr/yr = $6.92/hr
Annual rate based on 2080 hr/yrAnnual rate based on 2080 hr/yr$18,000/yr ÷ 2080 hr/yr = $8.65/hr$18,000/yr ÷ 2080 hr/yr = $8.65/hr
4343
Strategies for Making ContactStrategies for Making ContactChallengesChallenges
Updated recordsUpdated recordsMobility of studentsMobility of studentsWhen to callWhen to call
The Telephone Consumer Protection Act The Telephone Consumer Protection Act states that telephone solicitation cannot states that telephone solicitation cannot occur before 8:00 a.m. or after 9:00 p.moccur before 8:00 a.m. or after 9:00 p.m
Schools technically are not solicitorsSchools technically are not solicitors
State still recommends keeping within the State still recommends keeping within the 8 a.m. - 9 p.m. time frame8 a.m. - 9 p.m. time frame
p8p8
4444
Challenges (cont’d)Challenges (cont’d)
Answering MachinesAnswering Machines
Caller IDCaller ID
Reaching StudentsReaching Students
DisconnectsDisconnects
Callbacks Can’t Get ThroughCallbacks Can’t Get Through
p9p9
4545
Solving ProblemsSolving Problems
No finite list of problemsNo finite list of problemsSome may be area specificSome may be area specificSSC will help with solutionsSSC will help with solutions
Careful about making snap decisionsCareful about making snap decisionsAll solutions are not easyAll solutions are not easyRecord enough info – no need to recallRecord enough info – no need to recall
Keep own listKeep own listExperience in handlingExperience in handling
4646
Process ProblemsProcess Problems
ContactContact
Dr. Jill KrollDr. Jill Kroll
Office of Career and Office of Career and Technical PreparationTechnical Preparation
(517) 241- 4354(517) 241- 4354