COLLECTING FOLLOW-UP OUTCOMES EFFICIENTLY AND EFFECTIVELY · Getting permission to obtain follow-up information from school/ referral source/etc. Asking for these at the beginning

HOW DO YOU DO THIS IN THE REAL WORLD? COLLECTING FOLLOW-UP OUTCOMES EFFICIENTLY AND EFFECTIVELY

By: Sebrina Doyle, M.S.

Research Coordinator

Prevention Research Center

WHAT TYPES OF OUTCOMES DO YOU MEASURE?

What is happening (Process)? referrals attendance contact hours

How well is it being done (Quality/Fidelity)? facilitator skill session quality session fidelity participant satisfaction

What effect is it having (Short/Long-term Outcomes)? Process outcomes (goal achievement) participant outcomes (short and long-term) community outcomes (ultimate public health impact!)

BARRIERS

� No system in place � Difficulty getting a hold of family � Methods of survey collection difficult or ineffective

� Not enough staff time/resources to collect data � Lack of responses

� Needs of stakeholders do not match with what is feasible �  Inability to access data systems � Do not know what data to collect

� Do not have the tools needed

HOW DO YOU COLLECT FOLLOW-UP DATA?

�  By mail �  In person �  By phone

�  Interview �  Texting

�  By computer �  Using Survey Monkey or other online platform �  Using E-mail Remember, it is important to get creative and think outside of the box.

DO YOU HAVE A SYSTEM IN PLACE?

Who collects/tracks the data? � Therapist � Supervisor � Program Manager � Reception � Other

WHO PROVIDES THE DATA?

� Youth � Parents � Teachers/Schools � Facilitators/Volunteers � Archival � Referral Source Using multiple sources of data allows you to collect something even when you cannot collect everything.

THE BEGINNING IS AS IMPORTANT AS THE END…

� Getting buy-in for the process at the beginning � Gathering ALL contact information (multiple phone numbers, e-

mail, etc.) and getting permission to use them.

� Getting permission to obtain follow-up information from school/referral source/etc.

Asking for these at the beginning means that you do not have to worry about no-shows for treatment! All of these sources can be updated at termination if there is a closing session.

OFFERING INCENTIVES…

� Send small token “gifts” with surveys to increase completion rate (Pencils, wristbands, magnets, etc.)

� Offer entry into a “lottery” for a donated item � Others? � Most importantly, tell them WHY you are doing this. Some

people help because they like helping.

OTHER THINGS THAT HELP COMPLETION:

� Personal contact by therapist or someone they know from agency

� Handwritten rather than pre-printed envelope � Personal note on the back of the envelope � Postage paid envelope included with survey � Text or other reminder to let them know about the survey/

call � Multiple methods for data collection (mail/phone/email/etc.) � Getting others who have regular contact involved with data

collection (probation officers, schools, etc.)

DATA PITFALLS:

�  Missing data- Despite best efforts, people skip questions, fail to complete surveys, etc. What is your completion rate?

�  Asking the wrong questions- Are you focusing on knowledge and attitudes or just behavior? Are your questions directly related to program activities or outcomes?

�  Problems using data- Is your survey designed well? Can you easily understand the results you are getting?

THANK YOU! The EPISCenter is a project of the Prevention Research Center, College of

Health and Human Development, Penn State University, and is supported by funding from the Pennsylvania Commission on Crime and Delinquency and the Pennsylvania Department of Public Welfare as a component of the Resource

Center for Evidence-Based Programs and Practices.

For more information contact: Sebrina Doyle,

Research Coordinator Bennett Pierce Prevention Research Center

320E Biobehavioral Health Building University Park, PA 16802  

Phone: (814) 867-3016 Fax: (814) 865-3936

Email: [email protected] Web: www.episcenter.psu.edu

USING PARTICIPANT DATA, IS IT RESEARCH OR NOT? �  It is intended to be a contribution to “generalizable knowledge” and its

results are intended to probably be made public in some way.

�  It is not just for internal assessment or quality assurance purposes.

�  Research is usually described in a formal protocol that sets forth an objective and a set of procedures designed to reach that objective.

�  If you are in a “testing phase” for a home-grown program you may want to use an informed consent process that allows you to use the data for research.

�  Regardless of research or non-research status, you should consider protocols for securing confidential data if you do not already have one.