Evaluation of EHDI Programs ________________________ Terry Foust, Au.D., CCC-A/SLP Karen Muñoz, Ph.D., CCC-A Kathleen Watts, M.S. National Center for Hearing.

Evaluation of EHDI Evaluation of EHDI ProgramsPrograms

________________________________________________

Terry Foust, Au.D., CCC-A/SLPTerry Foust, Au.D., CCC-A/SLP

Karen Muñoz, Ph.D., CCC-AKaren Muñoz, Ph.D., CCC-A

Kathleen Watts, M.S.Kathleen Watts, M.S.National Center for Hearing Assessment & National Center for Hearing Assessment &

ManagementManagement

Utah State UniversityUtah State University

Why Evaluate?Why Evaluate?

AcountabilityAcountability Demonstrate program improvement Demonstrate program improvement

over timeover time Demonstrate evidence based practice Demonstrate evidence based practice Demonstrate attainment of program Demonstrate attainment of program

goals (1.3.6)goals (1.3.6) Required by grants, funding agenciesRequired by grants, funding agencies

Regular and continuous evaluationRegular and continuous evaluation Identifies areas to work on for Identifies areas to work on for

improvementimprovement

Identify Star PerformersIdentify Star Performers

Levels of EvaluationLevels of Evaluation

ScreeningScreening DiagnosticDiagnostic Early InterventionEarly Intervention State ProgramState Program Financing the programFinancing the program

Screening LevelScreening LevelStatisticsStatistics

Number of live birthsNumber of live births Number and percent screenedNumber and percent screened Number and percent missedNumber and percent missed Number and percent referredNumber and percent referred

Information needed to Information needed to identify possible solutionsidentify possible solutions

Equipment at each hospitalEquipment at each hospital Screening protocolScreening protocol ScreenersScreeners Screener trainingScreener training Designated program coordinatorDesignated program coordinator Notification of parent/physician/stateNotification of parent/physician/state Tracking protocolTracking protocol

Diagnostic LevelDiagnostic LevelStatisticsStatistics

Number and percent obtaining Number and percent obtaining outpatient testingoutpatient testing

Number and percent identified with a Number and percent identified with a hearing losshearing loss

Average time lag between screening Average time lag between screening and diagnostic testingand diagnostic testing

Average age of identificationAverage age of identification Number and percent lost to follow-upNumber and percent lost to follow-up

Information needed to Information needed to identify possible solutionsidentify possible solutions

Number of pediatric test sites with Number of pediatric test sites with appropriate equipmentappropriate equipment

Location of test sitesLocation of test sites Audiologist compliance with Audiologist compliance with

reporting proceduresreporting procedures Protocol for linkage with EIProtocol for linkage with EI

Early Intervention LevelEarly Intervention LevelStatisticsStatistics

Number and % enrolled in EINumber and % enrolled in EI Number and percent lost to follow-upNumber and percent lost to follow-up Average age enrolled in EIAverage age enrolled in EI Average age fit with amplificationAverage age fit with amplification Average time lag between diagnosis Average time lag between diagnosis

and interventionand intervention

Information needed to Information needed to identify possible solutionsidentify possible solutions

Timeliness of referrals to EI following Timeliness of referrals to EI following diagnosisdiagnosis

Timeliness of intervention following Timeliness of intervention following enrollmentenrollment

Training/knowledge of EI case Training/knowledge of EI case managers on issues related to hearing managers on issues related to hearing lossloss

Reporting protocol from EI to State Reporting protocol from EI to State EHDI coordinatorEHDI coordinator

State Program LevelState Program LevelSupportSupport

Parent-to-parent family supportParent-to-parent family support Dissemination of resources for familiesDissemination of resources for families Availability of medical home for all Availability of medical home for all

newbornsnewborns Availability of cultural and linguistic Availability of cultural and linguistic

family supportfamily support Performance feedback from state to Performance feedback from state to

hospitalshospitals

Tracking and Quality Tracking and Quality AssuranceAssurance

Program improvement over timeProgram improvement over time Identification of expected number of Identification of expected number of

infants with hearing lossinfants with hearing loss Attainment of program goals (1.3.6)Attainment of program goals (1.3.6) Existence of periodic and ongoing Existence of periodic and ongoing

evaluation, action plan, and monitoringevaluation, action plan, and monitoring Identification & implementation of Identification & implementation of

educational needseducational needs

Procedural Issues – keeping your Procedural Issues – keeping your head above water!head above water!

Hospital Site VisitsHospital Site VisitsEVALUATION CRITERIA

INPATIENT SCREENING ___ >98% of infants born are screened ___ >90% of infants screened pass initial screening ___ All births recorded into tracking system (Hi*Track)

OUTPATIENT RESCREENING ___ >90 of outpatients are rescreened prior to two months of age ___ 100% of infants needing outpatient screening are scheduled prior to discharge ___ PCP notified of need for follow up rescreening

DIAGNOSTIC FOLLOW UP ___ Diagnostic evaluations are completed by 3 months of age ___ All infants identified with hearing loss will receive appropriate medical, audiologic and

educational intervention services before 6 months of age ___ Results of diagnostic evaluations are reported back to the State ___ Physicians are notified of positive identifications – all infants with hearing loss will have

a medical home

FAMILY SUPPORT ___ Families of children positivley identified are referred to early intervention ___ All families receive culturally competent family support

REPORTING ___ Hospital participates in States EHDI tracking and surveillance System to minimize loss to

follow up. ___ Completes data transfer by required date as per state (ie. 10th of each month) ___ Reports transferred babies ___ Provides timely data updates on individual babies as requested

TRAINING ___ Program has competency guidelines for screeners ___ Trainers are experienced and competent according to program standards for screeners ___ Coordinators attend coordinator meetings References: T. Foust, MCHB, CDC and JCIH components/standards for evelaution, IHC evaluation standard

Montana Universal Newborn Hearing Screening

Program Evaluation

Hospital: _____________________________________ Date of Evaluation: ________________________________ Rating: Excellent = Successfully completes 100% of criteria during evaluation period Good = Successfully completes 75% of criteria during evaluation period Fair = successfully completed 50% of criteria during evaluation period Unsatisfactory – Successfully completed <50% of criteria during evaluation period

Inpatient Screening ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Outpatient Screening ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Diagnostic Follow-up ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Reporting ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Training ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Program Summary ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Information needed to Information needed to identify possible solutionsidentify possible solutions

Existence of financial support for Existence of financial support for diagnostic testing, and amplificationdiagnostic testing, and amplification

Existence of a hearing aid loaner programExistence of a hearing aid loaner program Mechanism for: Mechanism for:

Informing familiesInforming families

Informing practitionersInforming practitioners

Dissemination of materialsDissemination of materials

How to EvaluateHow to Evaluate

Hospital surveysHospital surveys Physician surveysPhysician surveys Parent surveysParent surveys Patient focus groupsPatient focus groups Getting patients back for follow up Getting patients back for follow up Data AnalysisData Analysis Hospital site visitsHospital site visits

Hospital SurveyHospital Survey

Physician SurveyPhysician Survey

Parent SurveysParent Surveys

Getting Patients Back for Getting Patients Back for Follow UpFollow Up

How are programs getting patients How are programs getting patients back for follow up?back for follow up?

Audiologists are keyAudiologists are key

Data AnalysisData Analysis

Importance of tracking software Importance of tracking software Is it flexible enough to generate a Is it flexible enough to generate a

wide variety of program statistics?wide variety of program statistics? Is it compatible with a variety of Is it compatible with a variety of

programs and equipment?programs and equipment? ExamplesExamples

Lessons Learned From Site Lessons Learned From Site VisitsVisits

Level of commitment to program is Level of commitment to program is evidentevident

Challenges are brought to life Challenges are brought to life Importance of onsite problem solvingImportance of onsite problem solving Equipment and data management Equipment and data management

processes are clearprocesses are clear Staff input reflects program healthStaff input reflects program health

Summary Summary

Evaluation takes time and effortEvaluation takes time and effort Must be well plannedMust be well planned