Program Evaluation: Using National TB Indicators … · TM. Presentation Outline • Describe NTIP and the National TB Program Objectives • Understand importance of NTIP in TB control

TM

Program Evaluation: Using National TB Indicators Project (NTIP)

Judy Gibson, MSN, RNProgram Evaluation Team (PET)

Field Services and Evaluation Branch (FSEB)Division of Tuberculosis Elimination (DTBE)

NCHHSTP/CCID/CDC

TB/Refugee Nurse Training Commonwealth of Virginia 2009

TM

Presentation Outline

•

Describe NTIP and the National TB Program Objectives

•

Understand importance of NTIP in TB control and program evaluation (PE)

•

Describe CDC/DTBE Program Evaluation (PE) approach

TM

NTIP and National TB Program Objectives

TM

TM

What is NTIP?

•

A program monitoring system•

Indicator reports-

inform progress toward national objectives

-

focus program evaluation efforts-

provide performance targets as benchmark for

assessment

•

http://www.cdc.gov/tb/publications/factsheets/stat istics/NTIP.htm

TM

Data Sources for NTIP

NTIP

Cases(RVCT*)

USCensus

Contacts(ARPE*)

Immigrants/Refugees(EDN*) TB GIMS*

*RVCT= Report of Verified Cases of Tuberculosis*ARPE= Aggregate Reports of Program Evaluation*EDN= Electronic Disease Notification System *GIMS= Genotyping Information Management System

TM

Selecting Potential NTIP Objectives

•

Stakeholders submitted wide range of objectives

•

Focus only on those with existing data•

Result: 28 potential objectives

•

Stakeholders ranked and grouped objectives into 15 categories

TM

National:

Established by the National Objective Workgroup

Based on the “current”

performance of a high

performing state

Method used uniformly across all objectives

Suggestions on methods accepted for future

Program:•

Set their own targets for objectives based on the capacity and resource availability

•

Targets to be agreed on with DTBE consultants

Establishing NTIP Targets

TM

National TB Program Objectives 2015

TM

NationalObjective

State Performance

Graph

State Data

Method

TM

Example: TB Treatment Completion

TM

7%

89%

0%

0%

1%

3%

Completed Within 12 Months

Completed After 12 Months

Moved

Lost to Follow-Up

Refused

Unknown/Other

Treatment Completion for Eligible Patients, by Treatment Outcomes,

State X, 2003

TM

Example: Know HIV Status

TM

Example: Know HIV Status (cont)

TM

Example: Contact Investigation

TM

Example: Drug Susceptibility Results

TM

Integrating NTIP into Program Practice

MonitorProgressToward

Objective

Met Target

Did not Meet Target

Develop Evaluation Plan to Understand Barriers/Challenges

Provide Evaluation Updates

Complete Evaluation

Refine/Develop Program Activities Based on Findings

(CoAg application)

Implement Improvements

Best Practices

TM

NTIP will

•

Reinforce the national priorities •

Measure progress and impact using existing data

•

Identify priorities for program improvement, reporting and technical assistance

•

Facilitate evidence based practices•

Enhance collaboration among partners at all levels

Summary

TM

Acknowledgements

Robert PrattValerie RobisonMaria RodriguezTom ShinnickTeresa SmithDeborah Sodt Barbara StoneWendy SutherlandZach TaylorWanda WaltonMaureen Wilce

Subroto BanerjiJose BecerraJames L Beebe Janice Boutotte Melissa Ehman Kim Field Katie Garman Phil Griffin Cathy GoldsboroughChristopher Hayden Marge HigginsCarol HoffErin Holt

Stephen HughesKashef IjazJohn JerebBillie Juni Awal KhanBeth Kingdon Linda LearyPhil LoBueBeverly MetchockHeather Morrow-

AlmeidaMassa NaritaTom Navin

TM

Access NTIP

TB Controllers e-mail their respective program consultants requesting access for staff member

https://66.155.122.64/ntip/nt/session/login.do

TM

Virginia NTIP in TB Control and Program Evaluation

TM

Treatment Completion in 12 Months

TM

Treatment Outcomes

TM

VA Program Evaluation Framework, 2008

Determine reasons that contacts •

Not identified

•

With LTBI not started on treatment

TM

Contacts Elicited

TM

Contacts with LTBI Starting Treatment

TM

Contacts by Disposition

TM

Program Evaluation DTBE Approach

TM

Field Services and Evaluation Branch (FSEB)

Kashef Ijaz MDChief

Joe ScavottoDeputy

Field Operations IGreg Andrews

Team Lead

Field Operations IIDan RuggieroTeam Lead

Program Evaluation Awal Khan, PhD

Team Lead

Medical AffairsSundari Mase, MD

Team Lead

TM

TB Program Consultant (PC) and Evaluation Team Rep (ER) Assignment for Technical Support by Consulting Regions

VIIPC: Paul TribbleER: Kai Young

(AK, CO ID, MT, NV, OR, UT, WA, WY)

VIIIPC: Andy Heetderks

ER: Judy Gibson*/Awal Khan(CA, Los Angeles, San Diego,

San Francisco,Awal (HI, AS, FM, GU, MH, MP, PW)

VIPC: Mark Miner

ER: Judy Gibson*(AZ, KS, NM, OK, TX, Houston)

IIPC: Vic Tomlinson

ER: Awal Khan(Baltimore, DC, DE, MD, NCPA, Philadelphia, VA, WV)

IIIPC: Gail Burns-Grant

ER: Awal Khan(AL, AR, FL, GA, LA, MS, SC, TN)

IVPC: Bruce HeathER: Linda Leary*

(KY, MI, Detroit, MN, ND, SD, WI, Puerto Rico, Virgin Islands)

IPC: Dawn Tuckey

ER: Kai Young(CT, ME, MA, NH, RI, VT, NY, NJ,NYC)

VPC: Regina GoreER: Linda Leary*

(IA, IL, IN, MO, NE, OH, Chicago)

*Backup supports: Kai Young for Linda Leary; Awal Khan for Judy Gibson

TM

Collaborative Approach -

Evaluation

State/LocalTB Program

National Goalsand Objectives

Evaluation Activity

(Cohort Review)

1-5 yr Plan

Program Consultant

Evaluation Team

NTIP

PENFocal point

ETN Focal point

TM

PET Activities and Services

Help state and local TB programs develop

•

Goals and objectives•

An evaluation plan and logic model

•

Evaluation questions and performance indicators

•

Data collection methods, instrument, and analysis plan

TM

PET Activities and Services (cont.)

Assist state and local TB programs

•

Monitor program goals and objectives•

Identify priority areas of program improvement

•

Identify implementation gaps / promote best practices

•

Provide feedback, input on PE capacity building activities

•

Identify training, educational needs for staff

TM

PET Activities and Services (cont.)

Support TB Program Evaluation Network(TB PEN) Steering Committee in program

evaluation activities.

TM

Program Evaluation Focal Point

•

Designated point of contact for evaluation activities in COAG site by March 2010

•

Serves as the subject matter expert

•

Participates in TB Program Evaluation Network (TB PEN)

•

Shares experiences and lessons learned

•

Facilitates program evaluation capacity building

TM

TB Evaluation Products

TM

Any Questions?

37