RI monitoring

Sca ing-up RI programme

monitoring

Dr Arindam Ray

M&E Focal Person, WHO NPSP

SEPIO Meeting: 20th May, 2011

Presentation outline

Strategy

Generating evidence

Feedback and action

Expansion of the system

Way forward

Strategy

Recommendations on M&E for RI

GoI has circulated RI monitoring formats & SOPs to all states in July, 2009

IEAG, 2010 recommended that RI monitoring should be conducted in high priority, low coverage areas

Programme monitoring is a key component in national MO Training Handbook

RI monitoring Strategy

Strategic monitoring of HRAs Recognized HRAs (polio SIA microplan) Areas missed in RI microplan Villages with vacant sub-centres Peri-urban underserved areas ANM with large catchment population Migrant locations

Session monitoring Availability of vaccines & logistics Safe injection practices Tracking & mobilisation efforts

H-t-H (Community) monitoring Immunization status in community Reason for Left out & Drop out

Coordinate between Govt counterparts and partner organizations

Timely feedback (Block, District and State) for programme decisions

Consulted RI and SIA Microplan

Plan for monitoring based on Priority

Session Site

Session held

Session Monitoring

Session not held

Area for House to House Monitoring

Ten Households with 12-35 m children

Information from RI card & family members

Feedback to ANM

No Session Monitoring

1. Listed HRA (SIA Microplan)

2. Areas missed in Microplan

3. Villages with vacant Subcentres

4. Peri-Urban Underserved areas

5. ANM with large catchment population

6. MOB/ WPV / VDPV area

7. Migrant/ mobile

RI Monitoring Process

30 min ~1 hour

1~2 hours

Visit Block for monitoring & feedback

xkWao dk uke%

i zFke ?kj o fpUg%

ok; k ¼gksrs gq,½

vafre ?kj o fpUg%

xkWao dk uke%

i zFke ?kj o fpUg%

ok; k ¼gksrs gq,½

vafre ?kj o fpUg%

jkt s'k egrks] Hkou jk; ]uUnkl ko] jkt sUæBkdqj]jkt dqekj fo'odekZ]y{e.k fo'odekZ] HkksykfeL=h] t hokuUn mi k/; k;

vl QhZjk; ds ?kj l sjk; unqKku fudsru] cka; seqM+dj QqVdqu jk; ]t eknkj jk; ] vksei zdk'kfo'odekZ nka; s l s ck; sa

pkSFkk fnu

fo'odekZ Vksyh

fcUn] gfjt u ,oavU;

i kapok fnu

Lo0 l R; k egrks] i jekegrks] l quhy fxfj] mÙkejk; ] eqds'k jk; ] f'kodqekj egrks] l qxzho egrks

ckys'oj jke] xks/kuegrks] l gohj egrks] mekjk; ] nsokyh egrks] l okZegrks

fcUn Vksyh ,oa vU;

;kno Vksyk

nwl jk fnu rhl jk fnui gyk fnu

; kno ,oa vU;l qjs'k jk; ds ?kj l s Lo0l afpr jk; ] vf{krkuUn jk; ]fl i kgh Bkdqj] eaMy MªkbZoj]nqxkZjk; nksuksarjQ] eksgujk; ] fHk[kkjh jk;

vl jQh jk; ds ?kj l sjk"Vªh; Kku fudsru cka; sy[ku jk; t ehankj jk;vksei zdk'kfo'odekZnk; sal scka; segkohj jk; ] jkek/kkjjk; ] jkenso jk;

cPpq jk; ds ?kj l sl q[kkM+h jk; ] uckc jk; ]Hkksyk jk; ] jfoUæ jk; ]ukxsUæ Bkdqj] dkaxzsl jk;

; kno ,oa vU;jkeckcqjk; ds?kj t qxyjk; ] jke dqekj jk; dsJ hHkksyk jk; ] j?kqoj Bkdqj]d"̀.kk jk;

i zFke ,oanwl jsfnu dkX?kjksa dk i quHkZze.k ,oai zfrj{k.k LFky l sdk; Zdjuk gSA

uskan

eqckjdiqj QkeZ inkol gom

t ol i ioyrk

gfjt u ,oa vU;ujs'kjk; ds?kj l spqUuqjk; ]l rsUæjk; ] yky l kgsc jk; ]Hkksyk l ko] mt su jk; ]f'koi wt u jk;

1st & 3rd

Wed-

2nd Sat -

Wed- 2nd

eqckjdiqj QkeZ

eqckjdiqj QkeZ

Uskantoli inkol gom;kno Vksy

k

ioyrk

3rd Sat -4th Wed-

4th Sat -1st Sat -

SIA Microplan

RI Microplan

? ?

?

Plan for Mubarakpur S/C (Session) and Bind tola(HtH)12

Generating evidence

Indicators derivable from RI session monitoring

Sessions held or not? If not, reasons why

ANM/vaccinator was present as per microplan (%)

Utilization of AVD Availability of vaccine and

logistics: Vaccines (BCG,DPT,OPV, MCV,TT,

Hep B, JE, PV) Diluents (for BCG, MCV, JE) Syringes (AD: >0.1ml & 0.5ml,

Disposable). Unusable vaccines

Mobilisation of beneficiaries

Injection safety

AEFI management & reporting

Waste disposal New born tracking Vaccine safety (frozen,

unusable VVM, expired vaccine etc.)

Key RI messages conveyed to parents of beneficiaries

Which message is missed

Availability of other logistics: Vit A, PCM, ORS Hub cutter, RI cards, R&B bags

Stock-outs Supervisory visits

Key Indicators Additional Indicators

Planned sessions not conducted: Jan10-Dec10: Bihar (with reasons)

Conducted,

86%

Not

conducted,

14%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Bihar (n=30,604)

Source: RI monitoring data (Jan10-Dec10)

Reason for session not held (n=4356)

64%5%

8%

16%

7%

ANM absent, vaccine/ logistics not deliveredANM absent, vaccine/ logistics available ANM present, vaccine/ logistics not availableOthersUnknown

% Availability of all vaccines at sessions sites (Year – 2010 & 1st Quarter 2011)

JBSA – Jachha Bachha Suraksha Abhiyan -UP

80% 80% 80% 80%

70%

80% 80%

70%

50%

80%

90%

80% 81% 80%

88%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

JBSA started form August’10

Visited1

Up.shp

BCG_0M0 - 4040.1 - 6060.1 - 8080.1 - 100

Visited1

Up.shp

BCG_0M0 - 4040.1 - 6060.1 - 8080.1 - 100

Data not available

<= 40%

40% to 60%

60% to 80%>= 80%Not monitored

UP districts Cumulative Jan’10 - Dec’10 : 80%Sessions held: 59,811

UP districts CumulativeJan’11 – Mar’11: State Average-

83%Sessions held: 17,573

KRI

SBD

STP

LLP

HDO

JAL

JNS

AHB

BJN

BAD

BRC

PIL

RBL

SHA

SUL

MZP

BBK

BRL

FTP

BNA

JNP

UNN

AZG

MZN

AGR

ALG

SHP

HMP

BLS

GND

PTG

KSN

MRD

BRP

GZP

BAL

GRPBST

KPN

CKT

MTR

MAI

KPD

MHB

FAI

ETA

SDN

MRT

LNO

DOR

JPN

FKB

CND

FER

ABN

RMP

MHG

ETW

KNA

KAN

AUR

GZA

HTR

KSM

SRW

MAU

SKN

VRN

BGT

GBN

BDH

AEFI of Mohanlal Ganj (Lucknow)

KRI

SBD

STP

LLP

HDO

JAL

JNS

AHB

BJN

BAD

BRC

PIL

SHA

MZP

BRL

FTP

BNA

JNP

UNN

AZG

MZN

AGR

ALG

SHP

BBK

HMP

BLS

GND

PTG

KSN

MRD

BRP

RBL

FAI

GZP

BAL

GRPBST

KPN

CKT

MTR

MAI

KPDSULCSN

MHB

ETA

SDN

MRT

LNO

DOR

JPN

FKB

CND

FER

ABN

RMP

MHG

ETW

KNA

KAN

AUR

GZA

HTR

KSM

SRW

MAU

SKN

VRN

BGT

GBN

BDH

KRI

SBD

STP

LLP

HDO

JAL

JNS

AHB

BJN

BAD

BRC

PIL

SHA

MZP

BRL

FTP

BNA

JNP

UNN

AZG

MZN

AGR

ALG

SHP

BBK

HMP

BLS

GND

PTG

KSN

MRD

BRP

RBL

FAI

GZP

BAL

GRPBST

KPN

CKT

MTR

MAI

KPDSULCSN

MHB

ETA

SDN

MRT

LNO

DOR

JPN

FKB

CND

FER

ABN

RMP

MHG

ETW

KNA

KAN

AUR

GZA

HTR

KSM

SRW

MAU

SKN

VRN

BGT

GBN

BDH

<Empty Picture><Empty Picture><Empty Picture><Empty Picture>

Mobilisation of Children Bihar, 2010

47%

53%

YES NO

41%

59%

YES NO

Due List PreparedCounterfoils Updated

N=21,706

N=26248

N=26,248

N=24511

Mobilisers Delivery of 4 key messages

71%

29%

YES NO

43%17%

3%10%

27%

0%

ICDS, ASHA & Others ICDS & ASHAOnly ICDS Only ASHA

Others None

% Any Mobilizer (ICDS/ASHA/Others) present at the session sites, Jan-Mar

2011, BiharState average

Jan-Mar2011: 91 %

Nil1 - 30 %31 - 59 %60 - 79 %>= 80 %Data Not Available

Col No.

KAIMURKATIHARKHAGARIAKISHANGANJLAKHISARAIMADHEPURAMADHUBANIMUNGERMUZAFFARPURNALANDANAWADAPATNAPURNIAROHTASSAHARSASAMASTIPURSARANSHEIKHPURA

Ga ya

Patna

Jamui

Rohtas

Purnia

Kaimur

Banka

Ara ria

Saran

Katihar

Siwan

Supaul

Madhubani

Nawada

Champaran We st

BhojpurBuxar

Nala nda

Muzaffarpur

Bhagalpu r

Aurang aabad

Si tamarhi

Vaishali

Darbhanga

Champaran East

Sam astipur

Gopalga nj

Saharsa

Beg usa ra i

Munger

Kha garia

Madhepura

Kis han ganj

ArwalLakhisaraiJe hanabad

She ikhpura

She ohar

88 89 90 9188

91 89 90 91 91 91 90 90 9188 89 90 90 92

0

10

20

30

40

50

60

70

80

90

100

Se

p'0

9

Oct

'09

No

v'0

9

De

c'0

9

Jan

'10

Fe

b'1

0

Ma

r'10

Ap

r'10

Ma

y'1

0

Jun

'10

Jul'1

0

Au

g'1

0

Se

p'1

0

Oct

'10

No

v'1

0

De

c'1

0

Jan

'11

Fe

b'1

1

Ma

r'11

Source: RI session monitoring data by WHO NPSP and Govt. of BiharN = 7840 RI session found held

Nil

1 – 30 %

31 – 60%

61 – 80 %

> 80 %Data Not Available

Source: RI session monitoring data N = 5,426 RI sessions found held

% Due list availability at the session site,

Jan-Dec 2010, JharkhandState average 2010:

65 %

GUMLA

GIRIDIH

RANCHI

PALAMU

DUMKA

LATEHAR

CHATRAGARHWA

SIMDEGA

SINGHBHUM WEST

HAZARIBAGH

KHUNTI

GODDA

BOKARO

PAKUR

DEOGHAR

SARAIKELLA

DHANBAD

JAMTARA

SINGHBHUM EAST

SAHIBGANJ

RAMGARH

KODERMA

LOHARDAGA

Safe Injection Practices Bihar, Jan – Dec 2010

4%

96%

Yes No

DPT Given at Mid-Thigh

ANM Touching the needle

95%

5%

Yes No

92%

8%

Yes No

65%

35%

Yes No

n=24,706

n=21,909

n=23,607

n=1078

Time of Reconstitution Mentioned on Vial

Hub of syringe is cut in hub cutter immediately

n = number of RI sessions monitored

Source: RI session monitoring data by WHO NPSP and Govt. of BiharN = 1,151 sites where Vaccination seen

% Syringes being cut by hubcutter immediately after use, Jan-Mar 2011, Bihar

State average Jan-Mar2011: 85 %

Nil1 - 30 %31 - 59 %60 - 79 %>= 80 %Data Not Available

Col No.

KAIMURKATIHARKHAGARIAKISHANGANJLAKHISARAIMADHEPURAMADHUBANIMUNGERMUZAFFARPURNALANDANAWADAPATNAPURNIAROHTASSAHARSASAMASTIPURSARANSHEIKHPURA

Ga ya

Patna

Jamui

Rohtas

Purnia

Kaimur

Banka

Ara ria

Saran

Katihar

Siwan

Supaul

Madhubani

Nawada

Champaran We st

BhojpurBuxar

Nala nda

Muzaffarpur

Bhagalpu r

Aurang aabad

Si tamarhi

Vaishali

Darbhanga

Champaran East

Sam astipur

Gopalga nj

Saharsa

Beg usa ra i

Munger

Kha garia

Madhepura

Kis han ganj

ArwalLakhisaraiJe hanabad

She ikhpura

She ohar

54

42

57 5955 53

6359

63 62

71 71

79

55

69 71

80

86 87

0

10

20

30

40

50

60

70

80

90

100

Se

p'0

9

Oct

'09

No

v'0

9

De

c'0

9

Jan

'10

Fe

b'1

0

Ma

r'10

Ap

r'10

Ma

y'1

0

Jun

'10

Jul'1

0

Au

g'1

0

Se

p'1

0

Oct

'10

No

v'1

0

De

c'1

0

Jan

'11

Fe

b'1

1

Ma

r'11

Monitored sessions with safe injection practices, Uttar Pradesh

100%

84%

92% 91%

30%

100%

87%93% 95%

30%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

AD Syringe used forinjectable vaccines

(n=56,896 / 16,961)

DPT givenanterolaterally

(n=52,994 / 16,111)

ANM not touching anypart of needle while

giving injection(n=55,149 / 16,555)

Reconstitution timewritten on vials

(n=49,133 / 14,855)

Syringe cutimmediately after use(n=41,686 / 11,509)

2010 2011 (1st Quarter)

n=monitored sessions where corresponding activities were observed ( 2010 / 2011- 1st Quarter)

Indicators derivable from household monitoring

No of Children in 12-35 months fully immunized (%)

Children receiving age specific antigens including

Hep B, MCV2, DPT Booster

Reasons for not getting the due vaccine

Drop out and left out rate Reasons for drop out & Left

out Areas where RI sessions are

not held in last 3 months

Mobilisation of beneficiaries Presence of ASHA/ AWW Participation by PRI/ SHG/

NGO Vaccination status in

community: By migration By religion By caste By gender By occupation By education

No of beneficiaries provided with RI/ MCP card (%)

Retention of RI / MCP card (%)

Key Indicators Additional Indicators

39% 39% 41% 45% 46% 47% 46% 50% 50% 50% 49% 50%

42% 43% 42%40% 40% 39% 40%

37% 37% 38% 39% 37%

18% 17% 17% 15% 14% 14% 14% 13% 13% 13% 12% 13%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jan

(n=4087)

Feb

(n=7142)

Mar

(n=15129)

Apr

(n=16189)

May

(n=11048)

Jun

(n=14176)

Jul

(n=22352)

Aug

(n=20403)

Sep

(n=12239)

Oct

(n=18920)

Nov

(n=10404)

Dec

(n=24545)

Fully immunized Partially immunized Unimmunized

n=Number of children 12 to 23 month of age (176,634)

Immunization status of monitored children 12-23 months, UP, Jan-Dec

2010

Fully immunized: 47.1%

Partially immunized: 39.1%

Left-outs: 13.9%

Full immunization status and BCG-measles drop out rates, Bihar, Jan-Dec

2010

BCG-Measles Drop-outBelow 1010 % to 30 %30 % to 50 %50% to 60 %

Up.shpFIC (Fully immunized coverage)

BCG-measles drop out rate

n = 72,162 children 12 to 23 month of age

Col No.

KAIMURKATIHARKHAGARIAKISHANGANJLAKHISARAIMADHEPURAMADHUBANIMUNGERMUZAFFARPURNALANDANAWADAPATNAPURNIAROHTASSAHARSASAMASTIPURSARANSHEIKHPURA

Gaya

Patna

Jamui

Rohtas

Purnia

KaimurBanka

Araria

Saran

Katihar

Siwan

Supaul

Madhubani

Nawada

Champaran West

BhojpurBuxar

Nalanda

Muzaffarpur

Bhagalpur

Aurangaabad

Sitamarhi

Vaishali

Darbhanga

Champaran East

Samastipur

Gopalganj

Saharsa

Begusarai

Munger

Khagaria

Madhepura

Kishanganj

ArwalLakhisaraiJehanabad

Sheikhpura

Sheohar

Col No.

KAIMURKATIHARKHAGARIAKISHANGANJLAKHISARAIMADHEPURAMADHUBANIMUNGERMUZAFFARPURNALANDANAWADAPATNAPURNIAROHTASSAHARSASAMASTIPURSARANSHEIKHPURA

Gaya

Patna

Jamui

Rohtas

Purnia

KaimurBanka

Araria

Saran

Katihar

Siwan

Supaul

Madhubani

Nawada

Champaran West

BhojpurBuxar

Nalanda

Muzaffarpur

Bhagalpur

Aurangaabad

Sitamarhi

Vaishali

Darbhanga

Champaran East

Samastipur

Gopalganj

Saharsa

Begusarai

Munger

Khagaria

Madhepura

Kishanganj

ArwalLakhisaraiJehanabad

Sheikhpura

Sheohar

20 - 39%40 - 49%50 - 59%60 - 90%

33%8%

7%

16%

5%6%

25%

Immunization status of monitored children, Jan-Dec, 2010, Bihar

61%

33%

6%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

BIHAR (n=72,162)

Reason for children not being fully immunized

Fully Immunized

PartiallyImmunized

No Immunization

n=children 12 to 23 months of age

19%

4%

30%

9%

5%

21%12%

N=23,783

N=4,234

Indicators derivable from Block/ District monitoring

Linkage with session monitoring

ANM/vaccinator vacancy status

Utilization of AVD Availability of vaccine

and logistics (stock outs)

Cold Chain Report-returns

Training status Supportive

supervision Review mechanism AEFI reporting VPD surveillance Functioning of

committees: Task Force ERT/ RRT AEFI committee

Key Indicators Additional Indicators

Feedback and action

November 2010:8 of 19 districts (39%)

Districts with HR blocks where RI monitoring data reviewed during the DTF meeting at least once during 2 months

January 2011:19 of 19 districts (100%)

Districts with HR blocks where RI monitoring data not reviewed during the DTF meeting at least once during 2 months

Districts with no HR blocks

Data not available

RI monitoring feedback in high risk areas: Use of District Task Force (DTF)

Information feedback process

Core indicators: Immediate verbal and/or written feedback to Block MO

and District Immunization Officer Written feedback of core indicators forwarded to State

Immunization Officer

Detailed analysis: Data is compiled, entered and cleaned at state level Exhaustive analysis (by district and blocks) shared with

districts by states

RI Monitoring feedback is regularly discussed and shared in RI Cell meetings

in Jharkhand, Bihar, UP, Orissa….

Expansion of the system

Scaling up RI monitoring

Following a state review meeting, Karnataka has started monitoring with special emphasis on northern districts

Rajasthan has also initiated RI monitoring since end of 2010

Govt officials in Punjab are spearheading monitoring of RI sessions for last six months

West Bengal has also initiated RI monitoring with more emphasis on HR districts like Murshidabad, Howrah etc

Medical colleges in Orissa are monitoring RI sessions in selected districts

Govt of Delhi has also launched RI monitoring in last 1 year

RI monitoring is planned to be introduced in the coming months to: Maharashtra, Madhya Pradesh, Kerala, Tamilnadu, Chhattisgarh, Assam…..

Every state to operationalise RI

monitoring with GoI formats

Formats to be used by all States:Session and household monitoring

Formats to be used by all States:Block and District level

Easy to use

Structured MCQ

KEYs given

One pager

Way forward…

All states are expected to initiate monitoring Adapt strategy and tools to local situations Ensure feedback and follow-up interventions Technical assistance is available from

partners New technologies: OMR/ PDA Oversight is critical to ensure quality of data Monitoring to be dynamic to capture

changing practices and priorities

Thank You

Strengthening Monitoring

Problem in the content: Multiple variables clubbed Some essential components missing No data on determinants No monitoring for mid-level operations at the Block and District

levels

Change in the Context: Evolving practices and priorities since 2009 MCV2 and Pentavalent are being introduced Hep B to be rolled out to all remaining states Lessons from session & RCA monitoring in MCUP Need for universalisation of the monitoring tools

Data tools also need further updating and revision