-
1
Decentralized Basic Education 1: Management and Governance
ICT Grants:
Monitoring & Evaluation
July 2008
This report is one of a series of special reports produced by
Research Triangle Institute (RTI),
Implementing Partner for the USAID-funded Improved Quality of
Decentralized Basic Education
(IQDBE) program in Indonesia
Special Monitoring Report
-
2
DBE1
ICT GRANTS
Monitoring & Evaluation
2008 USAID DBE1 – Management and Education Governance Indonesia
Stock Exchange Building Tower 1, 29
th Floor
Jl. Jend. Sudirman Kav. 52-53, Phone (62 21) 515 2772 • Fax (62
21) 515 5859
Jakarta 12190, Indonesia http://pdms.dbeindonesia.org
http://www.dbe-usaid.org
http://pdms.dbeindonesia.org/http://www.dbe-usaid.org/
-
3
DBE1 – ICT Grants Program Monitoring Activity
Outcomes and Recommendations
April/May 2008
Purpose of the Activity The purpose of the DBE1 ICT Grants
Program Monitoring Activity is to provide an insight
into the DBE1 ICT Grants Program to date, document lessons
learned, provide
recommendations and propose next steps for its implementation,
especially in regard to the
feasibility to conduct round II of the program with DBE1 cohort
2 districts, and in regard to
necessary activities to capture and document lessons learned
from existing grant programs.
I. BACKGROUND
1. Task Order Requirements:
Grants to support RFP objectives and wider community
Grants to include private sector
Grants to be competitive
Grants to show sustainable models for profit-driven education
hotspots (business plans)
Grants to use business and/or tech volunteers
Grants programs of DBE123 to be different from each other to
show approaches
2. DBE1 ICT Grants Program – Key Aspects:
One program for both education hotspot (ICT Access Grants) and
grants targeting
education management and governance (EMG) (ICT Innovation EMG
Grants)
Program objectives:
o Leverage ICT to contribute to more effective management and
governance of
education
o Leverage ICT to contribute to more effective information and
communication
among education stakeholders and the wider community
Two tier system:
Tier 1 grants range between $2,000 and $4,000 in funding.
Tier 2 grants range between $10,000 and $40,000 in funding.
Consortium requirements: At least one private sector actor and
one DEO agency
Cost-sharing requirement
Grant program as an opportunity to broker relationships and
build capacity
o Sensibilization and Awareness raising activities,
o Regional forums to broker consortium,
o Assistance (manuals, in-person) in grant application
development,
o Program (objective alignment, coherence, M&E) and budget
review
assistance (budget integrity, unit estimations, business
planning) for short
listed applicants,
o Review and Assistance of applicant financial management
practice for grant
management,
o Grantee resource Toolkit.
-
4
3. Related DBE1 Indicators:
IR 3: Increased use of Information Resources to Enhance
Education Management and
Governance:
o IR 3-24: Increased capacity of education stakeholders in
target districts to use
ICT in education management and governance.
o IR 3-25: Number of grants awarded to district governments in
collaboration
with the private sector to develop and implement activities that
increase use
of ICT and are suitable of wider application.
o IR 3-26: Number of grants awarded to district public
institutions in
collaboration with the private sector to develop and implement
”education
hotspots” that have sustainable business plans and are suitable
of wider
application.
For the purpose of this grants program and internal tracking, IR
3-24 has been further
differentiated into three key aspects:
1. Increased capacity for use of ICT among education
stakeholders
2. Increased access to functioning ICT infrastructure for
education stakeholders
3. Increased access to education information and services for
education stakeholders
4. Grantees to Date (Table 1):
14 Grants.
South Sulawesi (4), West Java (2), Banten (1), Central Java (3),
East Java (3), North
Sumatera (1).
USD 246,284 committed.
7 Tier 1; 7 Tier 2 grantees.
7 ICT Innovation EMG Grants and 7 ICT Access Grants
(hotspot).
Grantee main activities (Figure 1).
Figure 1. Distribution of Grantee Main Activities
-
5
Table 1. Grantees and Grantee Main Activities to Date
NO GRANT CATEGORY
GRANTEE NAME
(PRIVATE SECTOR
PARTNER)
LOCATION
GRANTEE MAIN ACTIVITIES
Inte
rnet
Café
(W
arn
et)
Lib
rary
MIS
EM
IS/M
IS
Educa
tion P
ort
al
eR
eso
urc
es
IT C
ente
r/
Connect
ivity/
Netw
ork
IT M
ain
tenance
/ Serv
icin
g
Sch
ool Connect
ivity
Tra
inin
g (
DEO
)
Tra
inin
g (
Oth
ers
)
1. Tier-1 EMG
Turatea Computer Center
Janeponto - South Sulawesi
2. Tier-2 EMG
YPK Amanah Pangkep - South Sulawesi
3. Tier-1 Hotspot
Indo Komputer Soppeng - South Sulawesi
4. Tier-1 Hotspot
PT Rekayasa Teknologi Informasi
Enrekang - South Sulawesi
5. Tier-1 Hotspot
PT Tridata Cakrawala
Tuban - East Java
6. Tier-1 Hotspot
Yayasan Tarbiyah Islamiyah
Sukabumi - West Java
7. Tier-2 EMG
CV Trisatya Pratama
Karawang - West Java
8. Tier-2 Hotspot
CV Almagada Jaya
Tangerang - Banten
9. Tier-2 EMG
PT Indomaya Wira Sejahtera
Karanganyar - Central Java
10. Tier-1 Hotspot
PT Indomaya Wira Sejahtera
Karanganyar - Central Java
11. Tier-2 EMG
CV Cosmo Jaya Klaten - Central Java
12. Tier-2 EMG
PT ITS Kemitraan Surabaya - East Java
13. Tier-1 EMG
PT ITS Kemitraan Surabaya - East Java
14. Tier-2 Hotspot
PT Indoukm Insis Tapanuli Utara – North Sumatra
Grantees Main Activities 6 5 4 5 2 8 1 3 13 7
5. Problem Statements according to Grant Applications:
Grant applications included the requirement of a problem
statement that would drive the
purpose/objectives of the proposed grants program. Problem
statements from the 14 grant
proposals are summarized below:
There are inefficiencies in education management and governance
(information-based
decision making, financial modeling, planning)
o There is limited capacity of DEO staff to make use of ICT for
education
management and governance ( available computers are
underutilized)
-
6
o There are communication gaps between DEO and education
stakeholders
(critical information arrives only in part and late)
o There are communication gaps between different DEO
agencies
o There are information discrepancies between different DEO
agencies
(fragmentation of management information systems, no access to
easy and
secure communication and information exchange channels)
o There is underutilization of existing infrastructure
(hardware, software or
connectivity limitations/failures)
There is a lack of self-study and/or complementary teaching and
learning resources
in “digestible” formats
There is limited capacity among teachers and relevant DEO
counterparts in
developing electronic teaching and learning resources to
complement existing
materials
There are inefficiencies in DPL management (user registration,
collection processing,
cataloguing, retrieval, user transactions and monitoring are
done manually, using
paper-based systems)
Education stakeholders lack structured avenues/opportunities to
access (education)
information and related services
Community members lack structured avenues/opportunities to
access reliable
(education) information/data and services (problem of social
inclusion)
II. GRANT IMPLEMENTATION MONITORING FINDINGS
The following are key findings of the April 2008 DBE1 ICT Grants
Program monitoring
activity, which included interviews with DBE1 staff managing and
administrating the grants,
review of grantee documentation, as well as site visits and
interviews to several grantees:
USAID and RTI grant regulations require heavy pre-award
determination efforts.
This resulted in considerable delays in grant
implementation.
USAID and RTI procurement regulations require significant
efforts in procurement.
This resulted in considerable delays in grant implementation for
most grantees.
During the rather lengthy pre-award determination and
procurement process several
of the private sector consortium leaders changed their status or
company name which
required updates to grant application materials and led to even
further delays.
Due to the rather lengthy pre-award determination and
procurement process, several
grantees had to withdraw their grant applications because of
changes in personnel or
regulations that prohibited them to continue the process and
participate in the
program.
A final 14 of 22 selected grantees signed a grant award
agreement with
DBE1/USAID and are currently in the implementation process.
Grantee consortia interviewed during the field visits show, in
all cases, very good
collaboration among partners, that is district public
libraries.
-
7
Current grantees show a wide variety of activities and different
approaches, both in
establishing education hotspots (ICT Access Grants), as well as
in using technology
to improve management and governance of education (ICT
Innovation EMG Grants).
Capacity building is an integral part of most grantees’ programs
and activities.
Status of grant implementation is varied. Some grantees have
already completed most
of their activities and are now in the ongoing monitoring phase
of their grants
program, others are still in the procurement stage for equipment
(e.g. to set up an
education hotspot).
Grantee capacity for sound M&E and reporting is very
limited.
III. RECOMMENDATIONS & NEXT STEPS
1. Recommendations:
The key recommendation in regard to the DBE1 ICT Grants Program
is to capitalize and
focus on the existing 14 grantees, rather than expanding the
grants program to more districts
and actors at this stage of the program.
Lessons learned from the program to date highlight challenges
DBE1 and grantees were
facing in regard to processes and procedures of grant award
pre-determination and equipment
procurement, which affected effective and timely processing and
implementation. At the same
time, the 14 existing grants already show a wide variety of
activities and in each a number of
different approaches that provide a unique opportunity to
identify lessons learned, study best
practices and serve as examples for future programs and
activities in this area, which was a
key objective for the overall grants program.
2. Key Questions to Move on:
The recommendation above, therefore, highlights the need for
sound and detailed monitoring
and documentation of the existing 14 grants program. In this
context, the following two key
questions need to be addressed in planning next steps:
(1) What do we need to do (now and in the coming months/years)
in terms
supporting and monitoring grantee activities?
(2) What systems do we need to put in place and what information
do we need to
collect (and when) to write a substantial evaluation report of
the grants program
in two years time.
3. Proposed Next Steps:
To address these questions, on the basis of the information
gathered and the recommendation
made, the following next steps are being proposed:
To confirm lessons learned and findings from the April 2008 site
visits:
1. Upon receipt of first progress report from each grantee,
review for completeness and
compliance with general reporting requirements.
2. Upon receipt of first progress report from each grantee,
carefully review all grantees
elaborations on Monitoring and Evaluation (M&E) of their
grants program:
-
8
a. Were indicators “customized” on the basis of DBE1 guidance
(per DBE1 ICT
Grantee Toolkit and related socialization/grantee briefing
activities).
b. Does the grantee indicate understanding on what indicators
are and how they
are supposed to provide progress information?
c. Does the grantee demonstrate capacity in implementing
appropriate M&E
activities, such as surveys or training evaluations, e.g.?
To address challenges in regard to M&E and reporting:
3. Provide personalized support to each grantee to clarify gaps
or aspects that were
found to be unclear, e.g., on
d. Required content for their progress reports (drawing on
examples in the grant
implementation toolkit, which all grantee have received upon
award).
e. The difference between the requirements to fulfill grant
progress payment
milestones and the requirements in regard to monitoring,
evaluating and
documenting their progress in achieving grant program
objectives.
f. Clarification on the key issues to be addressed by their
program (re-iteration
of connection between problem statement and program
objectives).
g. Customization of M&E indicators to their individual
programs (on the basis
of the examples provided in the grantee toolkit).
To manage and document grantees’ progress and the overall
program,
4. Carefully track each grantee’s progress on activity status
and reporting to DBE1, as
well as each grantee’s adherence to an appropriate M&E
framework, that addresses
information needs for a later overall grants program
evaluation.
In this regard, the following tools have been provided,
developed together with DBE1 EMIS
& ICT Specialist and M&E Specialist, to support
implementation of next steps and its key
task to carefully monitor grantee activities over the next 2
years:
1. A tool to facilitate tracking of grantee milestones and
progress payment disbursement,
including an overview on all grantees’ activities, and tools for
documentation of grant
program implementation status on monthly or quarterly basis.
This is Appendix 1is
attached to this report.
2. An updated ICT Grants Program M&E Framework (on the basis
of what had been
developed in July 2007). This is Appendix 2 is attached to this
report.
3. A tool to track overall ICT grants program indicators, as
well as customized grantee
indicator tracking sheets for each individual grantee. This is
Appendix 3 is attached to
this report.
It is anticipated that this support to grantees in areas of
M&E and careful tracking and
documentation on grant program activities and grantees progress
towards program objectives
will provide DBE1 with the information needed to write a
substantial evaluation report of the
grants program in two years time, highlighting different
approaches and models on the use of
ICT to strengthen management and governance of education and to
benefit the wider
community.
-
9
APPENDIX 1
Tracking of Grantee Milestones
and Progress Payment
Disbursement
-
10
INVENTORY OF GRANTS Instructions:
1. Please complete yellow highlighted cells.
Country: 2. Please add specific comments in a footnote
Project Name:
Project Start & End Date
RTI Project Number:
Contract/Agreement Number:
Grant Manager/Email:
PAS:
Number of GrantsTotal Committed
AmountTotal Obligated Amount
$0,00 $0,00
Simplified Cost Reimbursement
Grants (SIGs) Issued
Grantee Name Start
& End Date
Total Grant Amount Obligated Amount as
of 06/30/08
Grantee Name
Start & End Date
Total Grant Amount Obligated Amount as
of 06/30/08
Grantee Name
Start & End Date
Total Grant Amount Obligated Amount as
of 06/30/08
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$0,00 $0,00 $0,00 $0,00 $0,00 $0,00
Through June 30, 2008
Fixed Obligated Grants (FOGs) Issued Standard Cost
Reimbursement
Grants (STGs) Issued
Total Grants Budget
(Dollar Value)
-
11
ICT GRANT RECORDS As of May 31, 2008
NO GRANT USAID GRANTEE NAME LOCATION Grant Period
No APPROVAL IN RP IN USD IN RP IN USD IN RP IN USD
1 0209604-G-07-001 April 2007 Turatea Computer Center Janeponto
- South Sulawesi (Apr 17,2007 - Mar 17, 2008) 25.252.500 2.806
18.300.000 2.033 6.952.500 773
2 0209604-G-07-002 April 2007 YPK Amanah Pangkep - South
Sulawesi (Apr 30,2007 - Apr 30,2009) 415.152.500 46.128 99.493.375
11.055 315.659.125 35.073
3 0209604-G-07-003 May, 2007 Indo Komputer Soppeng - South
Sulawesi (Aug 1,2007 - Aug1, 2008) 31.243.000 3.471 21.730.000
2.414 9.513.000 1.057
4 0209604-G-07-004 May, 2007 PT Rekayasa Teknologi Informasi
Enrekang - South Sulawesi (Apr 1, 2007 - Dec 31, 2007) 25.925.000
2.881 7.777.500 864 18.147.500 2.016
5 0209604-G-07-005 June, 2007 PT Tridata Cakrawala Tuban - East
Java (June 18, 2007 - Feb 14, 2008) 69.754.000 7.750 61.304.000
6.812 8.450.000 939
6 0209604-G-07-006 June, 2007 Yayasan Tarbiyah Islamiyah
Sukabumi - West Java (June 27,2007 - May 28,2008) 60.670.400 6.741
37.430.000 4.159 23.240.400 2.582
7 0209604-G-07-007 06-Sep-07 CV Trisatya Pratama Karawang - West
Java (Sept 6, 2007 - Sept 4, 2009) 292.518.451 32.502 21.576.553
2.397 270.941.898 30.105
8 0209604-G-08-008 13-Nov-07 CV Almagada Jaya Tangerang - Banten
(August 22,2007 - July 22, 2009) 300.930.000 33.437 119.927.250
13.325 181.002.750 20.111
9 0209604-G-08-009 13-Nov-07 PT Indomaya Wira Sejahtera - T2
Karanganyar - Central Java (Oct 10, 2007 - Sept 10, 2009)
258.532.500 28.726 17.270.500 1.919 241.262.000 26.807
10 0209604-G-08-010 13-Nov-07 PT Indomaya Wira Sejahtera - T1
Karanganyar - Central Java (Sept 10, 2007 - Aug 19, 2008)
27.377.000 3.042 4.809.300 534 22.567.700 2.508
11 0209604-G-08-011 7-Jan-08 CV Cosmo Jaya Klaten - Central Java
(Jan 7,2008 - Jan 7, 2010) 347.461.000 38.607 26.125.000 2.903
321.336.000 35.704
12 0209604-G-08-012 4-Jan-08 PT ITS Kemitraan - T2 Surabaya -
East Java (Jan 17, 2008 - Jan 15, 2010) 311.557.100 34.617 - -
311.557.100 34.617
13 0209604-G-08-013 22-Feb-08 PT ITS Kemitraan - T1 Surabaya -
East Java (Feb 4, 2008 - Feb 4, 2009) 50.184.000 5.576 - -
50.184.000 5.576
TOTAL 2.216.557.451 246.284 435.743.478 48.416 1.780.813.973
197.868
GRANT AMOUNT REALISATION GRANT Obligated Amount
-
12
PROVINCE: SOUTH SULAWESI
1. TURATEA COMPUTER CENTER Bank Voucher
NO SCHEDULE AMOUNT REALIZATION REALISATION Balance NOTES
OF PAYMENTS PAYMENTS AMOUNT
1 17-Apr-07 3.800.000 30-Apr-07 3.800.000 - MAK/0000464/BV
17-Apr-07 6.550.000 30-Apr-07 6.200.000 350.000 1 unit Laptop
MAK/0000465/BV
2 17-Jul-07 4.500.000 18-Jul-07 4.500.000 - MAK/0000598/BV
3 17-Sep-07 3.800.000 4-Oct-07 3.800.000 - MAK/0000734/BV
4 27-Nov-07 3.800.000 3.800.000
5 17-Mar-08 2.802.500 2.802.500
TOTAL 25.252.500 18.300.000 6.952.500
2. INDO KOMPUTER
NO SCHEDULE AMOUNT REALIZATION REALISATION Balance NOTES
OF PAYMENTS PAYMENTS AMOUNT
1 14-Aug-07 5.280.000 20-Aug-07 5.280.000 - MAK/ 0000644/BV
14-Aug-07 6.520.000 29-Oct-07 7.600.000 (1.080.000) Equipments
MAK/ 0000750/BV
2 14-Sep-07 8.850.000 15-Jan-08 8.850.000 - MAK/ 0000852/BV
3 15-Jan-08 6.300.000 6.300.000
4 15-May-07 2.725.000 2.725.000
5 15-Jul-07 1.568.000 1.568.000
TOTAL 31.243.000 21.730.000 9.513.000
3. PT REKAYASA TEKNOLOGI INFORMASI
NO SCHEDULE AMOUNT REALIZATION REALISATION Balance NOTES
OF PAYMENTS PAYMENTS AMOUNT
1 16-May-07 7.777.500 7.777.500 - MAK/ 0001110/BV
2 19-Jul-07 10.370.000 10.370.000
3 18-Dec-07 7.777.500 7.777.500
TOTAL 25.925.000 - 7.777.500 18.147.500
4. YPK AMANAH (SIG with advance)
NO ADVANCES AMOUNT REALIZATION REALISATION OUTSTANDING NOTES
MONTH PAYMENTS AMOUNT ADVANCES
1 July 2007 15.625.000 17-Sep-07 13.525.375 - MAK/000708/BV
13-Feb-08 1.270.000 MAK/000910/BV
2 14-Dec-07 7.700.000 laptop & printer MAK/000798/BV
3 Dec 2007 4.180.000 20-Feb-08 3.634.000 546.000
MAK/000926/BV
4 1-Feb-08 62.500.000 10 unit computer MAK/000893/BV
5 15-May-08 6.264.000 personel fee oct 07 - Mar 08
MAK/001202/BV
6 19-May-08 4.600.000 1 unit generator Yamaha MAK/001208/BV
7
8
TOTAL 19.805.000 99.493.375 546.000
-
13
APPENDIX 2
Updated ICT Grants Program
M&E Framework
-
14
DBE1 ICT Grants Program – M&E Framework
I. DBE1 – Existing Results Framework
II. DBE1 – Proposed Results Framework
III. SIR 3-24.1 : Increased capacity for ICT use among Education
Stakeholders – Indicators
IV. SIR 3-24.2: Increased Access to Education Information –
Indicators
V. SIR 3-24.3: Increased Access to ICT - Indicators Framework
explanation: This framework tries to outline a systematic approach
of
integrating indicators and results with a focus on the grantee
level.
This framework provides a set of suggested indicators for each
main grantee activity, such as
training, establishment of a connectivity or network between
diknas and depag, development
of a web-based education information system, or establishment of
an education hotspot/public
Internet access point.
Each indicator in this framework is presented with an indicator
definition, method and
frequency of data collection, method of calculation, data proof,
the key person responsible for
tracking it, and with information on who will need to report it
to DBE1 how often and in what
format.
The indicator tables below are to be customized for each grantee
and used, as appropriate,
for the explanatory note accompanying the grantee indicator
tracking sheets developed in
parallel to this framework.
-
15
I. DBE 1 – EXISTING RESULTS FRAMEWORK
USAID Program Objective: More effective decentralized education
management
and governance
USAID IR3: Number of districts using enhanced ICT for data and
information reporting,
assessing best practices, and enhanced communication in the
field of education
DBE1 IR 3: Increased use of information resources to enhance
education management and
governance
DBE1 Intermediate Result 3-24: Increased capacity1 of education
stakeholders in target
districts to use1 ICT in education management and governance
DBE1 Intermediate Result 3-25: Number of grants awarded to
district governments in
collaboration with the private sector to develop and implement
activities that increase use of
ICT and are suitable of wider application
DBE1 Intermediate Result 3-26: Number of grants awarded to
district public institutions in
collaboration with the private sector to develop and implement
”education hotspots” that have
sustainable business plans and are suitable of wider
application.
II. DBE1 – PROPOSED RESULTS FRAMEWORK Below are DBE1
Sub-Intermediate Results (SIR) that are being proposed to
differentiate
critical aspects of Indicator 3-24 for the purpose of the ICT
Grants Program:
SIR 3-24.1: Increased capacity for ICT use among education
stakeholders
SIR 3-24.2: Increased access to education information
SIR 3-24.3: Increased access to ICT
The following are proposed indicators to track progress in
achieving each of the three
proposed SIRs.
1 We understand the capacity to “use” ICT as a function of
having access to ICT infrastructure, the
skills to make use of the infrastructure and access to
appropriate content (education information).
-
16
III. SIR 3-24.1: INCREASED CAPACITY FOR ICT UE AMONG EDUCATION
STAKEHOLDERS –
INDICATORS
A. ICT-RELATED TRAINING
The indicators below are for those grantees that organize
ICT-related training in some form for a variety/specific group of
education stakeholders.
# Indicator Indicator Definition Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1
By whom
How often When Format
3-24.1.a
# of education stakeholders attending an ICT-related training
course
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
a. Capture number of trainees attending at least one training
session, disaggregated by user groups
Count number of people that attended the training.
a. Training participants log
e.g. Training Manager
based on training schedule/ work plan
after every training session
e.g. Project Manager
Quarterly (or by progress reporting period) if a training had
been started in that quarter
Based on start of training program
Progress Report including summary on the training program
carried out during the last reporting period and a summary on the
indicator calculation, including copies of the training manuals and
participant lists.
“ICT-related” training is any training that teaches basic
computer or Internet skills, computer networking, servicing, and
maintenance skills, electronic resource production and evaluation
skills or general data entry and management skills.
Disaggregate by gender and user group2
A “training course” is considered a course that constitutes at
least 4 contact hours.
2 For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators
(diknas [incl. DPL staff], depag, DEB) are one user group,
school administrators (principal, vice-principal, other school
admin staff) are another, teachers, students and
parents each are separate user groups as well.
-
17
# Indicator Indicator Definition Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1
By whom
How often When Format
3-24.1.b
% of education stakeholders successfully completing an
ICT-related training course
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
a. Record training participation at every training session.
Divide number of education stakeholders that successfully
completed the course, by number of education stakeholders that
signed up for the course. Move decimal two places to the right and
add a percentage sign (e.g. 15 of 75 training participants signed
up for the course AND passed the exam: 15/75=0.2 or 20%).
a. Participant training log
e.g. Training Manager
a. based on training schedule
a. after every training session
e.g. Project Manager
Quarterly (or by progress reporting period) if a training had
been completed in that quarter
Based on start of training program
Progress Report including summary on training program carried
through and outcome of the indicator calculation, in the last
reporting period, with copies of training logs, and
testparticipation logs
“Successful” completion entails regular attendance and passing
of an end-of-course test (either internationally, nationally, or
institutionally standardized)
b. Record names of training participants that sit for the test
and pass the test
Disaggregate by gender and user group3
b. Test participation log
b. once b. after every course
“ICT-related” training is any training that teaches basic
computer or Internet skills, computer networking, servicing, and
maintenance skills, electronic resource production and evaluation
skills or general data entry and management skills.
A “training course” is considered a course that constitutes at
least 4 contact hours.
3 For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators
(diknas [incl. DPL staff], depag, DEB) are one user group,
school administrators (principal, vice-principal, other school
admin staff) are another, teachers, students and
parents each are separate user groups as well.
-
18
# Indicator Indicator Definition Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1
By whom
How often When Format
3-24.1.c
% of education stakeholders continuing on to another ICT-related
training course.
This indicator captures the percentage of education stakeholders
that successfully completed a paid or free ICT-related training
course of at least 4 contact hours, and are taking an additional
paid or free ICT-related training course of at least 4 contact
hours.
Compare log of trainees that start a new training course with
the test log from previous courses.
Divide number of education stakeholders continuing with another
training course, by number of education stakeholders successfully
completing training. Move decimal two places to the right and add a
percentage sign (e.g. 85 of 280 training participants continue on
to another training: 85/280=0.31 or 31%)4
a. Training logs
e.g. Training Manager
once At the beginning of each training course.
e.g. Project Manager
Quarterly (or by progress reporting period)
Based on start of training program
Progress Report including summary on training program carried
through during last reporting period, summary on indicator
calculation, accompanied by copies of training logs, and test
participation logs
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
Disaggregate by gender and user group2
b. Test logs
“ICT-related” training is any training that teaches basic
computer or Internet skills, computer networking, servicing, and
maintenance skills, electronic resource production and evaluation
skills or general data entry and management skills.
A “training course” is considered a course that constitutes at
least 4 contact hours.
4 Rounded for 2 decimals
-
19
# Indicator Indicator Definition Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1
By whom
How often When Format
3-24.1.d
% of stakeholders reporting to have medium or advanced level
computer and(/or) Internet skills
This indicator tracks education stakeholders that respond to a
survey, indicating that they have at least medium level, if not
advanced level computer and/or Internet skills. The survey will
provide adequate explanation on the skills each level entails to
facilitate the self-assessment5
Carry out a separate baseline and a training impact evaluation,
e.g., by providing questionnaires to a representative sample of
education stakeholders.
Divide the number of education stakeholders that were surveyed
and that responded to this question, indicating at least medium or
advanced level computer and Internet skills, by the total number of
education stakeholders that responded to this question. Move the
decimal two places to the right and add a percentage sign (e.g. 65
of 250 respondents report at least medium level computer and
Internet skills: 65/250=0.26 or 26%).
Filled-in questionnaires, and summary on data analysis; focus
group guides and notes from focus group discussions if
applicable
e.g., Project Manager
Twice Once at project inception, once at the end of the grants
program
e.g .Project Manager
twice Based on grant implementation schedule
First progress report and final report with summary on
evaluation procedure and instruments, summary on baseline results,
disaggregated by user group and gender, accompanied by copies of
original filled-in questionnaires from education stakeholders, and
a summary on each focus group discussion if applicable.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
Questionnaire-based surveying may be complemented with focus
groups with different education stakeholders to find out in more
detail about their current ICT use, skills, knowledge and
needs.
Disaggregate by gender and user group2
5 We still need to define the specific criteria defining basic,
medium and advanced level computer and Internet skills. Maybe Tita
has something that is used in an Indonesian
context already, if not, I may have some international
examples
-
20
# Indicator Indicator Definition Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1
By whom
How often When Format
3-24.1.e
% of education stakeholders reporting that the training has
provided them with critical skills for their work or studies
This indicator tracks the impact of the training in terms of
education stakeholders’ work or studies. It tracks education
stakeholders responding to a questionnaire and indicating that the
ICT-related training they received was “very critical” or “somewhat
critical” for their work and studies.
Capture data with the help of a self-assessment questionnaire
given to a representative sample of education stakeholders that
participated in an ICT-related training, conducted as part of the
grants program.
Divide the number of education stakeholders that were surveyed
and that responded to this question, indicating that the training
has provided them with “very critical”, or “somewhat critical”
skills for their work and studies, by the total number of education
stakeholders that responded to this question. Move the decimal two
places to the right and add a percentage sign (e.g. 160 of 250
respondents report that the training has provided them with
critical skills: 160/250=0.64 or 64%).
Filled-in questionnaires and summary on data analysis; focus
group guides and notes from focus group discussion if
applicable.
e.g., Project Manager
once Once at the end of the grants program
e.g., Project Manager
once Based on grant implementation schedule
Final report with summary on evaluation procedure and
instruments, summary on evaluation results, disaggregated by user
group and gender, accompanied by copies of original filled-in
questionnaires from education stakeholders, and a summary on each
focus group discussion if applicable.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
Questionnaire-based surveying may be complemented with focus
groups with different education stakeholders to find out in more
detail about their current ICT use, skills, knowledge and
needs.
Analyze open ended question on what skills and how they are
employed applying qualitative data analysis methods.
As part of the evaluation, inquire from education stakeholders
which specific skills they learned are critical and have changed
their every day work or studies.
Disaggregate by gender and user group2
-
21
# Indicator Indicator Definition Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1
By whom
How often When Format
3-24.1.f
% of education stakeholders reporting that after the training,
they have used ICT more frequently in their work than before
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
Carry out a separate baseline and a training impact evaluation,
e.g., providing questionnaires to a sample of education
stakeholders.
Divide the number of education stakeholders that were surveyed
and that responded to this question, indicating that they use ICT
more often for their work or studies than before their
participation in grantee-organized training, by the total number of
education stakeholders that responded to this question. Move the
decimal two places to the right and add a percentage sign (e.g. 134
of 250 respondents report using ICT regularly: 134/250=0.54 or
54%)6
Filled-in questionnaires, and summary on data analysis; focus
group guides and notes from focus group discussion if
applicable.
e.g., Project Manager
Once Onceat the end of the grants program
e.g, Project Manager
Once Based on grant implementation schedule
Final report with summary on evaluation procedure and
instruments, summary on baseline results, disaggregated by user
group and gender, accompanied by copies of original filled-in
questionnaires from education stakeholders, and a summary on each
focus group discussion if applicable.
“Use” is defined as using computers for any work-related
activity, using the Intranet or Internet, or using any education
information system or portal for work or studies.
Questionnaire-based surveying may be complemented with focus
groups with different education stakeholders to find out in more
detail about their current ICT use, skills, knowledge and
needs.
Disaggregate by gender and user group7
6 Rounded for two decimals
7 For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators
(diknas [incl. DPL staff], depag, DEB) are one user group,
school administrators (principal, vice-principal, other school
admin staff) are another, teachers, students and
parents each are separate user groups as well
-
22
B. ELECTRONIC EDUCATION RESOURCES
The proposed indicators below are for grantees that build
capacity among education stakeholders to develop and evaluate
electronic education resources (and in parallel build
general computer skills among process participants and improve
collaboration).
# Indicator Indicator Definition Data Collection
Method Method of Calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often
When Format
3-24.1.g
# of education institutions (schools and/or DEO/Kandepag/DEB
sections) involved in the e-resources development process
“Education institutions” are the institutions from which
education stakeholders come, that is: diknas, depag, DEB, DPL,
universities, schools, school committees, district public
libraries, teacher unions, etc.
Make inventory of people that are engaged in the resource
development process, track name and education institution
affiliation of each person
Count the number of different institutions represented in the
resource development process.
List of participants that explicitly highlights their position
and institutional affiliation;
signed participant’s list
e.g., Project Manager
once Based on project work plan, e.g. within month 1 of the
project inception
e.g. Project Manager
once Based on project work plan, e.g. within 30 days of grant
award
Initial or first process report with summary on participants of
the resource development process
3-24.1.h
# of resource development meetings held
This indicator tracks the number of meetings held during
resource development.
Collect meeting invitations and meeting participant lists
Count number of meetings held within one reporting period.
Meeting invitations; signed participant’s lists
e.g. Project Manager
Regularly by reporting period
Based on project work plan
e.g. Project Manager
Quarterly (or by progress reporting period)
Based on grant implementation schedule
Initial report,, and each progress report and final report, with
summary on activities carried out in the last quarter, highlight
number of resource development meetings held; copies of signed
meeting participants lists
3-24.1.i
% of resource development meetings attended by members of
Trying to assess the level of activity of diknas and depag staff
in the resource development process, this indicator intents to
track the percentage of resource development
Ensure that at every meeting participants sign a participant
list, including
Divide number of meetings at which all depag and diknas
representatives, who are involved in the process, were present by
the total number of
Signed participant’s lists after every meeting
e.g., Project Manager
Regularly by reporting period
Based on project work plan
e.g. Project Manager
Quarterly (or by progress reporting period)
Based on grant implementation schedule
Progress reports and final report with summary of meetings
held
-
23
# Indicator Indicator Definition Data Collection
Method Method of Calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often
When Format
all institutions involved
group meetings, at which all depag, diknas and DPK participants
were in attendance.
providing their name, position and affiliation with an education
institution, such as diknas, depag, university, etc.
resource development group meetings held in the last
quarter.
Move the decimal two places to the right and add a percentage
sign (e.g. at 3 of 6 meetings, all participants that are part of
the process were present: 3/6 = 0.5 or 50%).
and at what date during the last reporting period, accompanied
with copies of the originally signed participant lists
3-24.1.j
% of participants in the resource development process reporting
to have acquired critical skills and knowledge for their work
This indicator tracks the extent to which being involved in the
resource development process has increased skills and knowledge
among participants. In detail, it will track what percentage of the
participants explicitly express that involvement in the project has
provided them with “very critical “ or “somewhat critical” skills
and knowledge for their work.
Conduct a survey and/or interviews among the resource
development participants
Divide the number of resource development participants that
expressed having acquired “very critical” or “critical” skills in
the process, by the total number of participants involved in the
process.
Disaggregate by user group8 and gender
Survey questionnaires; interview guides and notes
e.g. Project Manager
once Based on project work plan, at the end of the grants
program
e.g. Project Manager
once Based on grant implementation schedule
Final report with summary on survey outcomes on skills,
knowledge and attitude of resource development participants; with
copies of original, filled-in questionnaires.
3-24.1.k
# of education institutions to which final resources have been
distributed
This indicator tracks the extent to which the e-resource
developed have been distributed to other education
institutions.
“Education institutions” are the institutions from which
education stakeholders come, that is: diknas, depag, DEB, DPL,
universities, schools, school committees, district public
libraries, teacher unions, etc.
Track which institutions the resources have been shared with, or
sent to.
Count the number of institutions.
Tracking list of resource package recipients
e.g. Project Manager
Once Based on project work plan, at the end of the grants
program
e.g. Project Manager
once Based on grant implementation schedule
Final report with list of institutions that have received the
resource package(s).
8 For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators (diknas [incl.
DPL
staff], depag, DEB) are one user group, school administrators
(principal, vice-principal, other school admin staff) are another,
teachers, students and parents each are separate user groups as
well.
-
24
# Indicator Indicator Definition Data Collection
Method Method of Calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often
When Format
3-24.1.l
% of respondents in education institutions having received the
resources reporting them useful for their teaching or
self-study
This indicator tracks how users value the resources for their
teaching or self-study.
“Education institutions” are the institutions from which
education stakeholders come, that is: diknas, depag, DEB, DPL,
universities, schools, school committees, district public
libraries, teacher unions, etc.
Conduct questionnaire survey among members of those education
institutions that have received the resources.
Questionnaire-based surveying may be complemented with focus
groups
Divide the number of respondents that were surveyed and that
responded to this question, indicating that the they find the
resources “very valuable”, or “valuable” for their teaching or
self-study, by the total number of respondents to this question.
Move the decimal two places to the right and add a percentage sign
(e.g. 160 of 250 respondents report that the resources are useful:
160/250=0.64 or 64%).
Survey questionnaires; focus group guides and notes if
applicable
e.g. Project Manager
Once Based on project work plan, at the end of the grants
program
e.g. Project Manager
once Based on grant implementation schedule
Final report with summary on evaluation procedure and
instruments, summary on evaluation results, disaggregated by user
group and gender, accompanied by copies of original filled-in
questionnaires from education stakeholders, and a summary on each
focus group discussion if applicable.
-
25
IV. SIR 3-24.2: INCREASED ACCESS TO EDUCATION INFORMATION –
INDICATORS
A. EDUCATION PORTAL/SYSTEM DEVELOPMENT
The proposed indicators below are for grantees that develop a
publicly accessible electronic management information system, such
as a web-
based education information system or library research system,
SMS-supported education or library information
system/gateway/center, or an
education portal of some kind.
# Indicator Indicator Definition Data Collection
Method Method of calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often When Format
3-24.2.a
# of times public system /portal has been accessed
This indicator tracks the frequency of system/portal use on a
daily basis.
“Accessed” is defined by a registered user logging in to the
system/portal, or, in an open system/portal, by a successful
request to the server.
Use of system /portal tracking statistics software. Where use
statistics should disaggregate by user group and gender, if
individual user profiles are being tracked through registration
before use.
Based on system/portal , statistical analysis on use, extract
the number of times the system has been access per day.
Calculate average of daily access per reporting period.
Print-out, summary graphics of the statistics on system/portal
use
e.g. Project Manager
Tracked daily
End of every day
e.g. Project Manager
Every reporting period, starting with launch of system
Based on grant implementation schedule
Progress report with summary on system statistics, e.g. via
graphics, charts or statistical tables
3-24.2.b
% of education stakeholders reporting use of the
system/portal
This indicator tracks the percentage of education stakeholders
who respond to a survey and who report using the system/portal.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
“Use of the system/portal” is defined as accessing the
system/portal (
Survey questionnaire send to a sample of education stakeholders
(representative of every user group).
Divide the number of education stakeholders that state that they
use the system/portal, by the total number of education
stakeholders that responded to this question. Move the decimal two
places to the right and add a percentage sign (e.g. 20 of 50
respondents use the system: 20/50=0.4 or 40%).
Report on frequency of use by asking respondents that report
Actual surveys (questionnaire) sent back by users and
analyzed
e.g., Project Manager
Once per year
Once after 6 months of system/ portal existence, from then on
every 12 months
e.g. Project Manager
Twice or three times based on grant timeline
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and samples of filled-in survey questionnaire
-
26
# Indicator Indicator Definition Data Collection
Method Method of calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often When Format
online or offline) for at least 5 minutes to either acquire or
upload education information of any kind
using the system to indicate at what level of frequency (once,
daily, weekly, bi-weekly, monthly, etc.)
Disaggregate by user group9 and gender.
3-24.2.c
% of education stakeholders reporting that the system/portal as
increased ease of access to education information
This indicators tracks the percentage of education stakeholders
who respond to a survey and report that the system as has made it
easier for them to access education information.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
Survey, questionnaire
Divide the number of education stakeholders that respond that
one of the benefits of the system/portal is “easier access to
education information”, by the total number of education
stakeholders that responded to this question. Move the decimal two
places to the right and add a percentage sign (e.g. 30 of 50
respondents report that the system/portal increased ease of access
to information: 30/50=0.6 or 60%).
Disaggregate by user group10 and gender.
Actual surveys (questionnaire) sent back by users and
analyzed
e.g Project Manager
Once per year
Once after 6 months of system/ portal existence, from then on
every 12 months
e.g. Project Manager
Twice or three times based on grant timeline
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and samples of filled-in survey questionnaire
3-24.2.d
% of education administrators (DEO, Kandepag,
This indicator tracks the percent of education administrators
that report the system/portal having improved communication among
each other.
“Education Administrators” are
Survey, questionnaire
Divide the number of education stakeholders that respond that
one of the benefits of the system/portal is “improved
Actual surveys (questionnaire) sent back by
e.g Project Manager
Once per year
Once after 6 months of system/
e.g. Project Manager
Twice or three times based on grant
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and
9 For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators (diknas [incl.
DPL staff], depag, DEB) are one user group, school
administrators (principal, vice-principal, other school admin
staff) are another, teachers, students and parents each are
separate user
groups as well. 10
For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators (diknas [incl. DPL
staff], depag, DEB) are one user group, school administrators
(principal, vice-principal, other school admin staff) are another,
teachers, students and parents each are separate user
groups as well.
-
27
# Indicator Indicator Definition Data Collection
Method Method of calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often When Format
DEB) reporting that the system/portal has improved communication
among each other
representatives from the following institutions: DEO, Kandepag,
and District Education Board (DEB).
communication”, by the total number of education stakeholders
that responded to this question. Move the decimal two places to the
right and add a percentage sign (e.g. 30 of 50 respondents report
that the system/portal improved communication: 30/50=0.6 or
60%).
Disaggregate by gender.
users and analyzed
portal existence, from then on every 12 months
timeline
samples of filled-in survey questionnaire
3-24.2.e
% of education administrators (DEP, Kandepag, DEB) reporting
that the system/portal has improved time-efficiency in their
work
This indicator tracks the percent of education administrators
that report the system/portal having improved time-efficiency in
their work.
“Education Administrators” are representatives from the
following institutions: DEO, Kandepag, and District Education Board
(DEB).
Survey, questionnaire
Divide the number of education stakeholders that respond that
one of the benefits of the system/portal is “improved
time-efficiency”, by the total number of education stakeholders
that responded to this question. Move the decimal two places to the
right and add a percentage sign (e.g. 30 of 50 respondents report
that the system/portal improved time-efficiency: 30/50=0.6 or
60%).
Disaggregate by gender.
Actual surveys (questionnaire) sent back by users and
analyzed
e.g. Project Manager
Once per year
Once after 6 months of system/ portal existence, from then on
every 12 months
e.g. Project Manager
Twice or three times based on grant timeline
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and samples of filled-in survey questionnaire
3-24.2.f
% of education administrators (DEP, Kandepag, DEB) reporting
that the system/portal has facilitated reporting
This indicator tracks the percent of education administrators
that report the system/portal has facilitated reporting of
education data
“Education Administrators” are representatives from the
following institutions: DEO, Kandepag, and District Education Board
(DEB).
Survey, questionnaire
Divide the number of education stakeholders that respond that
one of the benefits of the system/portal is “facilitated reporting
of education data”, by the total number of education stakeholders
that responded to this question. Move the decimal two places to the
right and add a percentage
Actual surveys (questionnaire) sent back by users and
analyzed
e.g. Project Manager
Once per year
Once after 6 months of system/ portal existence, from then on
every 12
e.g. Project Manager
Twice or three times based on grant timeline
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and samples of filled-in survey questionnaire
-
28
# Indicator Indicator Definition Data Collection
Method Method of calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often When Format
education data
sign (e.g. 30 of 50 respondents report that the system/portal
facilitated education data reporting: 30/50=0.6 or 60%).
Disaggregate by gender.
months
B. INTERNET CONNECTIVITY/LOCAL AREA NETWORK
The proposed indicators below are for grantees that focus
activities on connecting education stakeholders either via Internet
connectivity or
Local Area Networks (IT centers) of some kind.
# Indicator Indicator Definition
Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1 By
whom How often When Format
3-24.2.g
% of DEO and Kandepag sections and/or sub-district offices
and/or schools connected
This indicator captures the percentage of DEO and Kandepag
sections and/or sub-district offices and/or schools that have been
connected under this grants program.
“connected” defines a functioning, dedicated internet or
intranet (LAN, WAN) connection between two or more
sections/offices/schools
Conduct site observations or a short survey.
Survey should capture whether the section/offices/school had
been connected as part of the grants program or had previously been
connected.
Divide the number of sections/offices/schools that are connected
(either by the DBE grants program or otherwise) by the total number
of sections/offices/schools in the district.
Move the decimal two places to the right and add a percentage
sign (e.g. 31 of 96 sections/offices/schools are connected:
31/96=0.32 or 32%)11.
Site observation notes or survey questionnaire
e.g. Project Manager
once Based on project work plan, after connectivity has been
established
Project Manager
once Based on grant implementation schedule, after connectivity
has been established, e.g. in quarterly report July 07
Progress or final report including summary of survey analysis,
sample of original questionnaires filled out or copies of original
site observation notes.
3-24.2.h
% of education stakeholders reporting to exchange
This indicator tracks the percentage of education stakeholders
that report to use the network/connectivity established by the DBE1
grantee, and exchange information with other
Survey, questionnaire
Divide the number of education stakeholders that respond that
they use the connectivity/network for information exchange with
Actual surveys (questionnaire) sent back by
e.g. Project Manager
Once per year
Once after 6 months of system/
e.g. Project Manager
Twice or three times based on grant
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
11
Rounded for two decimals.
-
29
# Indicator Indicator Definition
Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1 By
whom How often When Format
information with other education stakeholders
education stakeholders.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
“Exchanging information” include activities, such as sending an
email, sharing files, saving a document in a shared folder,
accessing documents and files from somebody else in shared
folders.
other education stakeholders by the total number of education
stakeholders that responded to this question. Move the decimal two
places to the right and add a percentage sign (e.g. 30 of 50
respondents report that they use the connectivity/ network for
information exchange : 30/50=0.6 or 60%).
Report on frequency of information exchange by asking
respondents that report using the connectivity/network for
information exchange to indicate at what level of frequency (once,
daily, weekly, bi-weekly, monthly, etc.)
Disaggregate by gender and user group12.
users and analyzed
portal existence, from then on every 12 months
timeline
survey, and samples of filled-in survey questionnaire
3-24.2.i
% of education stakeholders reporting easy access to critical
information
This indicator tracks the percentage of education stakeholders
who respond to a survey and report that the connectivity/network
established by the DBE1 grantee as has made it easier for them to
access critical information.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers,
Survey, questionnaire
Divide the number of education stakeholders that respond that
one of the benefits of the connectivity/network is “easier access
to critical information”, by the total number of education
stakeholders that responded to this question. Move the decimal two
places to the right and add a percentage sign (e.g. 30 of 50
respondents report that the connectivity/network increased
Actual surveys (questionnaire) sent back by users and
analyzed
e.g. Project Manager
Once per year
Once after 6 months of system/ portal existence, from then on
every 12 months
e.g. Project Manager
Twice or three times based on grant timeline
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and samples of filled-in survey questionnaire
12
For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators (diknas [incl. DPL
staff], depag, DEB) are one user group, school administrators
(principal, vice-principal, other school admin staff) are another,
teachers, students and parents each are separate user groups as
well.
-
30
# Indicator Indicator Definition
Data Collection Method
Method of Calculation Data Proof Who How often
When
Data reported to DBE1 By
whom How often When Format
students, and parents.
“critical” information is defined as information relevant to
respondents’ work.
ease of access to information: 30/50=0.6 or 60%).
Disaggregate by user group13 and gender.
3-24.2.j
% of education stakeholders reporting easy access to ICT
This indicator tracks the percentage of education stakeholders
who respond to a survey and report that the connectivity/network
established by the DBE1 grantee as has made it easier for them to
access ICT in general.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
“critical” information is defined as information relevant to
respondents’ work.
Survey, questionnaire
Divide the number of education stakeholders that respond that
one of the benefits of the connectivity/network is “easier to
access ICT”, by the total number of education stakeholders that
responded to this question. Move the decimal two places to the
right and add a percentage sign (e.g. 30 of 50 respondents report
that the connectivity/network increased ease of access to ICT:
30/50=0.6 or 60%).
Disaggregate by user group14 and gender.
Actual surveys (questionnaire) sent back by users and
analyzed
e.g. Project Manager
Once per year
Once after 6 months of system/ portal existence, from then on
every 12 months
e.g. Project Manager
Twice or three times based on grant timeline
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and samples of filled-in survey questionnaire
C. LIBRARY INFORMATION SYSTEM
The proposed indicators below are for grantees that develop an
electronic Library Information System solely for the purpose of
increasing
efficiency of library services, such as membership management,
library resource management or library transaction management.
13
For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators (diknas [incl. DPL
staff], depag, DEB) are one user group, school administrators
(principal, vice-principal, other school admin staff) are another,
teachers, students and parents each are separate user
groups as well. 14
For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators (diknas [incl. DPL
staff], depag, DEB) are one user group, school administrators
(principal, vice-principal, other school admin staff) are another,
teachers, students and parents each are separate user groups as
well.
-
31
# Indicator Indicator Definition Data Collection
Method Method of calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often When Format
3-24.2.k
% of library staff reporting use of the system
This indicator tracks the percentage library who respond to a
survey and who report using the system.
“Use of the system/” is defined as accessing the system (online
or offline) for at least 5 minutes to manage library information of
any kind
Survey questionnaire send to all library staff.
Divide the number of library that state that they use the
system, by the total number of library staff that responded to this
question. Move the decimal two places to the right and add a
percentage sign (e.g. 20 of 50 respondents use the system:
20/50=0.4 or 40%).
Report on frequency of use by asking respondents that report
using the system to indicate at what level of frequency (once,
daily, weekly, bi-weekly, monthly, etc.)
Disaggregate by gender.
Actual surveys (questionnaire) sent back by users and
analyzed
e.g., Project Manager
Once per year
Once after 6 months of system/ portal existence, from then on
every 12 months
e.g. Project Manager
Twice or three times based on grant timeline
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and samples of filled-in survey questionnaire
3-24.2.l
% of library staff reporting that the system has improved
time-efficiency in their work
This indicator tracks the percent of library staff that report
the system has having improved time-efficiency in their work.
Survey, questionnaire
Divide the number of library staff that respond that one of the
benefits of the system is “improved time-efficiency”, by the total
number of library staff that responded to this question. Move the
decimal two places to the right and add a percentage sign (e.g. 30
of 50 respondents report that the system/portal improved
time-efficiency: 30/50=0.6 or 60%).
Disaggregate by gender.
Actual surveys (questionnaire) sent back by users and
analyzed
e.g. Project Manager
Once per year
Once after 6 months of system/ portal existence, from then on
every 12 months
e.g. Project Manager
Twice or three times based on grant timeline
Based on grant implementation schedule
Progress and/or Final report including an analysis of the
survey, and samples of filled-in survey questionnaire
V. SIR 3-24.3: INCREASED ACCESS TO ICT – INDICATORS
A. INTERNET ACCESS POINT
-
32
The proposed indicators below are for grantees that establish
public Internet access points of some kind (Warnet, hotspot,
internet café).
# Indicator Indicator Definition Data Collection
Method Method of Calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often When Format
3-24.3.a
# of education stakeholders using the public access point
This indicator tracks the number of education stakeholders using
the public access point.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators (from diknas, Kandepag, or
DEB), school administrators (school principal, vice-principal,
other school administration staff), teachers, students, and
parents.
“Using” is defined as entering the public access point
facilities at least once and purchasing and/or consume a specific
service (e.g. Internet access) or attend a training course or
workshop.
Track visits of education stakeholders with the help of a daily
user log;
Track number of users via user logs.
Calculate average of daily access per reporting period.
Disaggregate data by gender and user group15.
User logs e.g. Project Manager
Tracked daily
End of every day
e.g. Project Manager
Every reporting period, starting with launch of public access
point
Based on grant implementation schedule
Progress report with overview on public access point use, e.g.
attached user logs or print outs thereof
3-24.3.b
# of education stakeholders reporting access to ICT via the
public access point to be critical for their work or studies
This indicators tracks the number of education stakeholder that
use the public access point and responding to a questionnaire,
indicate, that access to computers and/or the Internet via this
access point is “very critical” or “somewhat critical” for their
work and studies.
“Education stakeholders” are defined as members of the following
5 user groups: education administrators
Capture data with the help of a questionnaire given to public
access point users.
Count number of users that respond to this question and say that
they find access to ICT (computer/Internet) via the public access
point “very critical” or “somewhat critical” to their work and/or
studies.
Disaggregate by user group16 and gender.
Filled-in questionnaire
Project Manager
Twice in year one, then yearly
After month 6 and 9 of operation, then yearly
Project Manager
Twice in year one, then yearly
After month 6 and 9 of operation, then yearly
Progress and final report including an analysis of the survey,
and samples of filled-in survey questionnaire
15
For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators (diknas [incl. DPL
staff], depag, DEB) are one user group, school administrators
(principal, vice-principal, other school admin staff) are another,
teachers, students and parents each are separate user
groups as well. 16
For the purpose of this M&E framework, 5 different user
groups have been identified and are made up of different education
stakeholders, that is, education administrators (diknas [incl. DPL
staff], depag, DEB) are one user group, school administrators
(principal, vice-principal, other school admin staff) are another,
teachers, students and parents each are separate user
groups as well.
-
33
# Indicator Indicator Definition Data Collection
Method Method of Calculation Data Proof Who
How often
When
Data reported to DBE1
By whom
How often When Format
(from diknas, Kandepag, or DEB), school administrators (school
principal, vice-principal, other school administration staff),
teachers, students, and parents.
3-24.3.c
% of months in operation, where revenue exceeds ongoing
costs
This indicator requires that the monthly revenue exceeds ongoing
costs.
“Monthly revenue” includes all income generated by services
offered at the public access point, such as internet access,
training courses, information research services, printing, etc.
“Ongoing costs” include all costs required to keep the public
access point running, including salaries, rents, communication
expenses, stationary expenses, minor repair expense on both,
building and equipment, all applicable taxes and fees.
Diligently manage accounting records and do a monthly analysis
of ongoing costs and revenue cash flows.
Count the number of months, in the last quarter, in which
monthly revenue has exceeded on-going costs.
Accounting records
e.g., Project Manager
Monthly After every calendar month
e.g., Project manager
Quarterly (or by reporting period)
Based on grant implementation schedule
Progress and Final Report including detailed overview on
on-going costs and revenue generated each month in the past
reporting period from selling services such as connectivity,
bandwidth, training, computer repair, computer equipment trading,
communication services for non-(computer) literate community
-
34
APPENDIX 3
ICT Grants Program Indicators
and Customized Grantee
Indicator Tracking Sheets
-
35
ICT GRANT PROGRAM INDICATOR
KEGIATAN INDIKATOR PETUNJUK
PUBLIC INTERNET ACCESS POINT (Hotspot/Warnet/
Internet Café)
# stakeholder pendidikan yang menggunakan hotspot Mohon didata
berapa orang stakeholder pendidikan (pegawai administrasi
sekolah/kec/kab/kota, guru, murid, orangtua, dsb) yang menggunakan
hotspot/Warnet/Internet Café dalam satu bulan
# stakeholder pendidikan yang yang menyatakan bahwa akses ICT
sangat penting bagi pekerjaan atau pendidikan mereka
Dalam periode satu bulan, edarkan questioner secara sederhana
kepada stakekholder pendidikan, apakah akses terhadap
hotpost/warnet/internet café tersebut (1) tidak penting; (2) kurang
penting; (3) penting; (4) sangat penting dalam pekerjaan atau
pendidikan mereka. Kemudian buatlah rekapitulasi berapa orang atau
persen yang menyatakan tidak, cukup, dan sangat penting
% bulan beroperasi dimana pendapatan melebihi biaya berjalan
Cukup dicatat berapa pengeluaran yang diperlukan untuk mengelola
hotpsot/warnet/internet café dan berapa pendapatan yang diperoleh
setiap bulanya. Catat mulai bulan keberapa pendapatan melebihi
pengeluaran
MIS (LIBRARY, EMIS, SMS Gateway or Education
Portal)
Berapa kali dalam satu bulan public system/portal diakses
Untuk mengisi data ini, cukup didata setiap hari orang-orang
yang mengakses portal dan buatlah rekapitulasi dalam satu bulan
% stakeholder pendidikan yang melaporkan penggunaan
system/portal
Secara acak (bukan hanya pengguna saja) lakukan wawancara dengan
stakeholder pendidikan (pegawai administrasi sekolah/kec/kab/kota,
guru, murid, orangtua, dsb) masing-masing sebanyak sebanyak 2)
orang, dan buat rekapitulasi berapa di antara mereka yang
menggunakan portal yang didanai oleh DBE1.
% stakeholder pendidikan yang melaporkan bahwa system/portal
memudahkan akses informasi pendidikan
Dalam tempo satu bulan, edarkan questioner secara sederhana
kepada setiap pengunjung, apakah keuntungan dengan adanya
system/portal: (1) tidak ada pengaruhnya, (2) tidak tersedia alat
bantu, (3) memudahkan; (4) sangat memudahkan dalam mengakses
informasi pendidikan.
% administrator pendidikan yang menyatakan bahwa system/portal
telah memperbaiki komunikasi diantara mereka
Mohon ditanyakan kepada (masing-masing 2 orang) staf Dinas
Pendidikan, perpustakaan daerah, kandepag, dewan pendidikan.
Bappeda, kesra pertanyaan sbb: Dibandingkan sebelum ada portal yang
didanai DBE1, apakah komunikasi di antara mereka saat ini (1)
menjadi lebih baik; (2) tetap sama/tidak ada perubahan.
% administrator pendidikan yang menyatakan bahwa system/portal
telah membuat waktu lebih efisien
Mohon ditanyakan kepada (masing-masing 2 orang) staf Dinas
Pendidikan, perpustakaan daerah, kandepag, dewan pendidikan.
Bappeda, kesra pertanyaan sbb: Dibandingkan sebelum ada portal yang
didanai DBE1, apakah keuntungan yang didapat? (1) waktu menjadi
efisien; (2) komunikasi dengan institusi lain menjadi efektif; (3)
lebih sering mengakses system/portal dibanding sebelumnya
-
36
KEGIATAN INDIKATOR PETUNJUK
% administrator pendidikan yang melaporkan bahwa system/portal
telah menfasilitasi mereka dengan data pendidikan
Mohon ditanyakan kepada (masing-masing 2 orang) staf Dinas
Pendidikan, perpustakaan daerah, kandepag, dewan pendidikan.
Bappeda, kesra pertanyaan sbb: Apakah system/portal telah
menfasilitasi mereka dengan data pendidikan? (1) Ya; (2) Tidak.
Kemudian buatlah rekapitulasi berapa orang/persen yang menyatakan
"tidak" dan berapa "ya"
eRESOURCES (electronic
teaching and learning
materials)
# institusi pendidikan (sekolah, Disdik, Kandepag, Dewan
Pendidikan) yang terlibat dalam pengembangan eResources
Mohon dicatat jumlah institusi pendidikan yang terlibat dalam
pengembangan eResources
# pertemuan pengembangan eResources Mohon dicatat berapa kali
dalam satu bulan pertemuan diadakan
% pertemuan pengembangan eResources yang dihadiri oleh seluruh
anggota dari institusi yang terlibat
Mohon dicatat berapa kali dalam satu bulan pertemuan yang
dihadiri oleh seluruh institusi pendidikan diadakan
% peserta pengembangan eResources yang melaporkan bahwa mereka
telah mendapatkan keterampilan dan pengetahuan yang sangat berharga
untuk pekerjaan mereka
Mohon ditanyakan kepada peserta pengembangan eResources
pertanyaan sbb: Apakah electronic teaching dan learning material
itu meningkatkan ketrampilan dan pengetahuan yang sangat membantu
dalam pekerjaan mereka? (1) Ya; (2) tidak. Mohon direkap berapa
yang menyatakan "ya" dan berapa yang menyatakan "tidak"
# institusi pendidikan dimana eResources yang final telah
didistribusikan
Mohon dicatat jumlah institusi pendidikan yang telah mendapatkan
eResources yang telah final (sekolah, dinas pendidikan, Dewan
pendidikan, dsb.)
% responden dari institusi pendidikan yang telah memperoleh
eResources yang menyatakan bahwa eResources tersebut sangat
bermanfaat untuk mengajar atau belajar sendiri
Mohon ditanyakan kepada anggota institusi pendidikan (minimal
masing-masing 2 orang) pertanyaan sbb: Apakah eResources tersebut
bermanfaat bagi proses belajar mengajar? (1) Ya; (2) Tidak. Rekap
jawaban responden dan nyatakan berapa yang menyatakan "Ya"
(bermanfaat) dan berapa yang "tidak" (tidak bermanfaat.
IT CENTER/ CONNECTIVITY/ NETWORK
% Seksi dari Diknas/UPTD/Kancam atau sekolah yang telah
tersambung
Harap didata, dari kabupaten/kota yang bersangkutan, berapa yang
telah mempunyai sambungan internet (untuk melihat % sekolah,
bandingkan proporsi sekolah yang punya internet dengan yang belum,
% seksi dari diknas, bandingkan jumlah seksi yang punya internet
dan yang tidak; demikian selanjutnya untuk KCD/UPTD.
% stakeholder pendidikan yg melaporkan terjadinya pertukaran
informasi dengan stakeholder pendidikan lainnya
Mohon ditanyakan kepada staf Diknas/ UPTD/KCD/sekolah
(masing-masing 2 orang) yang mempunyai jaringan ICT yang didanai
oleg DBE1 : Apakah mereka melakukan pertukaran informasi dengan
stakeholder lainya? (1) Ya; (2) Tidak. Jika "ya" berapa kali dalam
satu bulan?
-
37
KEGIATAN INDIKATOR PETUNJUK
% stakeholder pendidikan yang melaporkan kemudahan akses
informasi yang sangat dibutuhkan
Harap ditanyakan kepada staf diknas/UPTD/KCD atau guru
(masing-masing 2 orang) pertanyaan sbb: apakah keuntungan dengan
adanya jaringan internet bagi saudara? (1) memudahkan akses
informasi yang saya butuhkan dalam bekerja; (2) memudahkan
pertukaran informasi; (3) memudahkan berubungan dengan institusi
lain (Catatan: Jawaban boleh lebih dari 1).
% stakeholder pendidikan yang melaporkan akses ke ICT secara
mudah
Harap ditanyakan kepada staf diknas/UPTD/KCD atau guru
(masing-masing 2 orang) pertanyaan sbb: Dibandingkan dengan saat
sebelum ada jaringan ICT yang dibangun DBE1, bagaimana akses ke ICT
pada saat ini? (1) lebih mudah; (2) sama saja.
TRAINING
# stakeholder pendidikan yang menghadiri pelatihan ICT atau
program sejenisnya
Mohon didata berapa orang stakeholder pendidikan (pegawai
administrasi sekolah/kec/kab/kota, guru, murid, orangtua, dsb) yang
mengikuti pelatihan ICT atau sejenisnya. (Bisa dibandingkan sebelum
ada bantuan DBE1 untuk baseline dan sesudah ada bantuan DBE1)
% stakeholder pendidikan yang sukses mengikuti kursus ICT atau
program sejenisnya
Mohon didata berapa orang stakeholder pendidikan (pegawai
administrasi sekolah/kec/kab/kota, guru, murid, orangtua, dsb) yang
mengikuti pelatihan ICT atau sejenisnya sampai selesai dan
memperoleh sertifikat dan berapa yang tidak selesai/tidak
memperoleh sertifikat.
% stakeholder pendidikan yang melanjutkan kursus ICT atau
program sejenisnya
Mohon didata berapa orang stakeholder pendidikan (pegawai
administrasi sekolah/kec/kab/kota, guru, murid, orangtua, dsb) yang
mengikuti pelatihan ICT atau sejenisnya sampai selesai dan berapa
orang yang kemudian melanjutkan kursus lain yang sejenis.
% stakeholder pendidikan yag mempunyai ketrampilan
komputer/internet tingkat menengah (medium) atau lanjutan
(advanced)
Mohon didata berapa orang stakeholder pendidikan (pegawai
administrasi sekolah/kec/kab/kota, guru, murid, orangtua, dsb) yang
mengikuti pelatihan ICT atau sejenisnya dan apakah mereka bisa
mengoperasikan program berikut ini: (1) Word Processing; (2)
Spreadsheet; (3) Power point; (4) Browsing internet; (5)
internet/emal; (6) download file; (7) SIM. Categori: menguasi 4
program = Medium; 5 atau lebih
program=advanced
% stakeholder pendidikan yang menyatakan bahwa pelatihan telah
meningkatkan skill yang amat penting diperlukan dalam pekerjaan
atau studi mereka
Secara acak, tanyakan kepada peserta pelatihan (masing-masing
stakeholder 2 orang) pertanyaan sbb: Apakah pelatihan yang mereka
ikuti berhasil meningkatkan skill yang diperlukan dalam pekerjaan
dan pendidikan mereka? (1) Ya; (2) Tidak. Mohon direkapitulasi
berapa persen yang menyatakan "ya" dan berapa yang "tidak"
% stakeholder pendidikan yang menyatakan bahwa setelah training,
mereka telah menggunakan ICT lebih sering dibandingkan
sebelumnya
Secara acak, tanyakan kepada peserta pelatihan (masing-masing
stakeholder 2 orang) pertanyaan sbb: Dibandingkan dengan saat
sebelum mengikuti pelatihan, seberapa sering sdr menggunakan ICT?
(1) Sama; (2) lebih sering. Mohon direkap berapa persen yang
mengatakan "sama" dan berapa yang mengatakan "lebih sering"
-
38
KEGIATAN INDIKATOR PETUNJUK
Education Stakeholders: 1. Education Administrators
(Administrator Pendidikan),
2. School Administrators
3. Teachers
4. Students
5. Parents
6. others
Administrator Pendidikan: 1. Dinas Pendidikan
2. Perpustakaan Daerah
3. Kandepag
4. Dewan Pendidikan
5. Bappeda
6. Bagian Kesra
-
39
CUSTOMIZED GRANTEE INDICATOR TRACKING SHEET
JENEPONTO
MAIN ACTIVITY I. LAN NETWORK IN KANDEPAG
Pertanyaan-pertanyaan pokok : Apakah ada peningkatan akses
terhadap ICT bagi para