Assessing the Impact of ICT on Development – WSIS and MDGs ITU-UNU WSIS-MDG Team.

Assessing the Impact of ICT on Development – WSIS and MDGs

ITU-UNU WSIS-MDG Team

WSIS FORUM 2010, GENEVA, 11 MAY 2010, 2Assessing the Impact of ICT on Development – WSIS and MDGs

AIM

The aim of the session is three-fold:

1 To provide an overview of the joint ITU-UNU project on linking WSIS and MDG processes through research in the area of ICT impact assessment

2 To present preliminary results of this project:

oConceptual framework capturing knowledge needs of practitioners involved in developing and implementing ICT4D- and WSIS-related initiatives for better impact on the MDGs

oAnalysis of the WSIS stocktaking database, based on the conceptual framework

3 To highlight next steps of the project, obtain feedback on the project’s design and implementation, and discuss how interested parties could contribute to the project


OVERVIEW

1 PROJECT CONTEXT AND OVERVIEW

2 INITIAL RESULTS

2.1 CONCEPTUAL FRAMEWORK

2.1.1 ICT4D Impact Assessment

2.1.2 ICT and Development Theories

2.1.3 Conceptualizing ICT

2.2 ANALYSIS OF THE WSIS STOCKTAKING DATABASE

2.2.1 WSIS Database Summary

2.2.2 Deducing Plausible WSIS-MDG Connections

3 CASE STUDIES

4 DISCUSSION

5 NEXT STEPS


RATIONALE

Experiences from around the world demonstrate the potential of ICT on development, e.g

oon poverty by increasing market access and competitiveness of the poor, oon education by increasing access to learning through distance education, oon gender by increasing economic opportunities for women,oon health by connecting rural health providers with formal health systemsoon environment by monitoring and mitigation of environmental risks, oon governance by engaging citizens in policy- and decision-making processes, oetc.

However, the empirical evidence is yet to be explained by rigorous impact assessment frameworks:

ohow particular types of ICT initiatives can advance particular development objectivesohow to ensure that ICT portfolios are able to achieve the desired development impact by design.

Relating WSIS Action Lines and MDGs is an instance of a more general problem…


OBJECTIVES

The project has four main objectives:

1)Investigate the impact of WSIS Action Lines on the MDG process.

2)Establish the actual needs of practitioners, member states and international organizations in effectively assessing the impact of their ICT4D initiatives (WSIS Action Line projects) on MDGs

3)Provide an annotated repository of resources (frameworks, toolkits, etc.) to guide practitioners on the use of existing ICT Impact Assessment models, frameworks and measurement systems.

4)Develop a system for monitoring the impact WSIS process on the MDG process, integrated with the WSIS Stocktaking platform.


ACTION FRAMEWORK

Framework for survey design

A7 - Mobilizing

Actions

A6 - Proposal

Articulation

A5 - Issue Investigation

(Survey)

A4 - Analysis of the WSIS Stocktaking Database

A1 - Agenda Settings

Concepts and elements of the framework

A2 - State of the Art Review

Core issues to investigate A3 -ConceptualFramework


ACTION A2 – STATE OF THE ART REVIEW

A2 GOAL To comprehensively review the literature on theories, models, frameworks and approaches for ICT impact assessment

TASKS The review exercise will cover:

odevelopment theories and implications for the use of ICTochange and transformation theories vis-à-vis the nature of ICToimpact assessment model and frameworksoICT-related measurement indicators systems for MDGs

OUTPUTS The Repository of ICT Impact Assessment models and frameworks


ACTION A3 – CONCEPTUAL FRAMEWORK

A3 GOAL To develop a conceptual model and framework to guide ICT impact assessment and design of WSIS Action Line projects and portfolios

TASKS The activity involves:

oDeveloping a conceptual model to relate major concepts in different thematic areas covered in the state-of-the-art review A2

oRelating various theories, models and frameworks identified during A2 to provide a good understanding of the central issues involved in ICT impact assessment and design of ICT4D portfolios.

OUTPUTS Conceptual Framework Report


ACTION A4 – ANALYSIS OF THE WSIS DATABASE

A4 GOAL To investigate the nature of the initiatives documented in the WSIS Stocktaking Database, and attempt at generating some insights into probable connections between WSIS action lines and MDGs. The generated connections will provide initial proposition for further analysis and research in general.

TASKS Achieving this requires:

oSummarizing initiatives with respect to the associated Action Lines and MDGs

oGenerating associations between WSIS activities and MDGs in order to develop hypotheses for further investigating the impact of WSIS on the MDG process

oCarrying out analysis on the nature of the documented initiatives to determine their potential impact profile as a basis for recommendations on portfolio design

OUTPUTS WSIS Stocktaking Analysis Report


ACTION A5 – STATE OF PRACTICE AND NEEDS ASSESSMENT

A5 GOAL Guided by the Conceptual Framework (A3) and the results of the WSIS Stocktaking Database Analysis (A4), establish the state-of-practice in relations to impact assessment of WSIS initiatives.

TASKS Specifically:

oCarry out an online survey and targeted interviews on the nature and degree of impact assessment or evaluation carried out as part of (or as a follow up to) WSIS-related initiatives

oAnalyze the survey findings to establish specific needs of practitioners on the impact assessment of WSIS-related initiatives on the MDGs

OUTPUTS State-of-Practice Report on Impact Assessment of the WSIS process


ACTION A6 – PROPOSAL FOR ADDRESSING NEEDS

A6 GOAL To articulate the needs of practitioners for improved ICT impact assessment, as an action proposal by relevant stakeholders and existing players in this domain.

TASKS The tasks associated with this activity include:

oDevelop a statements of needs for different categories of users and formulate proposals on possible solutions to these needs

oProvide recommendations on other institutional issues such as improving data collection at individual, community, national, regional and global levels

OUTPUTS Proposal for Action


ACTION A7 – MOBILIZING PARTNERS TO ADDRESS NEEDS

A7 GOAL To mobilize practitioners and major international players in the area of impact assessment of ICT on MDGs and development in general, to provide feedback on the proposal and suggestions on the implementation in priority areas.

TASKS This activity will include:

oApplying multiple channels for the mobilization effort – WSIS forum, academic conferences, etc.

oObtaining an agreed action plan with clearly defined responsibilities of partners.

oDefining a framework for progress monitoring and reporting.

OUTPUTS Agreed Action Plan


OUTPUTS

The project is planned to produce the following outputs:

1)O1 - Conceptual Framework for ICT Impact Assessment in general and for WSIS Impact Assessment in particular, also disseminated as guidelines and toolkit

2)O2 - Repository of ICT Impact Assessment models, frameworks and methodologies

3)O3 - WSIS Stocktaking Database Analysis Report

4)O4 - State-of-Practice report on Impact Assessment of WSIS on the MDG process

5)O5 – Action Plan


ACTIONS AND OUTPUTS

Framework for survey design

A7 - Mobilizing

Actions

A6 - Proposal

Articulation

A5 - Issue Investigation

(Survey)

A4 - Analysis of the WSIS Stocktaking Database

A1 - Agenda Settings

Concepts and elements of the framework

A2 - State of the Art Review

Core issues to investigate A3 - ConceptualFramework

O1 - Conceptual Framework Report

O5 - Proposal for action

O4 - Survey findings on state-of-practice

and practitioner needs

O3 - WSIS Stocktaking Database Analysis

Report

O2 - Repository of theories, models frameworks, and tools on ICT4D impact assessment


OVERVIEW


2 INITIAL RESULTS








3 CASE STUDIES

4 DISCUSSION

5 NEXT STEPS


CONCEPTUAL FRAMEWORK – AIM

The Conceptual Framework (A3) action aims to:

oDevelop a conceptual model to relate major concepts in different thematic areas covered in the State-of-the-Art (A2) review.

oRelate various theories, models and frameworks to provide good understanding of the central issues involved in: (i) ICT impact assessment and (ii) design of ICT4D portfolio.

Specifically, we seek to determine the gap in the application of existing ICT4D theories/models with respect to the WSIS and MDG processes.

The anticipated value in our conceptual framework will therefore be in filing this knowledge gap for practitioners.


CONCEPTUAL FRAMEWORK – CONCEPTS

Defining Impact

o Positive or negative, primary or secondary long-term effect produced by a development intervention, directly or indirectly, intended or un-intended [DAC].

o Impact is an actual experience of an individual or community, therefore variables such as increase in population, increase in ethnic and racial diversity, relocation etc.

o Social impact refers to impacts actually experienced by humans (at individual or higher levels) in either a corporeal (physical) or cognitive (perceptual) sense [Schooten 06]

Defining ICT4D Impact Assessment

Contribution of an ICT initiative to concrete development objectives [Heeks et al. 09]


CONCEPTUAL FRAMEWORK – RELATING CONCEPTS

Concept Example

ICT mobile phone, e-learning

Output mobile phone network

Outcome Improved communication of communities with external world, price information on farm products, etc.

Benefit increased daily income for farmers, improved social ties for families, etc.

Impact reduced poverty

Objective halve between 1990 and 2015, the population with income is less than 1 dollar a day (MDG 1, Target 1)

Inputs Activities

Outputs Outcomes

Impact Objectives


CONCEPTUAL FRAMEWORK – MAJOR THEMATIC AREAS

ICT4D Impact Assessment

ICT and Development Conceptualizing ICT


CONCEPTUAL FRAMEWORK – ICT4D IMPACT ASSESSMENT

Heeks ’09, Klein 08, etc.


CONCEPTUAL FRAMEWORK – ICT AND DEVELOPMENT

Development Theories

Modernization Modernization

Human Development

Human Development

DependencyDependency

Others …Others …

Development Related Constructs:

AgencyChoiceCapabilityEmpowermentInstitutionResourcesVulnerability Context….

ICT

Examples - Choice Framework, Sustainable Livelihood Framework, Empowerment Framework, etc.


CONCEPTUAL FRAMEWORK – CONCEPTUALIZING ICT

Traditionally, ICT has been treated as a monolithic and homogenous entity, particularly in practice. However, ICT use is contextual and shaped by societies [Sein et al. 04, Orlikowski et al. 01]:

oHigh incidence of ICT project failures in terms of anticipated impacts on individuals and societiesoHigh prevalence of unintended benefits from ICT initiatives

Consequently, a number of perspectives have been proposed for understanding ICT [Sein et al. 04]:

oICT Use - how it is utilized?oICT Views - how it is conceived?oICT Impact - what kind of impact can it have?


CONCEPTUAL FRAMEWORK – IMPLICATIONS FOR WSIS INITIATIVES

Ensemble Imperative

Explicit consideration of the social context in the design of WSIS initiatives.

Proxy View

Policy shift from treating ICT as a tool to ICT as symbolizing development e.g. economic growth or empowerment for impact generation.

Direct application

Associating initiatives explicitly with concrete development goals from inception is critical.


OVERVIEW


2 INITIAL RESULTS








3 CASE STUDIES

4 DISCUSSION

5 NEXT STEPS


WSIS ANALYSIS – AIM

The aim of this action is:

oTo summarize recorded initiatives with respect to the associated Action Lines and MDGs

oTo generate associations between WSIS activities and the MDGs as a basis for developing hypothesis for further investigating the impact of WSIS activities on the MDG process

A WSIS Stocktaking Analysis Report will be produced as a result.


WSIS ANALYSIS – PROJECT VOLUME BY COUNTRY


WSIS ANALYSIS – PROJECT VOLUME BY COUNTRY AND STATUS


WSIS ANALYSIS – KEYWORDS USED IN TITLES


WSIS ANALYSIS – KEYWORD FREQUENCIES


WSIS ANALYSIS – PROJECT DISTRIBUTION BY ACTION LINES


WSIS ANALYSIS – PROJECT DISTRIBUTION BY MDGS


WSIS ANALYSIS – RELATING WSIS ACTION LINES AND MDGS

MDG1 MDG2 MDG3 MDG4 MDG5 MDG6 MDG7 MDG8C1C2C3C4C5C6C7C8C9

C10C11

Two forms of analysis:

oDeduction of rules from the database (4646 projects)

oManual mapping of 42 reported cases from international development reports

Single confirmation

Double confirmation

Approach rule mining

Support 0.6 (60% of cases supported the rule )

Confidence 0.9


WSIS ANALYSIS – WSIS-TO-MDG1 EXAMPLE

MDG 1: Eradicate extreme

poverty and hunger


OVERVIEW


2 INITIAL RESULTS








3 CASE STUDIES

4 DISCUSSION

5 NEXT STEPS


OVERVIEW


2 INITIAL RESULTS








3 CASE STUDIES

4 DISCUSSION

5 NEXT STEPS


OVERVIEW


2 INITIAL RESULTS








3 CASE STUDIES

4 DISCUSSION

5 NEXT STEPS


NEXT STEPS - STATE OF PRACTICE AND NEEDS SURVEY

The goal of this activity is to gather information on the efforts towards assessing the impact of ICT initiatives related to WSIS Action Lines to the MDGs.

Towards this, an online survey on Impact Assessment of WSIS initiatives on MDG will be hosted as part of the WSIS stocktaking activity, also serving to update information on programs and projects.

This survey will specifically investigate:

o If impact assessment is conceived a part of an initiative?o How are the results of impact assessment planned to be used?o How is the impact assessment carried out – rigor, evidence, participation of users?o What concrete evidence is available to support claims of positive (negative) impacts? o What are specific challenges facing practitioners in assessing their projects?o What kind of support do member states and practitioners need to improve the practice of

impact assessment and design of their project portfolios for higher impact potential?

The survey will be supplemented with selected interviews with project focal points.


NEXT STEPS - VERIFYING LINKAGES BETWEEN WSIS AND MDGS

The associations deduced from the stocktaking database and the manual mapping exercise will be investigated further through:

oAnalysis of evidence gathered through surveys and elaboration of case studies for detailed investigation of facts with respect to specific Action Lines

oFurther theoretical analysis of the possible connections between WSIS Action Lines and MDGS

Thank you for listening

ITU-UNU WSIS-MDG Team

Assessing the Impact of ICT on Development – WSIS and MDGs ITU-UNU WSIS-MDG Team.

Documents

development wsis

wsis forum

impact of ict

impact wsis process

impact of wsis action

wsis stocktaking database

wsis database summary

wsisrelated initiatives