Measurement Birth to Age 8 Abbie Raikes UNICEF and MELQO.

Measurement Birth to Age 8Abbie Raikes

UNICEF and MELQO

Main Points and Themes

• SDGs: Setting the context

• Creating Equity through Measurement: Measuring Child Development/Learning at the Population Level and Measuring Quality

• Realizing Contribution of Measurement: What’s next?

I. Setting the StageMeasurement and the SDGs

ECD Makes its Global Debut in the Sustainable Development Agenda

Explicitly mentioned in 4:2:

Goal 4, Target 4.2 by 2030 ensure that all girls and boys have access to quality early childhood development, care and pre-primary education so that they are ready for primary education

SDG goals are inter-linked, cover several goal areas and are meant to work together, e.g., Goal 1 (end poverty) Goal 2 (end hunger and improve nutrition), Goal 3 (healthy lives for all), Goal 5 (gender equity), Goal 16 (end violence against children).

Important Era for Early Childhood Development• Increasing awareness of the importance of investments in early

childhood development

• Many new program models and growing evidence of the importance of ECD through randomized trials and other research studies

• Measurement as one important element to track trends, investments at the population level

What’s different about SDG measurement?• National and regional monitoring, not just global• Emphasis on outcomes, not only access – quality becomes central• Expansive terms that include many undefined concepts• Many new measurement innovations• Resources invested in measurement are limited • Data must serve multiple users – globally comparable is one factor

among many, and must be able to use the data for improvement

Data Revolution: What does it mean, and how will ECD be a part?

Measurement Framework for Children Birth to 8 Years

Raikes, Britto & Dua, 2014. A Measurement Framework for Early Childhood: Birth to 8 Years of Age, Institutes of Medicine

Vanuatu

Measuring Children’s Development At or Near the Start of School

PRIDI EDI EAPS-ECDS STC MELQO

Measurement’s contribution to the SDGs• Create common language …

• For an understanding of “school readiness” that is developmentally-appropriate and begins at birth

• For a definition of quality that protects children’s rights and supports their development

• High leverage of data: Further establishing ECD

Indicators Update

• Agreement soon on one set of indicators for global monitoring• Two proposed indicators, Goal 4 – MICS ECDI and access to “organized learning”• Strong support from member states needed to ensure presence

• Sets of national, thematic, regional indicators

• Intended to apply to both high and low-income countries • A focus on equity: Reducing gaps between groups based on wealth, gender, and other

factors associated with inequity

Balance between high political leverage of comparable data and importance of normed, contextually-sensitive data

II. Global To LocalImplementing the Vision

Child development/learning measurement reflecting culturally-relevant normative development for all children Context measured too – families, schools, exposure to violenceActionable data that drives specific changes for children

Project-level measurement using variety of tools with little scalingUnclear use of data to systematically improve

Ideal

More Likely

Measurement in Service of Equity Principles for consideration …

- Ensuring all children are counted and skills acknowledged: Creating measures for use across all settings, and for all children birth to age 8 … and normed for local population?

- Environments as well as outcomes: Protecting children’s rights by recognizing family, community and school environments as critical

- Data show clear pathway to action and improvement

Steps towards Equity-Based Measurement

Context, Inputs/Outputs,

and Child Development Measured for

ALL GROUPS at Country Level

Clarity on Data Use

Begin with Principles, Toolkit of Methodologies and Existing Tools

Local Researchers and Stakeholders

Adapt or Build New

Data System Provides Ongoing

Feedback to Teachers,

Caregivers and Policy Makers

Road Map for Improvement:

Data-Informed

Policy Making and Practice

Better Environments for Children

As we move to larger-scale measurement …

Using data for improvement

Measure holistical

ly

Screening +

Population-Based

Various types of tools … and lots of ways to collect data

Type of Tool How it’s used and why How data are collected

Population-based To track groups of children and identify gaps between groups

Direct assessment, teacher report or parent report

New methodologies needed to include all children

Diagnostic or formative To identify specific skills for specific children

Likely through direct assessment or parent report

Screening To identify children who might be at risk for developmental delays for further assessment

Likely through direct assessment or parent report

Contextual To know quality/characteristics of home, school, communities

Likely through household surveys; also through GPS and other methods

And various tensions

• Integrating data from formative/diagnostic and population-based• If population-based, how to use to identify children are most in need and better

support children and families• If diagnostic or formative, how to create a comprehensive picture of all children

• Pros and cons of using common measures across diverse settings• Interpreting child development data in absence of cultural norms• How important is it to have globally or regionally comparable data?

• Quality measurement may require a different approach than child development and learning

Making Measurement Easier

1. One measure used everywhere: Same items, same administration everywhere, with a small amount of adaptation

2. Common core of items: One small set of items, may be part of larger and more culturally-adapted set

3. Common constructs, with items that may vary: May be able to “match” at level of construct, but with different items

4. Item bank: Lots of items, with little or no commonality from one place to the next

Measurement of Child Development and Learning At Scale: Two Projects

• Measuring Early Learning Quality & Outcomes (MELQO): Children’s development at start of school and quality of pre-primary learning environments (UNICEF, UNESCO, World Bank, Brookings Institution)

• WHO: Children’s development between the ages of birth and three years

World Health Organization: Global indicators for child development 0-3 years

• Are there potentially existing and validated cross-cultural and feasible indicators that can be adapted for wider use?

• What’s the magnitude of developmental fluctuation between children and between countries? Can we collectively define normal, concern, and delay (e.g. 18 month cut off for walking for children everywhere)?

• Which developmental milestones observable before age of 3 are reliable predictors of outcomes?

• What is a feasible process for implementing developmental indicators in earliest years of development?

Goals of WHO Child Development Indicators Project

1. To systematically assess data from developing countries on children less than 3 years of age for core items of global, developmental competencies.

2. To evaluate the feasibility and applicability of all valid items for surveys in a number of settings worldwide.

3. To field test and recommend a set of indicators to assess child development at population-level across the 0-3 age group.

Good item performance

ASQ - WordingDoes your baby walk along furniture (OR OTHER OBJECT) while holding on with only one hand?

MDAT - WordingWalks with help (using somebody’s hand as if led or a piece of furniture).

KDI - WordingWalks when held with one hand

MELQO Phase 1: Consensus to Country Action1) Convene expert groups on quality and child development/learning

2) Use existing measures to pre-test two new tools (CDL and quality) that are conceptually linked

3) Conduct institutional assessment on scaling the assessment

4) Validation – first nationally-representative study underway

Framework for Child Development/Learning

III. Quality MeasurementPromoting equity by focusing on environments

MELQO Quality Measurement

• Is it possible to use political leverage to measure quality? Big miss of MDGs … how can we do better this time around?

• Is it possible to define items or constructs that are comparable across countries or at least serve as a starting point?

• How can quality measurement be connected to child development and learning? How can it be built into monitoring & evaluation systems with emphasis on improvement?

Review of Quality Monitoring• Information on monitoring quality of services is limited. Most of the information

on systems to monitor children’s learning environments come from higher income countries, with more developed ECD systems.

• There is agreement on many key constructs of quality, which can be summarized to help develop feasible approaches to monitoring at scale.

• While many countries have established standards for ECCE quality, it is unclear if these standards are suitable for supporting children’s development outcomes and implementation of these standards is uneven.

• Few tools are available to help countries monitor quality at scale, especially in

developing countries, and only for some kinds of services

Child Development & Learning

InteractionsMaterialsInclusiveness

Professio

nal Development

Teacher

Motivation & CharacteristicsResources

School Leadership

Policy structure,

including M&E Framework

Connections

to national

standards, regulations and other

sectors

Classroom or setting level

Child-level

System level

Policy level

Child counts objects

Child Names Numbers

Counts up to 20

Teacher has training on

math instruction for young children and receives

regular feedback

Teacher works with small groups

of children to teach each child

at individual level

Child has access to counting

materials like blocks and other

small objects

Curriculum with age-appropriate expectations for math

Adequate funding for

preschools to buy materials, ensure reasonable class

sizes

Talk about numbers – open-ended questions and dialogue

If the child is to achieve these learning outcomes …

Classrooms should have these characteristics …

And programs must ensure this happens...

Through policies and funding at the national level

Existence of a strong, effective monitoring and evaluation system with reliable data and incentives to improve

Experience to Date …

• Constructs similar across settings?

• Possibly, but items may vary substantially from one place to the next• Varying degree of alignment between “global” concept with government standards

• Can we define quality ranges? Item banks? What is the best way to leverage global momentum for country action?

• Targeting the user of the data – defining what they most need to know – is a key next step

Implications for Equity in Measurement• Avoid high-stakes … • Need not measure ALL children or settings everywhere – sampling may protect

rights

• Definitions of healthy development, quality and inequity vary by place – must have cultural lens and local norms

• Using data to build effective professional development/M&E systems is very important• Don’t know as much we’d like on how to support this

IV. Next StepsGetting to Data for Improvement, and for All Children

A Quick Look at Country Capacity

• Willingness to engage

• But limited resources to support on-going measurement at scale• Few systems in place to support measurement over time

• Little structure to support partnerships with researchers/experts and government

A Quick Look at Regional and Global Capacity• Investments in measurement happening in many organizations, often

through tools

• Not clear how coordinated it is, or how best to maximize investments

• More investment now needed in building measurement systems, especially using measurement for improvement

• Work needed to define new methodologies to capture all children; cultural norming – efficiencies of scale

Goals for Using Collective Measurement Expertise

1) Invest in integrated measurement systems, birth to age 8 and for all children – innovations needed

2) Invest in process for cultural adaptation and norming

3) Invest in how data can be best used, and measurement of implementation of quality programs

4) Invest in local research and capacity for measurement

Discussion Questions

• What are the successes of measurement efforts to date? How can we build on these successes as we move to measurement at scale?

• How should we take advantage of global attention on ECD with need for cultural adaptation and relevance?

• How can we work together to support integrated action at the country level?

• Of the needed capacities outlined, from local research partnerships to new methodologies to capture all children, how can we use our collective strength to move forward?

Thank [email protected]