A Step-By-Step Guide to Developing a National Report Card on the Physical Activity of Children and Youth

Report Card – 101

A step-by-step guide to developing a national Report Card on the physical activity of

children and youth

Before reviewing this slide deck it is recommended that you read the background paper: Colley RC, Brownrigg M, Tremblay MS. The Active Healthy Kids Canada Report Card on Physical Activity for Children and

Youth. Health Promotion Practice 13(3):320-330, 2012.

The Report Card is a large undertaking. To achieve the desired quality and impact, it is important that the Report Card is not left until the last minute. Before you begin, ask yourself, Can you commit the resources (human and financial) needed to see a Report Card through to completion?

So you want to lead a Report Card in your country?

What is the Report Card?

• The Report Card is a synthesis of the most recently available published or unpublished data related to the physical activity of children and youth in a nation (new data collection is not required or expected).

• The Report Card is an evidence-based communications and advocacy tool.

What isn’t the Report Card?

• The Report Card is not a research study.• It is commonly misreported in the media that the

Report Card is a new national survey. While new analyses of existing survey data may be commissioned, the Report Card is a synthesis of the best available existing research.

Purpose of the Report Card

• To advance knowledge about how a country is being responsible in providing physical activity opportunities for children and youth.

• To influence issue stakeholders who affect physical activity opportunities for children and youth.

AHKC Report Card Framework

• The AHKC Report Card Model assesses the behaviours that contribute to overall physical activity and sedentary behaviour.

• The Model also assesses the contextual factors (e.g., strategies, investments, settings) that influence these behaviours.

• In this version 9 indicators are grouped into 3 categories to reflect their relationships with a child’s physical activity level.

• Physical activity level affects outcomes (e.g., happiness and fitness) that can affect the child’s overall physical activity levels.

AHKC Report Card Framework

Through the model we:• Amass the best possible evidence;• Identify gaps in the existing knowledge;• Provide recommendations for what we

can do to improve.

Content Development Process

How to do a Report Card(allow 8-12 months for development and production)

Month

Annual Cycle of Activities 1 2 3 4 5 6 7 8 9 10 11 12+

Identify Research Work Group, Project Manager , Principal Investigator

Indicator Meeting, form writing team

Comprehensive literature reviews in key topic areas

Unique data analyses submitted to PI by RWG

Summary of evidence document produced

Grade Assignment Meeting

Additional research to address gaps identified in grading

Identify design team, copyeditor, and translator

Production of draft 1, review, production of draft 2, review

Coordinate design and layout of RC

Coordinate copy editing, translation, and printing of RC

RC launch, release, and dissemination

Internal and/or external evaluation strategy development and execution

Communication and media strategy, issue advocacy

Annual Work Cycle to Support the Development and Production of a Physical Activity Report Card

ASSEMBLE YOUR TEAMStep 1

The Report Card Team

• Invite leading national researchers and policy experts to join the Research Work Group (RWG)• Identify a Principal Investigator (PI)• Hire a Project Manager/Lead Writer• Identify the design team, copy-editor,

translator (if required), web-designer

How to Choose a RWG?

• Recognized experts in a relevant field• Can provide access to survey data• Can commit to participating in the Indicator

Meeting and the Grade Assignment Meeting in person or by connecting remotely.

• Also consider, are you the PI or will the RWG choose a PI as a group?

The Development Team

• Will the PI be responsible for project management and writing, or will someone be hired for these roles?– Project Management and writing will require a

large time commitment. You may wish to involve a graduate student or hire a project manager.

– Inviting students to write sections allows them to gain experience in knowledge translation.

Production Team

• Efforts to identify the production team should begin as soon as the RWG and Development Team are in place – Allows time to brainstorm themes and cover story

ideas.– Prevents last minute struggle to find the right

people.

Communications & Media Specialist (CMS)

• A key purpose of producing the Report Card is to inform policy makers, practitioners, educations, and the media.

• Early identification of a CMS is important to ensure:– Relevant groups are engaged & informed– Key findings are effectively disseminated across

the country

GETTING STARTEDStep 2

Literature Search

• The search for content should begin immediately upon identifying your Lead Writer.– The amount of research available will vary widely

across countries. – RWG members should forward materials

proactively.

Literature Synthesis

• Don’t wait until the last minute– Countries with a strong research record should

allow greater time and resources for literature search and writing.

• Consider your audience – The Report Card should be written in an engaging

and jargon-free manner appropriate for media, government, and practitioner audiences.

THE INDICATOR MEETINGStep 3

Objectives

• Finalize the list of indicators to be graded• The 9 pre-specified indicators were required to

participate in the Global Matrix 1.0, although others of importance in your country can be included. See past Report Cards for examples. [Archive link]

• Agree on the data sets that will be used to inform the grades.

• Agree on the benchmarks/optimal scenarios indicators will be graded against.

Preparation: Summary of Data Sources

• The Project Manager prepares and provides a ‘Summary of Data Sources’ document to the PI and RWG in advance of the meeting• The summary should include all recently available

(this may mean within the past 5 years, depending on data availability) national and regional surveys related to each indicator.

Summary of Data Sources Document (example)

Overall Physical Activity Levels

CHMS (2010 – 2011) PAM (2010 – 2011)

Nationally representative sample of children and youth

Most provinces and territories included, 8 – 12 year olds

Accelerometry How many days last week did your child do physical activities for a total of at least 60 minutes per day?

During the past 7 days, on how many days were you physically active for a total of at least 60 minutes per day? Add up all the time you spent in any kind of PA that increased your heart rate and made you breathe hard some of the time. (0-7 days)

During the past 7 days, did you do any physical activities? (not including PE or gym class)

Indicator

Measures

Survey name & year(s) collected

Important details about the survey

Preparation: Proposed Benchmarks

• The PI and Project Manager propose benchmarks for each indicator to be circulated to the RWG before the meeting. This would be harmonized among countries in the case of the Global Matrix.

• Benchmarks should be based on evidence-informed guidelines where possible.– E.g., “the percentage of children and youth who

meet the Canadian Physical Activity Guidelines”

Indicator Meeting• Indicators– RWG decides if other indicators will be

included• Summary of Data Sources– RWG discusses and assesses completeness– RWG identifies and provides details for

additional sources if any were missed– Data gaps are noted

• Benchmarks– RWG discusses and finalizes benchmarks

Decisions You May Face

• How far back in time will you cover?• How often will you release a Report Card?• Will you include other indicators?– If you grade obesity it is likely that media will

only focus on obesity, at the expense of physical activity

• How to assign benchmarks to indicators without evidence-based guidelines? – E.g., active play

GRADE ASSIGNMENT MEETINGStage 4

Objectives

• Discuss and agree on grades for each indicator– Note and record rationale for grade

• Decide if new analyses are needed – New analyses may be commissioned if there are

existing data available

Grade Assignment Methodology

Grades are assigned based on examination of the current data and literature for each indicator against a benchmark or optimal scenario:

A (81-100%) = We are succeeding with a large majority of children B (61 - 80%) = We are succeeding with well over half of children C (41 - 60%) = We are succeeding with about half of children D (21 - 40%) = We are succeeding with less than half, but some, children F (00 - 20%) = We are succeeding with very few children

Other Considerations: Time Trends

• Grades may be graded up (+) or down (-) based on trends over time – E.g., if less than half of kids are achieving the benchmark,

but longitudinal data show there have been improvements over time, you may wish to recognize the improvement by changing a D to a D+

Other Considerations: Disparities

• Grades may be graded down (-) based on the presence of disparities such as age, gender, disability, ethnicity or SES, or up (+) if there is evidence past disparities are disappearing– E.g., if well over half of kids are achieving the benchmark,

but sex-based analyses show that girls are doing worse than boys, you may wish to change a B to a B-

Follow-Up

• Additional research may be gathered to address gaps identified during Grade Assignment Meeting.

• Final grades are confirmed by email or by teleconference if further discussion is needed.

PRODUCTIONStage 5

Production Tasks to be Completed

1. Produce long form draft2. Review long form draft3. Produce long form draft

24. Review long form draft

25. Produce short form

6. Coordinate design and layout of Report Card

7. Coordinate copyediting8. Translate long and

short form final versions (if required)

9. Coordinate printing10. Coordinate website

production, support tools, social media

Cover Story

• The media are instrumental in increasing the reach and impact of the Report Card.

• A cover story enables you to gain media attention and to shape the message.

• Rely on leadership from the CMS in selecting an attention-grabbing cover story.

• See Archive for past examples. [Archive link]

Communications and Media Strategy

• Plan the release strategy (event, media release)• Establish network of provincial/ territorial/ state

partners to support the release. –Distribute Report Card through their networks–Provide media spokespeople to respond to

ongoing media requests• Develop post-Report Card release

communications.

Issue Advocacy

• Target government and NGO Officials whose work in policy and programming impacts physical activity opportunities for children and youth.

• Conduct strategic industry meetings.• Present at local, national, and

international conferences and meetings.

Evaluation

• Options for evaluation include:– Report Card dissemination and media uptake. – Stakeholder feedback (e.g., Is the Report Card

meeting stakeholder needs?).– Process evaluation (i.e., Inputs, activities, outputs).– Evaluation of immediate, intermediate, and long-

term outcomes (e.g., Is the Report Card being used to increase awareness about physical activity in children and youth?).

Register Your Country

• To initiate the Report Card process in your country and/or to participate in the Global Matrix 2.0 visit http://www.activehealthykids.org/ and register your country.