Ces 2013 the role of technology and social media - raising the grade

Technology and social media : fostering youth participation in evaluationDominique Leonard, Heather Smith Fowler, Neil Price

CES Conference 2013, Toronto

Outline

1. Raising the Grade: program description

2. Key characteristics/features of good technology, youth engagement, evaluation

3. What do we know about the intersection of youth participation + technology + evaluation?

5. Retrospective on Year 1

6. Moving up the ladder of engagement: Year 2 and beyond

Raising the Grade: Description

35 Boys and Girls Clubs across Canada Funded by Rogers Youth Fund Started in Oct 2012 To date, just under 300 youth enrolled National program BUT focus on local flexibility to adapt Academics + tinkering with tech! http://www.raisingthegrade.ca/

Raising the Grade: major program components

1. Academic support 1:1 mentorship Personal development plans (MyBlueprint), e-portfolios Engaging projects and activites (driven by youth interests) Access to curated, high-quality academic resources

2. Rogers Tech Centres

3. Scholarship program

4. Evaluation and continuous improvement

Evaluation and Continuous Improvement

Coming together during the RFI Who is SRDC?

• Social Research and Demonstration Corporation (SRDC)• Canadian non-profit research firm• Why was SRDC selected to lead the evaluation?

– 20 years’ experience running demonstration projects– 10+ years testing new educational programs– Presence across Canada

Developmental, capacity building User-driven, utilization focused

Evaluation and Continuous Improvement

Year 1 – laying the groundwork• Each tool, measure mapped to relevant input, activity, output, or

outcome UNIQUE, SPECIFIC– Registration form– Youth baseline and annual surveys– Quarterly reports– Program snapshots– Weekly check-ins– Interviews, focus groups– Web analytics (Raising the Grade website, myBlueprint)

Features of good technology, youth engagement, evaluation

Technology Youth engagement Evaluation

Fast Fun What are the features of ‘good’ evaluation (i.e., rigorous, relevant, feasible/practical, transferable, etc.)?

When was the last time you had an evaluation that encompassed many of the characteristics of ‘good’ technology and youth engagement?

DYNAMIC TENSION!

Reliable Empowering

Ubiquitous Relevant, meaningful

Constantly updated/refreshed

Authentic (e.g., beyond tokenism)

Accessible, clear Timely (at the moment)

Engaging Developmentally appropriate

Comprehensive, multiple features, sequenced and layered

Flexible structure

What do we know about the intersection of youth participation + technology + evaluation?

Principles of meaningful youth participation have been around for decades – a lot of research on what works and doesn’t to foster youth participation in programs

What do we know about the intersection of youth participation + technology + evaluation?

Youth and participatory evaluation strategies (Sabo Flores, 2008): Training young people as interviewers Training young people as creative focus group facilitators Working with young people to develop surveys Using performance as a data collection strategy Using journaling in evaluation Approaching data analysis and report writing creatively

What do we know about the intersection of youth participation + technology + evaluation?

There is an emerging body of knowledge on what works (and what doesn’t) for technology use in evaluation (Web 1.0 – 3.0)

The challenge as evaluators: staying on top of emerging tools and technology

What do we know about the intersection of youth participation + technology + evaluation?

• Web 1.0: the “read-only” web• Web 2.0: the “read-write” web -- users able to interact with one another or

contribute content• Web 3.0: Existing data reconnected for other (smarter) uses

What do we know about the intersection of youth participation + technology + evaluation?

Mobile technology platforms (from Kerry Bruce, aea365.com)

Photovoice, Videos, Photostories Interactive mapping tools (e.g., DataMind, GIS) Webinars, YouTube

What do we know about the intersection of youth participation + technology + evaluation?

You only get 2 of the 3 covered in any one ‘field’/tools/guidance to inform how to make it work well

Approach to evaluation in other Club Tech programs• we’re gonna push it one step further

Retrospective on Year 1

What have we learned? What have we struggled with Web analytics Blogs Surveys Polls Social media Challenges

• Building tech capacity/know how• complexity of program (careers, PSE, homework , Rogers, etc.)• individualizing an experience for an incoming youth while

maintaining a structural integrity across a National program

Moving up the ladder of engagement: Year 2 and beyond

The ‘perfect storm’ for innovation Emerging ideas

• youth reflecting on their own baseline ‘portrait’• design and development of a new interactive activity

THANK YOU!