Leveraging Your Data

Leveraging Your DataLeveraging Your DataMatthew MatzMatthew Matz

Morrow County School DistrictMorrow County School District

Data in a minuteData in a minute

http://www.youtube.com/watch?v=ImAD8BOBOhw

WelcomeWelcome First things firstFirst things first

Sit backSit backRelaxRelaxPlease ask questionsPlease ask questionsFeel free to stop and discussFeel free to stop and discussPay close attention...Pay close attention...

Because you don't want to miss anything...Because you don't want to miss anything...

http://www.youtube.com/watch?v=Ahg6qcgoay4

Perception is realityPerception is reality But is reality perceived?But is reality perceived?

NopeNope It can't beIt can't be

The brain has to filter out all kinds of stimuli The brain has to filter out all kinds of stimuli before it can even begin to process the before it can even begin to process the information it receives.information it receives.

Part of that process in unconsciously selectivePart of that process in unconsciously selectiveThat's why statements like "you only hear what you That's why statements like "you only hear what you

want to" exist.want to" exist.

And if you need more proof...And if you need more proof...

http://www.youtube.com/watch?v=FWSxSQsspiQ

IntegrityIntegrity Because it's very easy to use slight-of-hand Because it's very easy to use slight-of-hand

with data, we have an obligation to be with data, we have an obligation to be honest.honest.

Even with the best of intentions, it can be Even with the best of intentions, it can be easy to skew reality with the wrong kind of easy to skew reality with the wrong kind of chart or graphchart or graph

The Shrinking DollarThe Shrinking Dollar This chart This chart

shrinks both shrinks both dimensions of dimensions of the bills to the bills to represent the represent the change in a change in a single value, single value, therefore therefore distorting the distorting the effect of effect of inflation.inflation.

Worst Pie Chart EverWorst Pie Chart Ever

Another exampleAnother example Is the world Is the world

more or less more or less dangerous dangerous than it was in than it was in the past?the past?

Graph what Graph what you thinkyou think

That's why he's so important!That's why he's so important! Data helps keep our Data helps keep our

perception of reality perception of reality groundedgrounded

Adding shock valueAdding shock value Think for a moment about the United States Think for a moment about the United States

SenateSenate It's a representative body, right?It's a representative body, right? Is it ethnically/racially representative?Is it ethnically/racially representative?

How will you get anyone to care?How will you get anyone to care?Shock them, of course.Shock them, of course.

If you haven't already figured this out, the next slide may If you haven't already figured this out, the next slide may be offensive, so please take it lightlybe offensive, so please take it lightly

"Senate Color by Numbers" from "Senate Color by Numbers" from America (The Book) America (The Book) by John Stewartby John Stewart

A dose of realityA dose of reality A note...the last slide is not entirely true. In A note...the last slide is not entirely true. In

2004, there were 96 White, 1 Black, and 3 2004, there were 96 White, 1 Black, and 3 Native American/Pacific Islanders in the Native American/Pacific Islanders in the senatesenate

Even so, the previous example is a shocking Even so, the previous example is a shocking way to use dataway to use data

Commercial Break – Pop Quiz!Commercial Break – Pop Quiz! How many e-mails were in route?How many e-mails were in route?

Over 1 millionOver 1 million What was the percentage of e-mails containing "miracle What was the percentage of e-mails containing "miracle

banana diet"?banana diet"? 7%7%

How many of those e-mails hit spam filters?How many of those e-mails hit spam filters? 3600036000

What percentage of people have no idea what Twitter is?What percentage of people have no idea what Twitter is? 26%26%

How may people left their phones in a cab?How may people left their phones in a cab? 2929

How many e-mails were being sent during a business How many e-mails were being sent during a business meeting?meeting? Over 2 millionOver 2 million

Pop QuizPop Quiz How did you do?How did you do? People don't generally don't remember People don't generally don't remember

numbersnumbersSo what was the point of that commercial?So what was the point of that commercial?

Sprint's network is Sprint's network is bigbigSprint's network is Sprint's network is fastfastLot's of people already use SprintLot's of people already use Sprint

Cool stuff, but how do I use it?Cool stuff, but how do I use it? You have a bunch of dataYou have a bunch of data

Cool stuff, but how do I use it?Cool stuff, but how do I use it? So you summarize itSo you summarize it

Cool stuff, but how do I use it?Cool stuff, but how do I use it? And you find what you want to highlightAnd you find what you want to highlight

If you think that's bad, the year before If you think that's bad, the year before was was 77%77%

The secret ingredientThe secret ingredient ContrastContrast

People can't remember numbersPeople can't remember numbersSometimes they can't even remember other Sometimes they can't even remember other

peoplepeople They can remember They can remember sharp differencessharp differences

Before and After photos (or even charts)Before and After photos (or even charts)OutliersOutliers

Comparisons to othersComparisons to others Inequities (Unfairness)Inequities (Unfairness)

Let's try one – Before and AfterLet's try one – Before and After The school calendarThe school calendar

It's easy to forget to look at other data besides It's easy to forget to look at other data besides achievement. Doing so may clue others into the effects achievement. Doing so may clue others into the effects a system has on achievement.a system has on achievement. 180 days180 days

Minus:Minus: ConferencesConferences Half-day in-serviceHalf-day in-service AssembliesAssemblies Emergency DrillsEmergency Drills TestingTesting Field TripsField Trips Spirit WeeksSpirit Weeks School days before a School days before a

3-day weekend3-day weekend

Equals 131 uninterrupted school daysEquals 131 uninterrupted school days

ComparisonComparison Comparing the master schedule against diploma Comparing the master schedule against diploma

requirementsrequirements This could be used to help determine staffing prioritiesThis could be used to help determine staffing priorities

In case you were wondering about the math: Each bar =In case you were wondering about the math: Each bar = (HS FTE) x (sections per FTE) x (% of diploma requirement)(HS FTE) x (sections per FTE) x (% of diploma requirement)

i.e. English = 4/24 = 17%i.e. English = 4/24 = 17%

ScalingScaling The last slide might seem a little dull, but it The last slide might seem a little dull, but it

introduces a trick called scalingintroduces a trick called scaling Scaling is important because it lets you put Scaling is important because it lets you put

unintelligible data in understandable (and unintelligible data in understandable (and more human) terms.more human) terms.

It works by using percentages to create It works by using percentages to create metaphors for the data you are trying to metaphors for the data you are trying to convey.convey.

Here's an example from Stephen Covey's Here's an example from Stephen Covey's The 8The 8thth Habit Habit

From From The 8The 8thth Habit Habit• Harris Interactive recently polled 23,000 U.S.

residents employed full-time within key industries and in key functional areas. Consider a few of their most stunning findings:

• Only 37% said they have a clear understanding of what their organization is trying to achieve and why.

• Only one in five was enthusiastic about their team's and organization's goals.

• Only one in five said they have a clear "line of sight" between their tasks and their team's and organization's goals.

• Only 15% felt that their organization fully enables them to execute key goals.

• Only 20% fully trusted the organization they work for.

From From The 8The 8thth Habit Habit

If this were a soccer team...• Only 4 of the 11 players would know which

goal is theirs. • Only 2 of the 11 would care. • Only 2 of the 11 would know what they are

supposed to do. • And 9 of the players would be competing

against their own team members rather than the opponent.

You could do this with PBS SET dataYou could do this with PBS SET data

You could do this with PBS SET dataYou could do this with PBS SET data

If that school were a Football teamIf that school were a Football teamAll 11 players would know that there were rulesAll 11 players would know that there were rules

But only 3 players would actually know themBut only 3 players would actually know themNo player feels the need to make a touchdownNo player feels the need to make a touchdown

And 6 players know not to get penaltiesAnd 6 players know not to get penalties10 players know decisions are made by coaches10 players know decisions are made by coaches

But 5 players don't trust the coachBut 5 players don't trust the coachAnd nobody feels any support from the ownersAnd nobody feels any support from the owners

The Class-O-GramThe Class-O-Gram

Remedial Math 1 Algebra 1

Remedial Math 2 GeometryAlgebra 1

Algebra 1 Algebra 2GeometryDropout/Alt

Geometry Pre-CalcAlgebra 2

Scale with FlairScale with Flair You are all 8th graders registering for 9You are all 8th graders registering for 9thth grade classes. grade classes.

The registration form states that your math teacher will select your The registration form states that your math teacher will select your freshman math class.freshman math class.

Each teacher will start with a class (a card) and will either pass or Each teacher will start with a class (a card) and will either pass or fail that classfail that class

Passing or failing determines what class is next, and after four Passing or failing determines what class is next, and after four classes (years/cards), the teacher will (or won't) "graduate"classes (years/cards), the teacher will (or won't) "graduate"

In the end, a semi-arbitrary choice by staff may determine whether In the end, a semi-arbitrary choice by staff may determine whether the student graduates, whether they go to college (and if they do, the student graduates, whether they go to college (and if they do, how many thousands of $$ will be spent taking remedial math how many thousands of $$ will be spent taking remedial math courses)courses)

To set this up, you need a lot of data scaled to the To set this up, you need a lot of data scaled to the audience/staff you are presenting to.audience/staff you are presenting to. The "Math Map" for your schoolThe "Math Map" for your school OAKS scores (disaggregated if you are so inclined)OAKS scores (disaggregated if you are so inclined) Historical Pass/Fail rates for each math section offeredHistorical Pass/Fail rates for each math section offered

A "Math Map"A "Math Map" From Spaulding HS in Rochester, NHFrom Spaulding HS in Rochester, NH

How to scale this activityHow to scale this activity If you have a staff of 40 teachers, and a If you have a staff of 40 teachers, and a

graduating class of 120 students, all of your graduating class of 120 students, all of your numbers are divided by 3.numbers are divided by 3.For instance, if 21 students are assigned to For instance, if 21 students are assigned to

Advanced Math, 60 students are assigned Advanced Math, 60 students are assigned Algebra 1, and 39 will be assigned Remedial Algebra 1, and 39 will be assigned Remedial Math, than it's 7, 20, and 13, respectively.Math, than it's 7, 20, and 13, respectively.

If 15 of the 60 kids will likely fail Algebra 1, than If 15 of the 60 kids will likely fail Algebra 1, than 5 of your teachers will "fail" it.5 of your teachers will "fail" it.

How to really shock them...How to really shock them... Disaggregate Disaggregate this activitythis activity

Scale the activity using pass/fail and OAKS Scale the activity using pass/fail and OAKS scores for subgroups such as Latino, ELL, SPED, scores for subgroups such as Latino, ELL, SPED, etc.etc.

Or just disaggregate for your largest subgroup.Or just disaggregate for your largest subgroup.Label your teachers as "Economically Label your teachers as "Economically

Disadvantaged" or not...Disadvantaged" or not...And in the end, let them reflect on what their And in the end, let them reflect on what their

status meant in their school.status meant in their school.

The The Disaggregated Disaggregated Class-O-GramClass-O-Gram

Remedial Math 1 Algebra 1

Remedial Math 2 GeometryAlgebra 1

Geometry Pre-CalcAlgebra 2

Algebra 1 Algebra 2GeometryDropout/Alt

Tips and TricksTips and Tricks Primacy-RecencyPrimacy-Recency

Data to StoryData to Story works if the works if the data doesn't have to be data doesn't have to be "processed" by the "processed" by the audience, i.e. just giving a audience, i.e. just giving a few large numbers outfew large numbers out

Story to DataStory to Data is better if the is better if the data does have to be data does have to be "processed" i.e. different "processed" i.e. different types of data are presented types of data are presented and comparedand compared

Don't dwell on the in-Don't dwell on the in-between. It's less likely to between. It's less likely to be remembered anyway.be remembered anyway.

DataDataDataDataData

Personal Story

DataDataDataDataData

Personal Story

???

Tell StoriesTell Stories Tell a bunch of stories and leave the charts Tell a bunch of stories and leave the charts

at home (If you can)at home (If you can) The human brain is wired for stories, and The human brain is wired for stories, and

retention of stories can reach 100% if it's retention of stories can reach 100% if it's delivered well.delivered well.

Hands-on activities have retention rates Hands-on activities have retention rates around 50%around 50%

Numbers and charts have retention rates Numbers and charts have retention rates around 20%around 20%

If you must use a chartIf you must use a chart Remember: Remember: ContrastContrast Also remember: Also remember: ClarityClarity Oh yeah: Oh yeah: ConsistencyConsistency

What is the What is the Main point Main point you want to highlight?you want to highlight? Make sure it's a different shade/colorMake sure it's a different shade/color

Is your graph clear, simple, and easy to read?Is your graph clear, simple, and easy to read? Don't try to show 5 things on a graph. Show 5 graphs that only Don't try to show 5 things on a graph. Show 5 graphs that only

show 1 or 2 things each.show 1 or 2 things each. Every graph should look the sameEvery graph should look the same

Same color scheme, same scale, same font. Same color scheme, same scale, same font. You want the You want the datadata to stick out, not the to stick out, not the y-axisy-axis..

Charts that don't workCharts that don't workOAKS Math Average

213

210

214

215

230

225

235

231

195

200

205

210

215

220

225

230

235

240

2008-09 2009-10

Year

Avg

. S

core 3rd

5th

8th

10th

Charts that workCharts that work

OAKS Math Growth

5

34

-5

3rd 5th 8th 10th

08-09 to 09-10

Because I promised itBecause I promised it The weekend Data WallThe weekend Data Wall

When you are looking at a large amount of data, When you are looking at a large amount of data, it's helpful for people to have intuitive visual it's helpful for people to have intuitive visual cues that reflect the data so that it can be cues that reflect the data so that it can be understood quickly and easily.understood quickly and easily.

Stoplights are a good place to start:Stoplights are a good place to start:Red, Yellow, GreenRed, Yellow, GreenPoor, Warning, GoodPoor, Warning, Good

Add brown=no data and blue=excellentAdd brown=no data and blue=excellent

Apply this to each student, and before you know Apply this to each student, and before you know it, you'll have a data wall.it, you'll have a data wall.

Fast Data WallFast Data Wall

Want to know more?Want to know more? PerceptionPerception

BlinkBlink by Malcolm Gladwell by Malcolm Gladwell Back to School...Back to School... and and The Cognitive Art of Educational The Cognitive Art of Educational

Technology Technology by Patrick Crispinby Patrick Crispin http://http://www.netsquirrel.com/powerpoint/pedagogy.htmlwww.netsquirrel.com/powerpoint/pedagogy.html

CrimeCrime The First Measured CenturyThe First Measured Century by PBS by PBS

http://www.pbs.org/fmchttp://www.pbs.org/fmc

Political HumorPolitical Humor America (The Book)America (The Book) by John Stewartby John Stewart

ExcellenceExcellence The 8The 8thth Habit Habit by Stephen Covey by Stephen Covey

Want to know more?Want to know more? Designing Good GraphsDesigning Good Graphs

The Visual Display of Quantitative InformationThe Visual Display of Quantitative Information by by Edward TufteEdward Tuftehttp://http://www.edwardtufte.com/tufte/books_vdqiwww.edwardtufte.com/tufte/books_vdqi Tips for Tips for

PresentationsPresentationsResources by Chip and Dan HeathResources by Chip and Dan Heath

http://http://heathbrothers.comheathbrothers.com/resources//resources/

Looking at Systems DataLooking at Systems DataUpdraft DowndraftUpdraft Downdraft by Marilyn Crawford by Marilyn Crawford

Thank you!Thank you!