Big data big_skills_data_visualization

Midnight January 28, 1986 Lives are on the line

Thanks to Edward Tufte

Night before the Flight

Jan 27,1986

Importance of Data Visualization

Estimated launch temperature 29º

13 Pages Faxed

13 Pages Faxed

3 different types of names

Damage (in overwhelming detail) but No Temperatures

13 Pages Faxed

13 Pages Faxed

Test engines, not launches, fired horizontally

Missing temperatures for 5 erosion damage flights

Temperatures but limited Damage

13 Pages Faxed

“blow by”, not more important “erosion”, (at hottest and coldest launches)

13 Pages Faxed

Predict Temperature

Recommendation

55 65 7560 70 80

1

Original Engineering data

2

3

damages atdamages atthe hottest the hottest and coldest and coldest temperature temperature

- - managementmanagement

Would you launch?

Congressional Hearings Evidence

No Damage LegendDamage hard to read

Congressional Hearings Evidence

Temperature correlation difficult

55 65 7560 70 80

1

Original Data

2

3

Clearer

1. Y-Axis amount of damage (not number of damage)55 65 7560 70 80

4

8

12

1. Y-Axis amount of damage (not number of damage)2. Include successes

55 65 7560 70 80

4

8

12

Clearer

1. Y-Axis amount of damage (not number of damage)2. Include successes3. Mark Differences

55 65 7560 70 80

4

8

12

Clearer

1. Y-Axis amount of damage (not number of damage)2. Include successes3. Mark Differences4. Normalize same temp

55 65 7560 70 80

4

8

12

Clearer

1. Y-Axis amount of damage (not number of damage)2. Include successes3. Mark Differences4. Normalize same temp

55 65 7560 70 80

4

8

12

Clearer

Damage on every flight below 65

No damage on every flight above 75

1. Y-Axis amount of damage (not number of damage)2. Include successes3. Mark Differences4. Normalize same temp

55 65 7560 70 80

4

8

12

Clearer

1. Y-Axis amount of damage (not number of damage)2. Include successes3. Mark Differences4. Normalize same temp5. Scale known vs unknown

55 65 7560 70 80

4

8

12

4

8

12

30 40 5035 45

XX

Clearer

Difficult

NASA Engineers Fail Congressional Investigators Fail Data Visualization is Difficult

But …

Lack of Clarity can be devastating

Counties in US 3101 Counties 50 pages

“The humans … are exceptionally good at parsing visual information.” Knowledge representation in cognitive science. Westbury, C. & Wilensky, U. (1998)

“If I can't picture it, I can't understand it”

Anscombe's QuartetI II III IV

x y x y x y x y10 8.04 10 9.14 10 7.46 8 6.588 6.95 8 8.14 8 6.77 8 5.76

13 7.58 13 8.74 13 12.74 8 7.719 8.81 9 8.77 9 7.11 8 8.84

11 8.33 11 9.26 11 7.81 8 8.4714 9.96 14 8.1 14 8.84 8 7.046 7.24 6 6.13 6 6.08 8 5.254 4.26 4 3.1 4 5.39 19 12.5

12 10.84 12 9.13 12 8.15 8 5.567 4.82 7 7.26 7 6.42 8 7.915 5.68 5 4.74 5 5.73 8 6.89

Average 9 7.5 9 7.5 9 7.5 9 7.5Standard Deviation 3.31 2.03 3.31 2.03 3.31 2.03 3.31 2.03Linear Regression 1.33 1.33 1.33 1.33

- Albert Einstein- Albert Einstein

Graphics for Anscombe’s Quartet

Do You Want?

Engineering Data?Engineering Data?

Pretty PicturesPretty Pictures

Do You Want?

Clean and Clear Clean and Clear

? ? ? ? ? ? ? ? ? ?? ?

Do You Want?

What is a day in the life look What is a day in the life look like for a DBA who has like for a DBA who has performance issues?performance issues?

Tuning the Database

Anscombe's QuartetI II III IV

x y x y x y x yAverage 9 7.5 9 7.5 9 7.5 9 7.5Standard Deviation 3.31 2.03 3.31 2.03 3.31 2.03 3.31 2.03Linear Regression 1.33 1.33 1.33 1.33

ComplexComplexAveragesAverages

Imagine Trying to Drive your Car

And is updated once and hourAnd is updated once and hour

Or would you like it to Or would you like it to look …look …

Would you want your dashboard to look like :Would you want your dashboard to look like :

How Can We Open the Black Box?

Max CPU

(yard stick)

Top Activity Top Activity

SQLSQLSessionsSessions

LOADLOAD