HYDSPIN Dec14 visual story telling

MAKING SENSE OF BIG DATA: VISUAL STORY TELLING

B GANES KESARI,

VP, GRAMENER

We handle terabyte-size data via non-traditional analytics and visualise it in real-time.

Gramener visualises

your data

Gramener transforms your data into concise dashboardsthat make your business problem & solution visually obvious.We help you find insights quickly, based on cognitive research,and our visualisations guide you towards actionable decisions.

A data visualisation and analytics company

Transaction data

Increasing volumes of data being churned out by systems

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

20

11

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

20

11

Social network data

Consumers embracing Web 2.0 & social media lifestyle

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

20

11

M2M data

Devices generating & logging data from every activity

Big data…

McKinsey Global Institute, Big Data, June 2011

Generation Analysis Consumption

creates opportunities

Volume of data

The larger the volume of data, the more likely it is that a firm will benefit from increasing use of data.

Variability

Greater fluctuations in performance offer more potential for a data-driven organisation to improve results.

Customer intensity

More customers (or stakeholders of any kind) offer greater potential for segmentation and tailored action.

Transaction intensity

This permits greater automation of decision making, allowing processing power to replace human judgement.

Turbulence

Frequency at which leaders and laggards change place in a sector indicates potential for disruption.

Each industry is poised to take advantage of big data to varying degrees. Some factors that increase the relevance of big data to an industry are:

… but at a high cost

Technology

Collection, storage, analysis and visualisation of data – all require investments in modern technology.

Talent

The deeper the analysis & data expertise a firm has, the better it can leverage data. But such talent is rare.

Organisational change

A shift in mind-set from experience-driven decision making to data driven decision making is required.

Data access

Collecting relevant data, storing it, and making it available to analysts in an easy manner requires investment.

Supplier ecosystem

A mature vendor ecosystem providing end-to-end or piece-wise solutions to these is not yet a reality.

Investments of various kinds are required to make the data actionable. It is not enough that the data just exists, or is collected. Some challenges are:

A DATA VISUALISATION

CHALLENGE…

You will see 3 questions.You have 30 seconds.

Try it!

Your timerstarts now

HOW MANY NUMBERS ARE ABOVE 100? 1

23 32 71 72 58 87 11 77 70 16

17 21 56 44 68 51 84 20 60 40

37 8 107 14 12 41 69 14 18 71

62 55 59 64 33 55 71 58 103 92

101 56 45 34 43 15 73 78 6 93

39 53 22 26 26 94 60 82 99 74

11 12 36 67 70 71 97 59 73 99

75 74 69 69 51 48 2 66 92 98

15 10 41 58 104 94 92 84 74 82

12 52 10 57 33 77 88 81 81 91

15 56 25 30 21 7 66 66 78 87

29 23 5 34 11 96 74 99 99 88

37 10 43 15 50 71 65 60 101 98

46 34 19 102 57 70 95 84 63 91

3 34 39 37 60 81 65 63 9 71

48 46 25 50 22 64 91 76 71 79

HOW MANY NUMBERS ARE BELOW 10? 2

23 32 71 72 58 87 11 77 70 16

17 21 56 44 68 51 84 20 60 40

37 8 107 14 12 41 69 14 18 71

62 55 59 64 33 55 71 58 103 92

101 56 45 34 43 15 73 78 6 93

39 53 22 26 26 94 60 82 99 74

11 12 36 67 70 71 97 59 73 99

75 74 69 69 51 48 2 66 92 98

15 10 41 58 104 94 92 84 74 82

12 52 10 57 33 77 88 81 81 91

15 56 25 30 21 7 66 66 78 87

29 23 5 34 11 96 74 99 99 88

37 10 43 15 50 71 65 60 101 98

46 34 19 102 57 70 95 84 63 91

3 34 39 37 60 81 65 63 9 71

48 46 25 50 22 64 91 76 71 79

WHICH QUADRANT HAS THE HIGHEST TOTAL?

23 32 71 72 58 87 11 77 70 16

17 21 56 44 68 51 84 20 60 40

37 8 107 14 12 41 69 14 18 71

62 55 59 64 33 55 71 58 103 92

101 56 45 34 43 15 73 78 6 93

39 53 22 26 26 94 60 82 99 74

11 12 36 67 70 71 97 59 73 99

75 74 69 69 51 48 2 66 92 98

15 10 41 58 104 94 92 84 74 82

12 52 10 57 33 77 88 81 81 91

15 56 25 30 21 7 66 66 78 87

29 23 5 34 11 96 74 99 99 88

37 10 43 15 50 71 65 60 101 98

46 34 19 102 57 70 95 84 63 91

3 34 39 37 60 81 65 63 9 71

48 46 25 50 22 64 91 76 71 79

3

A DATA VISUALISATION

CHALLENGE…

We’ll answer the same questions again.But with simple visual cues.

See how long it takes.

Your timerstarts now

23 32 71 72 58 87 11 77 70 16

17 21 56 44 68 51 84 20 60 40

37 8 107 14 12 41 69 14 18 71

62 55 59 64 33 55 71 58 103 92

101 56 45 34 43 15 73 78 6 93

39 53 22 26 26 94 60 82 99 74

11 12 36 67 70 71 97 59 73 99

75 74 69 69 51 48 2 66 92 98

15 10 41 58 104 94 92 84 74 82

12 52 10 57 33 77 88 81 81 91

15 56 25 30 21 7 66 66 78 87

29 23 5 34 11 96 74 99 99 88

37 10 43 15 50 71 65 60 101 98

46 34 19 102 57 70 95 84 63 91

3 34 39 37 60 81 65 63 9 71

48 46 25 50 22 64 91 76 71 79

HOW MANY NUMBERS ARE ABOVE 100? 1

HOW MANY NUMBERS ARE BELOW 10? 2

23 32 71 72 58 87 11 77 70 16

17 21 56 44 68 51 84 20 60 40

37 8 107 14 12 41 69 14 18 71

62 55 59 64 33 55 71 58 103 92

101 56 45 34 43 15 73 78 6 93

39 53 22 26 26 94 60 82 99 74

11 12 36 67 70 71 97 59 73 99

75 74 69 69 51 48 2 66 92 98

15 10 41 58 104 94 92 84 74 82

12 52 10 57 33 77 88 81 81 91

15 56 25 30 21 7 66 66 78 87

29 23 5 34 11 96 74 99 99 88

37 10 43 15 50 71 65 60 101 98

46 34 19 102 57 70 95 84 63 91

3 34 39 37 60 81 65 63 9 71

48 46 25 50 22 64 91 76 71 79

WHICH QUADRANT HAS THE HIGHEST TOTAL? 3

23 32 71 72 58 87 11 77 70 16

17 21 56 44 68 51 84 20 60 40

37 8 107 14 12 41 69 14 18 71

62 55 59 64 33 55 71 58 103 92

101 56 45 34 43 15 73 78 6 93

39 53 22 26 26 94 60 82 99 74

11 12 36 67 70 71 97 59 73 99

75 74 69 69 51 48 2 66 92 98

15 10 41 58 104 94 92 84 74 82

12 52 10 57 33 77 88 81 81 91

15 56 25 30 21 7 66 66 78 87

29 23 5 34 11 96 74 99 99 88

37 10 43 15 50 71 65 60 101 98

46 34 19 102 57 70 95 84 63 91

3 34 39 37 60 81 65 63 9 71

48 46 25 50 22 64 91 76 71 79

Humans are

pattern-seeking

story-telling

animals.

Amit Kapoor, http://narrativeviz.com/playbook

Amit Kapoor, http://narrativeviz.com/playbook

Amit Kapoor, http://narrativeviz.com/playbook

Can we understand the brief history of elections in India?

How have the political fortunes changed over time?

How did the biggest election of them all unfold in 2014?

VISUALIZING THEGENERAL ELECTIONS

EXPLORATORY | INTERACTIVE

~300 Parties fielding 8000 candidates

~800 Mn Registered Voters

~1 Mn booths served by 20 Mn people

~21,000 Votes/sec of live results

Varied data on several parameters

A Big Data problem… in every sense

India’s General Elections landscape…

https://gramener.com/election/parliament

LIVE ELECTION ANALYSIS

Our CNN-IBN Microsoft Election Analytics Canter, which you can see at www.bing.com/elections or election-results.ibnlive.in.com, served over 10 million requests on 16th May 2014 — the day of India election results.

This is one of the largest real-time visualisations that we (and perhaps many others) have attempted

http://ibn.gramener.com/live

<<Video recreating

how the Election results unfolded>>

Does any party hold a consistent 100% failure rate?

Which party holds record for being most persistent in adversity?

Which party’s candidates have lost deposits for nearly a decade?

INDIA’S MOST PERSISTENT PARTY

EXPLANATORY| STATIC

https://gramener.com/election/parliament#story.ddp

VISUALIZING WEATHER

How did weather change in India over the past century?

What were the hottest and coldest places?

Are there places that exhibit some interesting patterns?

EXPLANATORY| VIDEO

Image credit: https://www.flickr.com/photos/vesiaphotography/11627471004

100

YE

AR

SO

FIN

DIA

’SW

EA

TH

ER

1901

1911

1921

1931

1941

1951

1961

1971

1981

1991

2001

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

http://www.youtube.com/watch?v=WT0Aq41BaOQ

STORIES FROM TEXT

Can business impacting stories be mined from large bodies of text?

Can investors better read companies by studying Investor earning calls?

Can companies understand what analysts want & be better prepared?

EXPLANATORY| INTERACTIVE

Image credit: ttps://www.flickr.com/photos/a_mason/3009985823

Web Scraping Tokenization Part-of-Speech tagging

TransformEntity detection

Text Analytics Engine

HOW IS THE TEXT PROCESSED?

Analytics Engine

Compute

Visualization Engine

Ticker Qtr #Qns

AAPL 53% 3

AAPL 51% 7

GS 52% 6

MSFT 53% 4

... ... ...

MS 54% 9

JP 53% 6

... ... ...

Data Extraction

Ticker Qtr %Gr

AAPL 53% 23%

AAPL 51% -35%

GS 52% 95%

MSFT 53% 101%

... ... ...

MS 54% 14%

JP 53% 20%

... ... ...

https://gramener.com/transcriptanalysis/

VISUALIZING MOVIES

What are the popular, critically acclaimed ones?

Where do my preferences figure?

Which one should I watch next?

EXPLORATORY| INTERACTIVE

The Shawshank

Redepmption

The Godfather

The Dark Knight

Titanic

The Phantom

Menace

Twilight

New Moon

Wild Wild West

Transformers

The Good, The

Bad, The Ugly

12 Angry

Men

7 Samurai

Taare Zameen

Par

Rang De

Basanti

Yojinbo

MORE VOTES

BETTER RATED

Many unwatched movies

Few unwatched movies

Mix of watched & unwatched

Few watched movies

Many watched movies

Movies on the IMDb

3 Idiots

https://gramener.com/imdb/

http://demo.gramener.com:7056/twitteranalysis.html

BEST PLACES TO LIVE

FINDING ‘BEST PLACES’ TO LIVE IN

Can we plug into public data to better understand cities?

Can we identify the best places to live?

Can this be customized to an individual level?

EXPLORATORY| INTERACTIVE

Image credit: https://www.flickr.com/photos/dynamosquito/2431025077

http://indiatoday.intoday.in/best-cities-2014.jsp

WHAT DOES THE

WORLD SEARCH FOR?

What are some questions that interest people ?

How does this vary across countries?

Can we do ongoing ‘search-listening’?

EXPLORATORY| INTERACTIVE

Image credit: https://www.flickr.com/photos/uberculture/2561190022

https://gramener.com/search/#questions/how-to-

Amit Kapoor, http://narrativeviz.com/playbook

gramener.com blog.gramener.com http://slideshare.net/gramener

Ganes KesariTwitter: @kesaritweetsEmail: [email protected]

Session Slides available on Slideshare at:http://www.slideshare.net/gramener/hydspin-dec14-visual-story-telling