The Story of BIG DATA

Presented by Krishna Kumar & Sanil Kumar, Architects @Huawei India

Human Evolution Video

Stores 1 petabyte to 2.5 petabytes…

Using just ~20watts power

26 different ways to code its "bits."

“a computer with the same memory and processing power would require 1 gigawatt of power, or "basically a whole nuclear power station to run one computer that does what our 'computer' does with 20 watts”

- Tom Bartol, a neuroscientist at the Salk Institute.

Lascaux cave painting.

About 20,000 years old…

Sophisticated and appealing to modern tastes

Quipu a wearable data storage and communication system

500 to 5000 years ago

cord color, manner of connection, relative placement, cord spacing, the types and placement of knots on individual cords

Tally sticks

Incised bones

Rock Art

picture writing

a different picture for every word..!

Early Scripts…

Volume

Velocity

Veracity

Variety

Value

Verification

Visualization

Low

Slow

High

Less

High

Not Critical

Simple, Easy

extremely large data sets that may be analyzed computationally to reveal patterns, trends, and

associations, especially relating to human behavior and interactions

1991 , Tim Berners-Lee

1999, the term Big Data appeared in Visually Exploring Gigabyte Datasets in Real Time, published by the Association for Computing Machinery

Volume

Velocity

Veracity

Variety

Value

Verification

Visualization

Low Massive

Slow Accelerated

High Uncertain Data

Less All Kind

High Low per Volume

Not Critical Very Critical

Simple, Easy Complex, Access

Volume Massive

Velocity Accelerated

90% of the world’s data was

created in the last 2 years.

Veracity Uncertain Data

~80% of the data generated is ‘dirty’

Variety All Kind

Different Industries…complex to derive value

Value Low per Volume

security / privacy / compliance

This is why there was so much uproar about the NHS hospitals sharing information with Google. People thought that if Google is sharing all this hospital information then Google will know too much about me.

Verification Very Critical – Unknown, Unknown!

Too much of information to be presented!

Visualization Complex, Access

Needs a framework to handle….!

A Simple one…

A Bit more complex…

CUSTOMER CARE IN 2020

Operator: “Thank you for calling Pizza Hut . May I have your… Customer: “Heloo, can I order..” Operator : “Can I have your multi purpose card number first, Sir?” Customer: “It’s eh…, hold……….on……889861356102049998-45-54610? Operator : “OK… you’re… Mr Singh and you’re calling from 17 Jalan Kayu. Your home number is 4094! 2366, your office 76452302 and your mobile is 0142662566. And now you are calling from home!” Customer: “How did you get all my phone numbers? Operator : “We are connected to the system Sir” Customer: “May I order your Seafood Pizza…” Operator : “That’s not a good idea Sir” Customer: “How come?

• Operator : “According to your medical records, you have high blood pressure and even higher cholesterol level Sir” Customer: “What?… What do you recommend then?” Operator : “Try our Low Fat Hokkien Mee Pizza. You’ll like it”

Customer: “How do you know for sure?” Operator : “You borrowed a book entitled “Popular Hokkien Dishes” from the National Library last week Sir” Customer: “OK I give up… Give me three family size ones then, how much will that cost?” Operator : “That should be enough for your family of 10, Sir. The total is $49.99 Customer: “Can I pay by! credit card?” Operator : “I’m afraid you have to pay us cash, Sir. Your credit card is over the limit and you owe your bank $3,720.55 since October last year. That’s not including the late payment charges on your housing loan, Sir.” Customer: “I guess I have to run to the neighbourhood ATM and withdraw some cash before your guy arrives” Operator : “You can’t Sir. Based on the records, you’ve reached your daily limit on machine withdrawal today” Customer: “Never mind just send the pizzas, I’ll have the cash ready. How long is it gonna take anyway?” Operator : “About 45 minutes Sir, but if you can’t wait you can always com and collect it on your motorcycle…”

Customer: ” What!” Operator : “According to the details in system ,you own a Scooter,…registration number 1123…” Customer: ” ????” Operator : “Is there anything else Sir?” Customer: “Nothing… by the way… aren’t you giving me that 3 free bottles of cola as advertised?” Operator : “We normally would Sir, but based on your records you’re also diabetic……. “ Customer: #$$^%&$@$%^ Operator : “Better watch your language Sir. Remember on 15th July 1987 you were convicted of using abusive language on a policeman…?” Customer: [Faints]

Which one sir?

That’s what I am

figuring out!

Network resiliency and big data applications

Solving network congestion for big data applications

Network consistency more of a focus than latency in big data

Prepare now for big data scalability later Network partitioning to handle big data

Application awareness for big data

networks Dig Deeper on Software-defined data

center

The phrase data science really took off after O’Reilly’s Strata Conference in 2011. That

conference brought 1,500 “data scientists” together.

Anthony Goldbloom, co-founder and CEO of Kaggle (https://www.kaggle.com/ ):

“data science will be bigger than software engineering (20 million software engineers now) in the next decade. Using R or the Python Data tools will take over SAS, Excel, etc.”

Combining data with robotics

NASA using it to analyze real-time data on Mars

Big data is use to predict treatment plans

and to predict diseases

Machine Learning technology will take us where computers predicts our diseases, also able to to find new disease patterns

Medical Training will be rewritten

Modern doctoring begins with a boot-camp experience: endless days of unending shifts, as young interns are forced to ingest—and deliver—diagnoses

with reflex like expertise. All that will go.

Computers to read emotions

-- combining the sensors with machine learning and with data algorithms to do what we always believed were very human skills

Real-time streaming : Window of opportunity for all!

Real-time streaming, insights into data will be the hallmarks of Big Data winners

“Autonomous agents and things”

will continue to be a huge trend, according to Gartner, including robots, autonomous vehicles, virtual personal assistants, and smart advisers

Prescriptive analytics…

- Google’s self-driving car to predict what is going to happen, when it is going to happen and why it is going to happen.

Cognitive technology will be the new buzzword

Big Data predictions on Euro Football 2016 turning true!

This clearly indicates that big data is not only transforming sport and is also changing the games that are based on sports.

Big Data Analysts from Bayern Munich predicted Portugal 67% of winning!!

“Fast data” and “actionable data”

will replace big data?! According to some experts that is going to happen! The argument is that big isn’t necessarily; focus on asking the right questions!

Using alterations in the way quartz refracts light, we could store data in a superdense form for centuries, according to researchers. UNIVERSITY OF SOUTHAMPTON

Future with nanotubes, quartz, etc.

Carbon nanotubes from being used as transistors for computer chips. It could mean the end of the silicon computer chip - IBM

Nanobots do not exist yet, but when they do, futurists predict possible uses for nanorobots will include molecular manufacturing (nanofactories) and medical nanobots that steer autonomously through your blood stream making repairs and guarding against infection.

Future with Photonics

Photonic integrated circuits have already given us a new perspective to the power and versatility of computation in the future.

Technology giants such as Intel, IBM, Microsoft and Google already have made huge investments in this direction.

Future with Quantum Computing

A regular computers have to solve one problem at a time in sequence, but quantum computers can solve multiple problems at the same time. That kind of

speed as the potential to revolutionize entire industries.

Accurate Weather forecasting ; Faster drug discovery; No traffic problems; Accelerated space discovery

There’s a philosophical debate on AI. Is AI teaching computers to beat humans? Where do we go?

IBM’s supercomputer Deep Blue beat world chess champion Garry Kasparov in a six game match.

IBM Watson defeated two of Jeopardy's greatest champions Ken Jennings and Brad Rutter - IBM Watson: 2016, Google created an AI that beat

Go world champion Lee Sedol. DeepMind Go system taught itself how to play Go. There were not as many experts involved, so this is a mind blowing breakthrough.

1997 Deep Blue

2011 Watson 2016 DeepMind

Machine Evolution Video

Thank You! Krishna & Sanil

Disclaimer: Images & many data are taken from Internet and only used for information sharing. We do not claim any other rights/correctness to it. Also no commercial usage of these slides allowed.