Introduction to Cloud Computing and Big Data

Introduc)on to Cloud Compu)ng and Big Data

Waheed Iqbal, Ph.D h<p://www.waheediqbal.info

12th May, 2015

Google Trends: Cloud Compu)ng, Big Data, and Distributed Systems

Cloud Compu)ng Defini)on

Cloud compu)ng is a model to provide scalable resources (network, storage, applica)ons, services, compu)ng power etc.) over the Internet with minimal management efforts.

Cloud Compu)ng Defini)on (Cont.) Na)onal Ins)tute of Standards and Technology (NIST) has published 16th draS of Cloud Compu)ng defini)on. Cloud Compu)ng model is composed of following five essen)al characteris)cs: 1.  On-‐demand self service (get resources/services without human

interven)on) 2.  Broad network access (accessible using mobile, laptop, tablets, and

worksta)ons) 3.  Resource pooling (different physical and virtual resources dynamically

assigned and reassigned according to consumer demand)

4.  Rapid elas)city (shrink and grow capabili)es) 5.  Measured services (resource usage monitor, control, and report

transparently)

Cloud Compu)ng Ecosystem De

ploymen

t Mod

els

Service Mod

els

Cloud Ecosystem (Cont.)

Source: h<p://blog.ascens-‐ist.eu/2011/03/dreaming-‐of-‐fluffy-‐clouds/

Dynamic Provisioning

Lets discuss more about the most important characteris)c (Rapid Elas)city/Dynamic Provisioning) of Cloud Compu)ng!

Dynamic Provisioning (Cont.)

•  In tradi)onal compu)ng model, two common problems : 1. Underes)mate system u)liza)on which result in under provision

Resources

Demand

Capacity

1 2 3

Resources

Demand

Capacity

1 2 3

Resources

Demand

Capacity

Time (days) 1 2 3

Loss Users

Loss Revenue


2. Overes)mate system u)liza)on which result in low u)liza)on #

•  How to solve this problem ?? –  Dynamically provision resources

Unused resources

Demand

Capacity

Time

Resources


•  Cloud resources should be provisioned dynamically – Meet seasonal demand varia)ons – Meet demand varia)ons between different industries – Meet burst demand for some extraordinary events

Demand

Capacity

Time

Resources

Demand Capacity

Time

Resources

Mul)-‐)er Web Applica)on

Lets discuss a case using dynamic provisioning in mul)-‐)er web applica)ons!

Mul)-‐)er Web Applica)on (Cont.)

•  Single-‐)er web applica)on: consists only web server mostly to serve sta)c pages and dynamic pages without database interac)on

•  Mul)-‐)er web applica)on: consists on Web server, DB server, Applica)on server, Batch job processors etc

•  A single )er resource management is easy comparing to mul)-‐)er applica)on!


Network

Web Server Database Server


Network



Network


0 100 200 300 400 500 600 700 800 900

0 20 40 60 80

Respon

se Tim

e (m

s)

Number of Users/Request


Network


0 100 200 300 400 500 600 700 800 900

0 20 40 60 80

Respon

se Tim

e (m

s)

Number of Users/Request


Load balancing helps to maintain performance!


Network

Web Server Database Server Load Balancer


Network



Network

Web Server Database Server Load Balancer

Cloud Datacenter

Internet

Cloud Compu)ng: Take Home Message

Source: Introduc)on to Amazon Web Services by Jeff Barr, Senior Web Services Evangelist

Data Growth

•  Google (as of around 2009) processes around 24 petabytes of data every day

•  This is quite a lot, how much? Lets try to visualize the scale of data!

Let's imagine that a single byte is represented by a single grain of rice

1K or 1024 bytes would a bowl of rice

1 MB 1 GB 2 containers

1 TB 2048 Shipping Containers

The Model Has Changed…

The Model of Genera)ng/Consuming Data has Changed

Old Model: Few companies are genera)ng data, all others are consuming data

New Model: all of us are genera)ng data, and all of us are consuming data

Big Data Defini)on

No single standard defini)on! “Big Data is high volume, high velocity, and/or high variety informa7on assets that require new forms of processing to enable enhanced decision making, insight discovery and process op7miza7on.” (Gartner)

“Big Data is a data that is difficult to store and process using tradi7onal techniques on commodity hardware to analyse and extract knowledge.” (Waheed)

Who’s Genera)ng Big Data

Social media and networks (all of us are genera)ng data)

ScienJfic instruments (collec)ng all sorts of data)

Mobile devices (tracking all objects all the )me)

Sensor technology and networks (measuring all kinds of data)

Big Data: 3V’s (Model for Describing Big Data)

Type of Data •  Rela)onal Data (Tables/Transac)on/Legacy Data)

•  Unstructured Data / Text Data (Web, Applica)on/Server Logs)

•  Semi-‐structured Data (XML) •  Graph Data

–  Social Network

•  Streaming Data –  You can only scan the data once

Big Data and Cloud Compu)ng Technologies

Real-‐)me Tweeter Analy)cs: An Example

Ques)ons and Discussion

Acknowledgment

•  Some of the material used are copied from: –  Lecture Notes on Introduc)on to Cloud Compu)ng –  Introductory slides of course CS525 Large-‐Scale Data Management by Dr. Mohamed Eltabakh

–  Big-‐Data Tutotrial by Marko Grobelnik –  Big-‐Data Lecture Slides by Ruoming Jin's – What is cloud compu7ng by Read Maloney, Product Manger, Amazon Web Services

– Most of the images used in this presenta)on are taken from the Internet