Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Accelerating Innovation with

Cloud Computing

Hari Vasudev

India Hadoop Summit - Bangalore

February 2010

I’m not selling anything

Cloud Computing is NOT

about saving money

Yahoo! is Perfect for Cloud Computing

HUNDREDSOF PROPERTIES / PRODUCTS

600MUNIQUE USERS / MONTH

300M+YAHOO! MAIL USERS / MONTH

HUNDREDSOF PETABYTES OF STORAGE

BILLIONSOF OBJECTS STORED

PETABYTESOF TRAFFIC DAILY

Yahoo! Cloud Strategy

• Creating a private Cloud for Yahoo!

• Optimizing for global Yahoo! properties

• Data processing and serving environments

• Multi-year effort

• Open Source

Inside Yahoo!’s Cloud

Yahoo!’s Open Source for Cloud

Cloud Solving Industry-wide Problems

• Mail abuse detection

• Dependent on globally synchronized data

• Cloud storage

• Global data replication

• Consistency

• Fast and easy to use

• Developers focus on task at hand

• Organizational commitment

• Investment• Investment

• Time

Cloud Computing is worth it!

Advertising

Optimization

& Delivery

Content

Optimization

Search

Index

Yahoo!’s Cloud Use CaseCaching, Load Balancing

Machine

Learning (e.g. Spam filters)

& DeliveryOptimization

Image/Video

Storage &

Delivery

RSS

Feeds

Attachment

Storage

Cloud improves dynamiccontent refresh rates and content refresh rates and consumer access speed

Cloud abstracts away scale for processing enormous for processing enormous

data sets

Cloud speeds advertising optimization by improving

15

optimization by improving infrastructure utilization

Cloud Speeds Time To Market

• YQL

• SQL-like language

• Query, filter, and join data across

web services

• YQL Open Data Tables built • YQL Open Data Tables built

on Cloud storage

• Simple and fast integration and

deployment

• Immediate access to global,

replicated, fast, reliable data store

16