eXtreme Scale 15min Intro

®

© 2010 IBM Corporation

Introducing WebSphere eXtreme ScaleMarch, 2010

Ted Kirby

WebSphere eXtreme Scale Technical Evangelist

IBM Software Group | WebSphere software

© 2006 IBM Corporation 2

Typical Application ArchitectureWhere’s the bottleneck in this picture?



Memory Hierarchy

CPU Memory

Disk

20ns

20ms

CPU Memory

20ns

2ms

Net: Put data in memory. Exploit network for capacity and availability.

Data must get to CPU local memory to be processed.

machine machine

network

Get Data From Access Time Factor

Memory 20 ns 20 x 10-9 sec 1

Over network 2 ms 2 x 10-3 sec 100,000

Disk 20 ms 20 x 10-3 sec 1,000,000



Cache

CPUMemory

Disk

DB

EIS

20ns

20ms

2ms

Cache

Event Time Get Data From

Cache hit 20 ns Memory

Remote Cache hit 2 ms Over network

Cache miss* 20 ms Disk

*On cache miss, put data from disk in cache,

hoping for cache hit next time.

+ Larger cache increases cache hits.

+ Cache hits reduce response time and load on the disk.

network

Keep a copy of high-use data in a chunk of memory called a cache.



Traditional Cache eXtreme Scale Cache

JVM

App

Server

App

Logic

JVM

App

Server

App

Logic

Cache

A, B, C

JVM

Cache

D, E, F

JVM

JVM

App

Server

App

Logic

Cache

A

JVM

App

Server

App

Logic

Cache

A

machine machine

JVM

Cache

JVM

Cache

JVM

Cache

JVM

Cache

1,000s

machinemachine

eXtreme Scale data grid

WAY bigger cache



Partitioning: The key to Linear Scaling

A - Z

A B C D E F G

H I J K L M N

O P Q R S T

U V W X Y Z

Encyclopedia is partitioned into volumes on shelves.

eXtreme Scale partitions caches into shards in containers, which are stored in JVMs.

•A container holds a variable number of shards, limited by JVM size (2GB, or 64-bit…)

•Can have multiple containers on a machine.

•Elastic: The number of machines may vary over time. (Like opening more registers at the supermarket.)

•Add machines to handle more load

•Automatic recovery from node failures



What is WebSphere eXtreme Scale?

Very powerful cacheScales from simple in-

process topologies to powerful distributed topologies.

Platform for building powerful XTP/Data Grid applications

Form of in-memory databaseManage application stateScales to 1000’s of serversSometimes referred to as

Distributed Application State Management.

A flexible framework for realizing high performance, scalable and data-intensive

applicationsNew York San Francisco

London Shanghai

3 Use Cases



IBM WebSphere eXtreme Scale

Proven mature product:– Third major release of product with V7.0– Public References– Private References– Used at some of the largest web sites in the

world

Lightweight runtime footprint (15MB jar)

Integrates with all versions of WebSphere and almost any Java-based application container or Java Virtual Machine (1.4.2 and higher)

Proven multi-data center capabilities

Proven low-latency access to data

8



Common set of patterns

Side Cache

– Hibernate / OpenJPA L2 cache

– Dynamic Cache (ND 6.1/ ND 7.0)

• Page fragment cache

• Web services cache

• Command cache

– SOA State store

– SOA Result cache

• ESB cache mediation

Benefits

– Improved response time

– Reduced load on DB

9

9

AppAppAppApp




Inline backend cache

– Loaders used to integrate with an existing data service

– Read through cache

– Write through cache

System of Record Data Store

– Cache is used as the system of record

– Write behind technology pushes changes asynchronously to the backend.

10

10

AppAppAppApp

Loader




HTTP Session Store

– Why use database for short-lived data?

– Application uses standard HTTPSession APIs

– Can be shared between applications sharing same domain name

– Grid runs inside web cluster JVMs using affinity

Application Session

– Application uses eXtreme Scale APIs to manipulate the shared session

– Works even when web apps use different domain names

Multi data center support for sessions

– SOA Sandbox exercise coming soon

11

11




XTP (eXtreme Transaction Processing)

– Lowest possible latency

– Application code runs in grid itself

• Map/Reduce API supported

– Events routed to correct partitions for processing

• Gateway forwards events to correct partition

• JVM hosting a partition directly subscribes to events

– Write behind

• Databases relegated to durable log and reports

12

12



Simple Caching ScenariosChallenges

• Redundant calls, doing something over and over again, on expensive systems

• Generally, to access data that does not change much (e.g., user profiles)

• Often, multiple systems accessed (e.g. to fetch different parts of user profile)

• Takes too long to retrieve the data, which gets worse as transaction volumes grow

Offload Redundant Processing

Benefits• Free up expensive backend

systems for critical tasks • Reduce costs of system cycles for

repetitive data retrieval • Increase performance through in-

memory, network cache

Challenges• Web sites that need better management

and automatic fail-over of Web sessions – usually Commerce or retail-related sites

HTTP Session Replication

Benefits• Automatic “drop-in” eXtreme

Scale cache without invasive coding changes

• Higher availability and performance for revenue-producing applications

Challenges • Web applications that use (WAS) Dynamic

Cache service and need better performance and scalability of their caching investment

“Drop-In” extension for Dynamic Cache

Service Provider

Benefits• Better performance: turbo-charge

WAS caching layer via eXtreme Scale “Drop-In” cache with no coding changes

• Better scalability: Entire eXtreme Scale grid is now your WAS cache

• More efficient: Single seamless network cache to store data



Client Usage: One of the largest retail US banks

Retail Banking & Investments

35x reducedresponsetimes

22 Milliononline banking users

Next-generation Online Banking – Before: 700ms to login with 2 backend calls – After: 20ms to login with profile cache access– $6M/year cost savings in MIPs reduction – 700k logins per hour across 3 data centers– 8Gb of data transfer per hour between DC’s– 60 million page views/day– 10 million logins per day– 3 active data centers

$500kreducedcosts per month

20x reduction in “lost sessions”

14

Side Cache - ESB Cache Mediation



Entertainment 10x reducedresponsetimes

7 Billion requests per day

Fantasy Sports Web Infrastructure– Before: 60ms response time against database– After: improved to 6ms response time – 450k concurrent users– 80k requests per second up to 1M in 2011– 6 weeks from concept to production

Client Usage: Social Media Website

15

OracleORMGridApp ORMGrid

ORMGrid

Inline system-of-record cache,with write-behind.

Partitioning smashes scaling barrier.



Benefit Summary

Caching

–Improved response time

–Reduced load on DB

Partitioning

–Smashes the scaling barrier

–Allows elasticity for linear scaling, predictable response times



Next StepsThe Getting Started with WebSphere eXtreme Scale Wiki

http://bit.ly/eXtremeScaleStart -- First page is an annotated bibliography

“Understanding WebSphere eXtreme Scale and how it works” developerWorks article

http://bit.ly/eXtremeScaleIntro

Download WebSphere eXtreme Scale for free and build a trial apphttp://bit.ly/eXtremeScaleTrial

Run WebSphere eXtreme Scale in the clouds! -- on Amazon EC2

http://bit.ly/extremescaleAMI

Engage with the Emerging Technology Community on My developerWorkshttp://bit.ly/dwEmerge

Engage with the Emerging Technology Community on My developerWorkshttp://facebook.com/websphereemerge

Read Billy Newport’s blog on XTPhttp://www.devwebsphere.com/



18

eXtreme Scale 15min Intro

Documents

cache miss

cache hits

remote cache

larger cache

event cache

app server app logic

powerful cache scales

server logic jvm app