The Next Generation Application Server – How Event Based Processing yields scalability

Guy KorlandGuy Korland

R&D Team LeaderR&D Team Leader

Agenda

• Preview

• The Problem

• The Solution

• Event Containers

• Example

• Benchmarks

• Customer Use Cases

• Summary Q&A

About me…

• Core Team Leader – GigaSpaces since 2005

• MSc – Technion (Prof. Roy Friedman)

• PhD candidate – Tel Aviv University (Prof. Nir Shavit)

• Lead Deuce STM – (www.deucestm.org)

Java Software Transactional Memory

GigaSpaces XAP – Designed For:

Performance

Scalability

Latency

GigaSpaces Evolution

Single space

Load Balance

Partition & Replication

SLA container

Event Container

NG Application

Server

PaaS Cloud

2000 2003 2005 2006 2007 2008 2009

Not going to talk about…

• Jini (Java SOA)

• Data Grid implementation.

• Map-Reduce.

• JDBC/JMS/JPA.

• Cloud computing.

• Batch processing.

• Mule ESB.

• WAN vs LAN

• Different languages interoperability.

• TupleSpace model extension.

• JDK improvements (RMI, Reflection, Serialization, Classloading…)

Agenda

• Preview

• The Problem

• The Solution

• Example

• Benchmarks

• Summary Q&A

Today’s Reality – Tier Based ArchitectureSeparate technology implementation

Bottlenecks in all areas where state is stored, architecture can’t scale linearly!

Separate technology implementation

AuctionOwner

Traditional Architecture - path to complexity… (marktplaats.nl)

Auction Service

Bid Service

Trade Service

Place bid

Info Service

Timer Service

Auction

Service

AA BB TT II

Bidder

Validate

Result

ProcessBid

Bid Accepted

Bid Result

Process Trade

Get Bid Result

Traditional Architecture - path to complexity…

Business tier

Back-up

Redundancy doubles network traffic

Bottlenecks are created

Latency is increased

Separate failover strategy and implementation for each tier

Bidder

AuctionOwner

Auction Service

Bid Service

Trade Service

Info Service

Timer Service

AA BB TT II

Do you see the Problem?

Business tierScalability is not linear

Scalability management nightmare

Back-upBack-up

AA BB TT

Bidder

AuctionOwner

There is a huge gap between peak and average loads

100,000,000

200,000,000

300,000,000

400,000,000

500,000,000

600,000,000

700,000,000

800,000,000

900,000,000

1,000,000,000

1,100,000,000

1,200,000,000

1,300,000,000

J-04 M-04 M-04 J-04 S-04 N-04 J-05 M-05 M-05 J-05 S-05 N-05 J-06 M-06 M-06 J-06 S-06 N-06 J-07 M-07 M-07 J-07 S-07

Bottlenecks, Performance, Scalability and High availability headaches

Bad PublicityRevenue LossCustomer DissatisfactionRegulatory Penalties

TBA – Summary

• Historically the following has been done…

– Tune, tune and tune configuration and code

• Once a bottleneck has been resolved, the next one glooms

– Hardware over provision

• To make sure that at peak times the response times were still acceptable

– Hardware upgrades

• To get rid of bottlenecks, whose origin was impossible to track down

– Alternative patterns

• Avoiding 2-phase-commit, using patterns like ‘compensating transactions’

• Using Active/Passive failover, to make the response times faster, risking and

in fact accepting potential data-loss

• Partition the database, but not for size-reasons

Agenda

• Preview

• The Problem

• The Solution

• Example

• Benchmarks

• Summary Q&A

Event Containers

Based on JavaSpaces

The Master Worker Pattern

GigaSpaces - based on Shared Transactional Memory

• Write – writes a data object

• Read – reads a copy of a data object

• Take – reads a data object and deletes it

• Notify – generates an event on data updates

Write + Read Data Caching

Write + Notify Messaging - Pub/Sub

Write + Take Master Worker

Event Containers

AB T I

Step 1 – Create a Processing Unit

Business tier

Processing Unit

Single model for design, deployment and management

No integration effort

Manage data in memory

Collapse the tiers

Collocate the services

Auction Service

Bid Service

Trade Service

Info Service

Timer Service

Bidder

AuctionOwner

AB T I

Step 2 – Async Persistency

Processing Unit

Validate

Process Bid

Process Trade

Process Results

Place Bid

Get Bid Results

Persist for Compliance & Reporting purposes:

- Storing State- Register Orders- etc.

Collocation of data, messaging and services in memory:

Minimum Latency (no network hops)

Maximum Throughput

Auction Service

Bid Service

Trade Service

Info Service

Timer Service

Bidder

AuctionOwner

AB T I

Step 3 – Resiliency

Processing Unit

Single, built-in failover/redundancy investment strategy

Fewer points of failure

Automated SLA driven failover/redundancy mechanism

Continuous High Availability

SLA Driven Container

Backup

AB T I

Processing Unit

Step 3 – Resiliency

Automated SLA driven failover/redundancy mechanism

Continuous Availability

Self Healing Capability

Backup

Single, built-in failover/redundancy investment strategy

Fewer integration points mean fewer chances for failure

BackupPrimary

Step 4 – Scale

Processing Unit

Write Once Scale Anywhere:

Linear scalability

Single monitoring and management engine

Automated, SLA-Driven deployment and management

Scaling policy, System requirements, Space cluster topology

Backup

AB T I

Backup

AB T I

Event Containers

Step 5 – Auto Scale Out

Processing Unit – Scalability Unit

Single Processing Unit Processing Unit - Scaled

Involves Config Change

No code changes!

Processing Unit – High-Availability Unit

Sync Replication

Primary - Processing UnitBusiness logic – Active mode

Backup - Processing UnitBusiness logic – Standby mode

Database Integration - Async persistency

Sync Replication

Primary - Processing UnitBusiness logic – Active mode

Backup - Processing UnitBusiness logic – Standby mode

Mirror Process

Initial Load

Async Replication

XAP = Enterprise Grade Middleware

• Scale-out application server

End 2 End scale-out middleware for: Web, Data, Messaging, Business logic

Space Based Architecture – designed for scaling stateful applications In-memory

• Proven performance, Scalability, Low latency, Reliability

• SLA Driven

• Unique database scaling solution that fits cloud environment

In Memory Data Grid

O/R mapping support

• Support major Enterprise languages

Java, .Net, C++

Agenda

• Preview

• The Problem

• The Solution

• Example

• Benchmarks

• Summary Q&A

Built-in Event Containers

• Polling Container• Notify Container

Processing Unit

PollingEvent Container

ServiceBean

NotifyEvent Container

ServiceBean

Messaging

Take Write Notify

Polling Container

• Used for point-to-point messaging• Container polls the Space

for events• Comparable with the

way Ajax works

Processing Unit

PollingEvent Container

ServiceBean

Take Write

Notify Container

• Used for publish-subscribe messaging

• Space notifies the container Processing Unit

NotifyEvent Container

ServiceBean

Notify

Typical Application

Service Grid Summary

Powerful Universal ContainerJava/Net/C++

Distributed

Fault Tolerant

Object based

Transactional

Publish/Subscribe

Agenda

• Preview

• The Problem

• The Solution

• Example

• Benchmarks

• Summary Q&A

Event Containers

@SpaceClass(fifo=true)public class Data { … @SpaceId(autoGenerate = true) public String getId() { return id; } public String setId(String id) {

this.id = id; }

public void setProcessed(boolean processed) { this.processed = processed; } public boolean isProcessed(boolean processed) { return this.processed; }}

The POJO Based Data Domain Model

SpaceClass indicate that this is a SpaceEntry – SpaceClass includes classlevel attributes such as FIFO,Persistent…

SpaceId used to define the key for that entry.

@SpaceDataEvent to be called when an event is triggered.

public class DataProcessor{

@SpaceDataEvent public Data processData(Data data){

… data.setProcessed(true);

//updates the space return data;

Data Processor Service Bean

Updates the data in the Space.

<os-events:polling-container id="dataProcessorPollingEventContainer" giga-space="gigaSpace">

<os-events:tx-support tx-manager="transactionManager"/>

<os-core:template>

</bean>

</os-core:template>

<os-events:listener>

<os-events:annotation-adapter>

<os-events:delegate ref="dataProcessor"/>

</os-events:annotation-adapter>

</os-events:listener>

</os-events:polling-container>

Wiring Order Processor Service Bean through Spring

The event Template

The event Listener

public class DataFeeder {

public void feed(){

Data data = new Data(counter++); data.setProcessed(false);

//feed data gigaSpace.write(data); }}

Data Feeder

Feed Data

Remoting – Taking one step forward

Reduce

public interface IDataProcessor {

// Process a given Data objectData processData(Data data);

Remoting – IDataProcessor Service API

@RemotingService public class DataProcessor implements IDataProcessor {

public Data processData(Data data) { …

data.setProcessed(true); return data;

Remoting - DataProcessor Service

public class DataFeeder { private IDataProcessor dataProcessor; public void setDataProcessor(…) {

this.dataProcessor = dataProcessor; }

public Data feed(){ Data data = new Data(counter++);

// remoting call return dataProcessor.process(data) }}

Remoting - Order Feeder

Summary

Agenda

• Preview

• The Problem

• The Solution

• Example

• Benchmarks

• Summary Q&A

Scale up Throughput Benchmark – Physical Deployment Topology

Remote (multiple machines , multiple processes)

white box Client

X4450GigaSpaces

4 spaces , one per GSC

X4450GigaSpaces

4 spaces , one per GSC

Switched Ethernet LAN

Embedded (one machine , one process)

X4450 Client

GigaSpaces8 spaces

x4450 - stac-sun-1 - Embedded Space - TP vs. Multiple Threads - 8 Partitions

200000

400000

600000

800000

1000000

1200000

1400000

1600000

1800000

2000000

1 2 3 4 6 8 10 12 16 20 30

Client Threads

Write TP

Read TP

Take TP

Scale up Throughput Benchmark – Embedded mode

1.8 Million read sec!

1.1 Million write/take

Scale up Throughput Benchmark – Remote mode

x4450 - stac-sun-3 - Remote Space - TP vs. Multiple Threads - 4 partitions

100000

1 2 3 4 5 8 12 16 20 24 26 30 34 38 42 46 50 54 58

Client Threads

ec) Write TP

Read TP

Take TP 90,00 read sec!

45,00 write/take sec!

Agenda

• Preview

• The Problem

• The Solution

• Example

• Benchmarks

• Summary Q&A

Event Containers

Web Container Grid

Web application – Pet Clinic

Classic Architecture – Step 1- Request Submission

Web PU

Processing UnitProcessing UnitProcessing UnitProcessing Unit

ServiceBean

Replication

ServiceBean

Replication

Primary 1 Backup 1 Primary 2 Backup 2

1. User Click Submits request

Apache Load-Balancer

2. Route Request

Web PU

3. invocation 3. invocation

Async Mirroring

Get request and invoke

Service

Data Grid

Classic Architecture – Step 2- Retrieve Results

Web PU

Processing UnitProcessing UnitProcessing UnitProcessing Unit

ServiceBean

Replication

ServiceBean

Replication

Primary 1 Backup 1 Primary 2 Backup 2

3. User getting Page

Apache Load-Balancer

2. Route Request

Web PU

1. Result returned

AggregatedResult

Result

1. Result returned

Result

Reducer

Async Mirroring

Page Generation

Data Grid

Web Application Benchmark Results - Capacity

Web Benchmark (pet clinic) - Latency vs. Users

010203040506070

1 2 3 4 5 10 15 20 40 50 60 70 80 90 100

1 Server

2 Servers

3 Servers

Web Application Benchmark Results - Capacity

Web Benchmark (pet clinic) - Latency vs. Users

50 100

ency 1 Server

2 Servers

3 Servers

Game Server

Query Notify

Intercepts updateevents

Game Servers

Table Feeder

Loading Game Tables into the partitioned spaces

Randomly updates the game tables

Publisher (lobby)Game Table Directory

Game Table search

Player search

Partitioned Space

Pub/Sub messaging

Scaling out

Publisher (II)

GameTable

Space Based Architecture – Game Server

Publisher Servers

Partitioned SpaceGameTable

Game ServersGigaSpaces Service Grid

Notify / Query

Uploading 30,000 players for 6000 tables

Randomly updates game tables

Java runtime

Physical backup

Running continues query per user

Space Based Architecture – Game Server

Dynamic repartitioning and load sharing I

Partitioned Space

Indexed Notify / Query template

Notify / Query template

Dynamic repartitioning and load sharing II

Partitioned Space

SLA Driven Container SLA Driven Container

Partitioned Space

Backup Space

• 6000 tables

• 30,000 players

Partitioned Space

Backup Space

• 4000 tables

• 20,000 players

Partitioned Space

Backup Space

• 2000 tables

• 10,000 players

Throughput: ~6K/secThroughput: ~12K/secThroughput: ~18K/sec

Scaling

Agenda

• Preview

• The Problem

• The Solution

• Example

• Benchmarks

• Summary Q&A

Thank You!

Appendix

SLA:• Failover policy• Scaling policy• Ststem requirements• Space cluster topology

PU Services beans definition

SLA Driven Deployment

Fail-OverFailure

Continuous High Availability

VM 1 ,2GGSCGSC

VM 3 , 2GGSCGSC

Dynamic Partitioning = Dynamic Capacity Growth

VM 2 ,2GGSCGSC

Max Capacity=2GMax Capacity=4GMax Capacity=6G

E FPartition 1Partition 1

A BPartition 2Partition 2

Partition 3Partition 3

In some point VM 1 free memory is below 20 % - it about the time to increase the capacity – lets move Partitions 1 to another GSC and

recover the data from the running backup!

Later .. Partition 2 needs to move… After the move ,

data is recovered from the backup

VM 5 , 4GGSCGSCVM 4 ,4GGSCGSCA B

E FPartition 1Partition 1

P - PrimaryP - Primary

B - BackupB - Backup

Executors

Task Executors – Task Execution

Executing a task is done using the execute method

AsyncFuture<Integer>future=gigaSpace.execute(newMyTask(2)

);intresult=future.get();

Processing UnitClient

TaskProxy

Result

Task Executors – Task Routing

Routing a task can be done in three ways1. Using the task itself2. Passing a POJO to the execute method3. Specifying a routing-parameter in the executemethod

Client

Processing Unit Processing Unit Processing Unit

ClientProxy

Result

Task Executors – DistributedTask Execution

Executing a distributed task is done using the execute method

AsyncFuture<Integer>future=gigaSpace.execute(newMyDistTask()

);intresult=future.get();

Client

Task Task Task

ClientProxyAggregated

Result

Task Task Task

ResultResult

Result

Reducer

Task Executors – DistributedTask Routing

Routing a distributed task can be done1. In the same ways as with the plain Task interface2. By broadcasting3. Specifying a number of routing-parameters in the executemethod

Client

Task Task

ClientProxyAggregated

Result

Task Task

ResultResult

Reducer

Service Executors

Operations

IMDG Basic Operations

Application

WriteMultipleApplication

Application

ReadMultiple

Application

TakeMultiple

Application

ExecuteApplication

Notify

IMDG Access – Space Operations – Write Operation

write-operation writes a new object to a space

Instantiate an object

Set fields as necessary

Write the object to the space

Auctionauction=newAuction();auction.setType("Bicycle");gigaSpace.write(auction);

Application

IMDG Access – Space Operations – Read Operation

read-operation reads an object from a space

A copy of the object is returned

The original copy remains in the space

Build a template/query (more on this later)

Read a matching object from the space

Auctiontemplate=newAuction();AuctionreturnedAuction=gigaSpace.read(template);

Application

Object SQL Query Support

Supported Options and QueriesOpeations: =, <>, <,>, >=, <=, [NOT] like, is [NOT] null, IN.

GROUP BY – performs DISTINCT on the POJO properties

Order By (ASC | DESC)

SQLQuery rquery = new SQLQuery(MyPojo.class,"firstName rlike '(a|c).*' or ago > 0 and lastName rlike '(d|k).*'");

Object[] result = space.readMultiple(rquery);

Dynamic Query SupportSQLQuery query = new SQLQuery(MyClass.class,“firstName = ? or lastName = ? and ago>?");

query.setParameters(“david”,”lee”,50);

Supported Options via JDBC APICOUNT, MAX, MIN, SUM, AVG , DISTINCT , Blob and Clob , rownum , sysdate , Table aliases

Join with 2 tables

Non Supported

HAVING, VIEW, TRIGGERS, EXISTS, BETWEEN, NOT, CREATE USER, GRANT, REVOKE, SET PASSWORD, CONNECT USER, ON.

NOT NULL, IDENTITY, UNIQUE, PRIMARY KEY, Foreign Key/REFERENCES, NO ACTION, CASCADE, SET NULL, SET DEFAULT, CHECK.

Union, Minus, Union All.

STDEV, STDEVP, VAR, VARP, FIRST, LAST.

# LEFT , RIGHT [INNER] or [OUTER] JOIN

IMDG Access – Space Operations – Take Operation

take-operation takes an object from a space

The matched object is removed from the space

Build a template/query (more on this later)

Take a matching object from the space

Auctiontemplate=newAuction();AuctionremovedAuction=gigaSpace.take(template);

Application

IMDG Access – Space Operations – Update Operation

update is equivalent to performing take and write

Executed in a single atomic call

AuctionItemitem=newAuctionItem();item.setType("Bicycle");gigaSpace.write(item);item=gigaSpace.read(item);item.setType("Motorbike");ObjectreturnedObject=space.update(item,null,Lease.Forever,2000L,

UpdateModifiers.UPDATE_OR_WRITE);

Application

Update

IMDG Access – Space Operations – Batch API

Apart from the single methods GigaSpaces also provides

batch methods

The methods are:

writeMultiple: writes multiple objects

readMultiple: reads multiple objects

updateMultiple: updates multiple objects

takeMultiple: reads multiple objects and deletes them

Notes:

Performance of the batch operations is generally higher

Requires one call to the space

Can be used with Template matching or SQLQuery

• writeMultiple writes the specified objects to the space.

Auction[]auctions=newAuction[]{newAuction(10),newAuction(20)

auctions=gigaSpace.writeMultiple(auctions,100);

Application

WriteMultiple

• readMultiple reads all the objects matching the specified template from the space.

Auctionauction=newAuction();Auction[]auctions=gigaSpace.readMultiple(auction,100);

Application

ReadMultiple

• takeMultiple takes all the objects matching the specified template from the space.

Auctionauction=newAuction();Auction[]auctions=gigaSpace.takeMultiple(auction,100);

Application

TakeMultiple

• updateMultiple updates a group of specified objects.

Auction[]auctions=newAuction[]{newAuction(10),newAuction(20)

auctions=gigaSpace.updateMultiple(auctions,100);

Application

UpdateMultiple

IMDG Summary

Powerful shared memory Service

Distributed

Fault Tolerant

Object based

Single and Batch Operations

Transactional

The Next Generation Application Server – How Event Based Processing yields scalability

scale processing unit

batch processing

data object

business tier scalability

collocation of data

data caching

data updates

sladriven deployment

Technology

Performance & scalability of Couchbase Server – Couchbase....

Scalability - Number of Users Per Presentation Server

SQL Server Megaservers: Scalability, Availability,...

Towards Scalability for Federated Identity Systems … ·.....

Windows Server 8 Hyper-V Component Architecture … Server.....

Print Server Scalability and Sizing Technical Overview White...

Performance and Scalability Benchmark: Oracle ... · PDF...

Moving ERP to SQL Server Yields Substantial...

ArcGISArcGIS Server Performance and Scalability Server...

ArcGIS for Server: Performance and Scalability Testing...

ArcGIS Server: Optimizing Performance and Scalability

Oracle9i Application Server: Availability, Scalability...

Using Server and Storage Virtualisation: Our journey to...

Design for Scalability - Pennsylvania State University...

SIMPLE Presence Traffic Optimization and Server Scalability

Windows Server 2012 Beyond · PDF fileWindows Server 2012...