Scalable Transactions in the Cloud

1 CloudTran

Scalable Transactions in the Cloud

Matthew Fowler, NT/e

CloudTranCloudSave

2 CloudTran

?

• And the answer is– platform for mainstream Java developers– to use IMDG– for scalable, commercial applications– without worry and minimal hassle– for commercial advantage

• It's a lump of middleware– built on, adding value to GigaSpaces

3 CloudTran

2001

• WebLogic/J2EE specialisation

• One week training course– 4-point architecture for dummies

• Messed-up architecture– revenue down

• Automating server-side applications– J2EE/EJB– Spring/Hibernate

4 CloudTran

3-5...5-10...10-20...1,000,000 • Tracy's story: the path of successful apps

– Database– Caching– In-memory Data Grid

• The 50,000 club• Application scale drivers

– Mobile phone growth, iPhone Apps– Micropayments– e-commerce continued growth

5 CloudTran

Get an edge with performance

Please wait

“Latency really matters ... 100ms of latency costs

1% in sales.” Amazon

.....................“... almost half of visitors will abandon a site if they perceive a page or feature

takes longer than 2 seconds to load. ”

GetElastic

“An extra 0.5 seconds in

search page generation time

dropped traffic by 20%.” Google

6 CloudTran

6.5m, x10yrs, $400bn/yr

• Mainstream Java developers– 6.5m– most have 5-10 years experience– 50 million man-years experience

• Plain old application development market– $400bn/year

• Can they build an IMDG application?– How can IMDG go mainstream?

7 CloudTran

Explaining it to your Mom / Boss

IMDG - SOR

Persistent Storage

8 CloudTran

Explaining it to a techie

1. System of Record in IMDG. Keep DB for• warehouse apps/BI• sleeping at night.

2. Catching the money:– ACID transactions

– throughput, scalability, bullet-proof reliability – distributed, data + messaging

3. ORM - Object references, not foreign keys.Easy to program. Entity groups for performance.

9 CloudTran

In-Memory Data Bases - Are You Crazy?

• What's it worth:– Loss of sales, traffic - 5% vulnerable, saved by

speed of IMDG

• For $100m/year co:– $5m/year revenue for good behaviour– Customer/order/product data - 2million * 16Kb– 8 servers in grid for 32GB live data

• 8 servers isn't a lot• Worth doing the numbers!

10 CloudTran

Low Reliability

Complicated Programming

Unintended Consequencesof

Unknowing

Distributed Transactions

Fear and loathing ...

11 CloudTran

, 1, 2, 3, ...

• Other alternatives– forget transactions, forget databases

• Dan's the Man

• GoogleApps on V2 last we heard

12 CloudTran

How is it possible?

Distributed Cloud Transactions

• Grid connected– Helland's get out clause

• System of Record is in the grid– No voting - 1PC not 2PC

• Commit to backed-up memory• Leverage the GigaSpaces platform

– SBA/Entity Groups, Transactions, SQL Queries, Backups

Redefining the problem

13 CloudTran

200/ ... 2,000 ... 20,000/second

14 CloudTran

Transactions you can count on

15 CloudTran

Transactions you can count on

16 CloudTran

Herding Cats - Java Style

• How to distribute data

• How to find it

• How to resolve references – IMDG versus user view: FK ↔ OO

• Atomicity on failure

• Timeouts

• Scalability

• Consistency and isolation

17 CloudTran

The 'T' Word

• GigaSpaces Local Transactions

• GigaSpaces Distributed Transactions

• Mirror service– see Cat-Herding 101

18 CloudTran

How CloudTran ORM works

Client

TxB

Gridsearch

OL

Data Data

OrderService

Partitioning

(entity groups)

Co

mm

it

Confirm

Con

firm Com

mit

Commit

Datasources

Tx

Messaging

19 CloudTran

300 .. 700 .. 900 .. 2,100

• Performance of transaction buffer– Tiny Transactions per second

20 CloudTran

Virgin Airways

LastMinute.com

IMDG

In-Cloud Federated Applications

IMDG

CloudTran -Federated

Transaction Buffer

21 CloudTran

Scalable transactions in the cloud?

– platform for mainstream Java developers– to use IMDG– for scalable, commercial applications– without worry and minimal hassle– for commercial advantage– GigaSpaces

CloudTran

22 CloudTran

End