Logging Last Resource Optimization for Distributed Transactions in Oracle Weblogic Server

Logging Last Resource Optimization for Distributed Transactions in

Oracle WebLogic Server

T. Barnes, A. Messinger, P. Parkinson, A. Ganesh, G. Shegalov, S. Narayan, S. Kareenhalli

OLTP: Online Transaction Processing

Transaction is an ACID contract● Atomic – all or nothing

● Consistent – from the application perspective

● Isolated – masked concurrency through locking or snapshots

● Durable – once committed changes survive subsequent failures

Checking = 2000Savings = 8000

Checking = 1000Savings = 9000

time

beginc -= 1000s += 1000commit

OLTP: Single Resource● A and D are typically implemented using Write-Ahead Logging

● Transaction recovery is “simple”: REDO phase, UNDO phase.BEGIN TRANSACTION

/* LSN = 1: log for undo and redo in MM buffer*/

UPDATE Accounts SET balance = balance – 1000 WHERE Number = 1

/* LSN = 2: log for undo and redo in MM buffer*/

UPDATE Accounts SET balance = balance + 1000 WHERE Number = 2

/* LSN = 3: log commit and force (5-6 orders slower)*/

COMMIT TRANSACTION

Accounts LSN=1

1 1000

2 8000

Accounts LSN=2

1 1000

2 9000

Accounts LSN=0

1 2000

2 8000

OLTP: Distributed / Two-Phase Commit

Transaction Coordinator

Resource 1 Resource 1

prepare --> force-log prepare force-log prepare

<-- OK <-- OK

commit --> force-log commit force-log commit

<-- ACK <-- ACK

Like a wedding ceremony

Coordinator: Will you ...? (prepare)

Resource: I will (OK)

Coordinator: I pronounce you … (commit)

2PC is A CI D

● 2PC is not about Concurrency Control.

● 2PC transaction is therefore

○ Globally Atomic

○ Locally Isolated

○ Locally Consistent

○ Globally Durable

OLTP: Queued Transactionsclient` app server database

begin transaction

req_q.enqueue(req1)

commit transaction

begin transaction

creq = req_q.dequeue()

resp = creq.execute()

res_q.enqueue(resp)

commit transaction

begin transaction

resp = res_q.dequeue()

process(resp)

commit transaction

WebLogic Server: Java EE++

● App containers: Web (Servlets, WS), EJB, app clients

● Services: Messaging (JMS),

Transactions (JTA), Database (JDBC), …

Example: Queued Transactions (JEE)

@MessageDriven(

mappedName="jms/inboundQueue“

activationConfig = {@ActivationConfigProperty(

propertyName = “connectionFactoryJndiName",

propertyValue = “jms/inboundConnectionFactory"

)}))

@TransactionAttribute(TransactionAttributeType.REQUIRED)public class OrderMDB implements javax.jms.MessageListener {

@Resource javax.jms.Queue outboundQueue; @Resource javax.jms.ConnectionFactory outboundCf; @Resource javax.sql.DataSource orderDataSource; public void onMessage(Message message) { java.sql.Connection jdbc = orderDataSource.getConnection();

javax.jms.Connection jms = outboundCf.createConnection();

// update SQL via JDBC, notify via JMS connections …

}

}

“School” Presumed Abort 2PC TM Resources

Timeline

prepare

yes

force-log prepared

commit

force-log commit

all-prepared: force-log commit

all-commit: log end

ack

2n+1 writes, 4n messages

PA2PC (1): TM (Coordinator)

PA2PC (2): Resource (Participant)

“Real Life” XA 2PC

TM Resources

Timeline xa_prepare

ack_preparedforce-log prepared

xa_commitforce-log commit

all-prepared: force-log commit

all-commit: log end

ack_committed

xa_startack_started

ack_endedxa_end

2n+1 writes, 8n messages

Standard 2PC Optimizations

● 1PC: if only one resource enlisted, prepare skipped

● Read-Only: if voted read-only, commit skipped

● XA ceremony of xa_(start|end) is always present

Nested 2PC: Coordinator Role Transfer

TC Res2 Res3

prepare commit

commitcommit

● Last Resource is committed in one phase

● 2n messages/ 2n-1 forced writes

● Known topology: linked Databases

[Gray’78]

p

c

c

WebLogic Design Constraints and Goals

● No control over foreign XAResource, TM and topology

● Broadband: minimize blocking RPC, not messages

● Unneeded XA on Res3: save xa_start, xa_end

Typical WLS Deployment

● JMS and TM share the same FileStore

● Collocated JMS connection cost is negligible

● JDBC Datasource is remote: blocking RPC

● DB internal resources (locks, latches, etc.) are more

expensive and JEE is not a single client

● Outbound JMS notifies about a JDBC update

● Ideally: JDBC updates visible before JMS updates

JDBC as Logging Last Resource

● User enables a non-XA JDBC Datasource as LLR

○ LLR table WL_LLR_<server> in the DS schema

○ No XA overhead for the LLR

● TM log is local log UNION LLR table log

○ WLS does not boot if any LLR table is unavailable

● Restriction: 1 LLR datasource / Transaction

● No coordinator transfers as in Nested 2PC

XA 2PC Commit with LL Resource

1. Prepare concurrently all non-LLR XAResources

2. Insert XID into the LLR table

3. Commit the LLR-Resource

4. If 3 is successful, commit non-LLR XAResources

5. Lazy garbage-collection of 2PC records of completed

transactions is piggybacked on future LLR transactions

LLR Failure Recovery

● Failure before LLR.commit() => global abort

● Failure during LLR.commit() => similar to media failure

○ Wait until LLR Datasource / table is available for read

○ Presence of the LLR commit log decides the global outcome

○ If unavailable for AbandonTimeoutSeconds log abandoned

● JVM/OS crash: TM scan local log UNION LLR

○ Usual transaction outcome resolution

● 2PC recovery guarantees are not compromised

LLR Savings

Back-of-the-envelope for the single-threaded case with Jeff Dean’s numbers [Google key notes]: ● xa_start (RPC),

● xa_end (RPC),

● xa_prepare (RPC + force-log)

● Insert into LLR table + commit done via single RPC

------------------------------------------------4xRTT + 1xDiskSeek = 4x500,000ns + 10,000,000ns = 12 milliseconds

LLR in DS Wizard: Non-XA Driver

LLR in DS Wizard: Safe unlike Emulate

Research Workload EAStress2004 [SPECjAppServer’04]

EAStress2004 Benchmark HW/Setup

Driver 1 (3x Dealer, 3x Manufacturing)

Driver 2 (3x Dealer, 3x Manufacturing)

External Supplier Emulator

2x Quad Core 3.7Ghz x86_64, 2MB L2, 8GB RAM

2x Quad Core 2.7Ghz x86_64, 2MB L2, 16GB RAM

2x Quad Core 2.7Ghz x86_64, 2MB L2, 16GB RAM

System Under Test

Oracle WebLogic Server 11gR1 (Middle Tier)

Oracle RDBMS 11gR1 EE(Database Tier)

2x Quad Core 2.83Ghz x86_64, 6MB L2, 16 GB RAM

4x Quad Core 2.7Ghz x86_64, 2MB L2, 64GB RAM

Performance Evaluation (Utilization)

MDB scenarioMT = WLS on JrockItDB = Oracle Database

EAStess2004 v1.08, IR = 700 (not reviewed by SPEC)

Performance Evaluation (Response Time)EAStess2004 v1.08, IR = 700 (not reviewed by SPEC)

Purchase Manage Browse Manufacturing

XA 4.20 2.40 5.40 3.75

LLR 1.50 1.20 1.90 3.00

improvement 2.8x 2x 2.8x 1.25x

Future Improvements (probably in 12c)

● LLR does not detect Read-Only

● Transaction GUID instead of LLR table for Oracle

Thank You. Questions?

oracle.com/weblogicoracle.com/benchmarks

WebLogic FileStore

● XA-capable KV store on local file system

● Mime design: allocate under write-head

○ fast writes

○ slow recovery

○ works well up to a couple of GiB

● Transactional use: for JMS messages and JTA logs

● Non-transactional use: Diagnostics and Config