Benchmarking OTM and Java - Is Your Platform Limiting Performance

Benchmarking OTM and Java:Is Your Platform Limiting

Performance?

OTM SIG User Conference ‘09

AgendaSetting ExpectationsOTM – Hardware and PlatformsTools – System and MonitoringBenchmarksHands-on Benchmarking (Real-time)Interpreting and Applying to OTMQ & A

This Presentation WILL:Teach you to benchmark various OTM platformsShow you how to interpret the benchmark resultsShow real-world platform results

This Presentation will NOT:Teach you to load-test an OTM installationShow you how to predict OTM SCA (scalability / clustering) performanceUtterly fail (I hope!)

Hardware / PlatformsCPU and Hardware Platform Matter! A LOT!CPU Speed – Not a Good Indicator of Performance

Other factors (cores, memory bandwidth, on-chip cache) necessitate benchmarking

OTM Requires both high multi-threading AND high single-thread performance

Lots of cores and high per-core performance

Performance of Current PlatformsLinux / x86-64Windows / x86-64 (note: memory limitations)SolarisHP-UX / PA-RISC

Note: HP-UX / Itanium currently unknown

AIX / POWER

Operating System / Stats – Helpful but LimitedReview system performance under production load for the previous 2 weeksUtilize System Tools to Monitor

sar / kSartop / prstat / topas / etc

Utilize Tools to TrendNagios / Munin / etc

Benchmarking – Which Benchmarks?

VolanoMarkDaCapoSoap StoneHammerora

Why?Java basedReplicate OTM behaviorEasily repeatable

Why Not Load-Test OTM?

Complicated to setupTime-consuming to runRequires OTM installed AND configuredNot Necessary to determine platform capacity

Benchmarking - VolanoMarkRunning VolanoMark (Hands-On Exercise)Full instructions at:

http://www.otmfaq.com/forums/blogs/chrisplough/6-benchmarking-part-1-java-jvm-performance-volanomark.html

Benchmarking - VolanoMarkVolanoMark

Java-based benchmark that simulates high transactional and multi-threaded loadReflects the performance of the following OTM activities

Web UI, Agents, Integration, General Workflow, General OTM Activities (not including optimization and planning based)

Higher numbers are better

Benchmarking - DaCapoRunning DaCapo (Hands-On Exercise)Full instructions at:

http://www.otmfaq.com/forums/blogs/chrisplough/12-benchmarking-part-3-java-jvm-performance-dacapo.html

Benchmarking - DaCapoDaCapo

Java-based benchmark that simulates highly computational, algorithmic, single-threaded processingReflects the performance of the following OTM activities

Optimization and Planning / Bulk Planning

Lower numbers are better

Benchmarking – Soap StoneRunning Soap Stone (Hands-On Exercise)Full instructions at:

http://www.otmfaq.com/forums/blogs/chrisplough/13-benchmarking-part-4-real-world-network-performance-soap-stone.html

Benchmarking – Soap StoneSoap Stone

Java-based benchmark that tests data throughput between servers and replicates application protocols, such as HTTP, RMI and RAW.Reflects the throughput and protocols utilized between the various OTM Tiers

Browser / Web: HTTPWeb / App: RMIApp / DB: RAW

Higher numbers are better

Benchmarking – HammeroraRunning Hammerora (Hands-On Exercise)Full instructions at:

http://www.otmfaq.com/forums/blogs/chrisplough/11-benchmarking-part-2-oracle-db-performance-hammerora.html

Benchmarking – Hammerora

Just Kidding!!!There’s no way we’d have enough time

Benchmarking – HammeroraHammerora

Benchmark based on the TPC-C and TPC-H benchmarks.Reflects the performance and scalability of the DB TierLower numbers are better

Interpreting the ResultsThese benchmarks have linear trends (twice as fast is twice as fast)VolanoMark

Higher is betterApplies to Web and App tiers

DaCapoLower is betterApplies to App tier

Soap StoneHigher is better40Mbit or faster Web App400Mbit or faster App DB

HammeroraHigher is betterApplies to all tiers

Online ResourcesPerformance

kSarhttp://ksar.atomique.net/

Nagioshttp://www.nagios.org/

Muninhttp://munin.projects.linpro.no/

BenchmarkingFull Replication Details

http://www.otmfaq.com/forums/blogs/chrisplough/

VolanoMarkhttp://www.volano.com/benchmarks.html

DaCapohttp://dacapobench.org/

Soap Stonehttp://soap-stone.sourceforge.net/

Hammerorahttp://hammerora.sourceforge.net/

Q & A and Discussion

Questions?

Thank You!

[email protected] x701

www.MavenWire.com