| J avaOne 2003 | Session 2690 Performance Monitoring for the Java ™ Virtual Machine (JVM ™ ) Brian Doherty, Ben Kim, David Stoutamire J ava Performance E ngineering S un Microsystems, Inc.
| J avaO ne 2003 | S ess ion 2690
Performance Monitoring for the Java™ Virtual Machine (JVM™)
Brian Doherty, Ben Kim,David StoutamireJ ava P erformance E ngineeringS un Microsystems, Inc.
| J avaO ne 2003 | S ess ion 26902
Presentation Goal
L earn about non- intrus ive performance monitoring and the tools available for monitoring production systems utiliz ing the J ava ™ software s tack.
| J avaO ne 2003 | S ess ion 26903
Speaker’s Qualifications
• Brian, Ben, and D avid are all members ofthe J ava ™ P latform P erformance E ngineeringgroup at S un Microsystems
• We represent expertise in performance measurement, analys is , and tuning of industry s tandard benchmarks, customer applications, the J ava ™ class libraries , and the J avaHotS pot™ virtual machine
| J avaO ne 2003 | S ess ion 26904
Performance Measurement—A Product Lifecycle Process
D on’t s top monitoring your application just because it’s been deployed!
O bserve your production applications us ing minimally invas ive tools that are becoming better integrated across the software s tack.
| J avaO ne 2003 | S ess ion 26905
Agenda
• What is performance monitoring
• Monitoring J ava™ applications─ J ava™ software s tack─ D istributed applications
• D emonstration─ T ools─ A nalys is─ T uning
| J avaO ne 2003 | S ess ion 26906
Monitoring Goals
• Maintain and manage system health– D etection of performance events– D etection of application progress or liveness
• P erformance tuning– D rilling down to find causes– Identifying solutions
| J avaO ne 2003 | S ess ion 26907
Performance Management Process
Monitor
Start
Analysis
Design
Code
Deploy
ProfileNo
Yes
PerformanceOK
Benchmark
| J avaO ne 2003 | S ess ion 26908
Profiling Vs. Monitoring
• P roduction vs . D evelopment– P roduction systems have more restrictive
measurement needs– P roduction loads vary over time– P roduction loads difficult to s imulate
| J avaO ne 2003 | S ess ion 26909
Profiling Vs. Monitoring
Profiling MonitoringIntrus ion High Low
D etail High LowA ccuracy High LowR equired K nowledge High Low
| J avaO ne 2003 | S ess ion 269010
Complexities
Hardware
Operating System
Virtual Machine
Application Server
Application
T he J ava ™ S oftware S tack
| J avaO ne 2003 | S ess ion 269011
Complexities
C lientT ier
WebT ier
J 2E E ™ A pplicationS erver T ier
D atabaseT ier
T he D eployment A rchitecture
| J avaO ne 2003 | S ess ion 269012
Monitoring Tools
• Network management systems
• O S monitoring tools
• J V M monitoring tools
• J 2E E™ monitoring tools
Demo
| J avaO ne 2003 | S ess ion 269014
What We Saw
• S ymptoms─ E xcess ive memory utilization─ R estart
• T ools seen here─ R untime.freeMemory()─ gcore─ S erviceability A gent
• Identified source of memory consumption
| J avaO ne 2003 | S ess ion 269015
Other Tools
• P roduction─ - verbose:gc
• D evelopment─ - verbose:gc─ J V MP I/J V MD I profiling and debugging tools
Demo
| J avaO ne 2003 | S ess ion 269017
What We Saw
• S ymptoms─ Incons is tent response times
• T ools seen here─ V isualG C
• Identified explicit S ystem.gc() calls
| J avaO ne 2003 | S ess ion 269018
Other Tools
• P roduction─ - verbose:gc─ mpstat—identify s ingle threaded G C─ pstack—print O S thread dump─ S IG INT —print J ava thread dump
• D evelopment─ -X prof─ J V MP I/J V MD I profiling and debugging tools─ S erviceability A gent
Demo
| J avaO ne 2003 | S ess ion 269020
What We Saw
• S ymptoms─ P oor system utilization─ L ow throughput
• T ools used─ Y A X (perfmeter)
• C P U only ~50% utilized─ S IG INT (thread dump)
• T hreads waiting on locks• Identified source of lock contention
| J avaO ne 2003 | S ess ion 269021
Other Tools
• P roduction─ B ytecode instrumentation─ prstat / top─ pstack
• D evelopment─ -X prof─ J V MP I/J V MD I profiling and debugging tools
| J avaO ne 2003 | S ess ion 269022
What We Covered
Hardware
Operating System
Virtual Machine
Application Server
Application
Monitoring the J ava™ S oftware S tack
| J avaO ne 2003 | S ess ion 269023
What We Covered
• UNIX stat tools– vmstat, ios tat, mpstat,
nets tat ks tat• UNIX proc tools– pstack, pmap, prs tat/top,
truss/strace• Hardware counters– cpustat, busstat•Misc. tools– S E T oolkit
Monitoring the J ava™ S oftware S tack
Hardware
Operating System
Virtual Machine
Application Server
Application
| J avaO ne 2003 | S ess ion 269024
What We Covered
• jvmstat tools• C ommand line options– - verbose:[gc|class |jni]– - X prof– - X X options• S erviceability A gent• J S R 174 interfaces
Hardware
Operating System
Virtual Machine
Application Server
Application
Monitoring the J ava™ S oftware S tack
| J avaO ne 2003 | S ess ion 269025
What We Covered
• J MX interfaces– S NMP , C IM/WBE M• Bytecode instrumentation– V arious commercial tools• J V MP I, J V MD I tools– C ommercial and free
Hardware
Operating System
Virtual Machine
Application Server
Application
Monitoring the J ava™ S oftware S tack
| J avaO ne 2003 | S ess ion 269026
Summary
• P erformance measurement needs to be a product lifecycle process─ Not just a development process
• Non- intrus ive tools are critical for measurement of production applications
• T ools integrated across the software s tack ease the measurement and analys is process
| J avaO ne 2003 | S ess ion 269027
Resources
• HotS pot performance documentationhttp://java.sun.com/docs/performance
• jvmstathttp://www.sun.com/developers/coolstuff
• S olaris ™ performance documentationhttp://docs .sun.com
• J S R 174: Monitoring and Management S pecification for the J ava™ virtual machine
Q&A