© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Evaluating performance for optimizing HP ArcSight ESM deployments Praki Prakash, Performance Team #HPProtect
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Evaluating performance for optimizing HP ArcSight ESM deployments Praki Prakash, Performance Team #HPProtect
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2
About me
Praki Prakash Software Engineer, HP ArcSight • Joined ArcSight in May 2014 • Performance Team Lead • Previous life
– VMware Inc. • Architect of CapacityIQ • Management Solutions
– Yahoo Inc. • Yahoo Personals Search
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3
Understanding ESM performance
About this presentation
ESM performance characteristics • Quantify hardware resource utilization across
various dimensions • Enable understanding ESM resource requirements
for your workloads • Provide a footing for resource planning for your
ArcSight installation
• Work in progress
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4
Multifaceted
ESM performance
Workload • Incoming events • Assets • Rule execution • Channels, Active lists, Session lists • Reports, Queries
Needs • Computing power • Memory • IOPs • Network bandwidth
ESM
Storage
Active Channels Reports
Correlation Events
Archive
Events
Events
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5
Building a performance model
Our approach
Key indicators • Events per second • Latency of user operations
Workload decomposition • Incoming events at various rates • Different package configurations • Simulated user activity (active channels, reports and queries) • Events per second processed • System resource utilization
Events
Resources
Users
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6
You gotta have tools!
Tools
PerfSight • Purpose-built tool for performance testing ESM and Logger • Virtual connectors to replay events captured from real environments • Simulated users exercising Active Channels, Reports, Queries • Data monitors for capturing EPS rate • Custom analytics scripts for result analysis
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7
System under test
ESM 6.5c CORRe • Running on HP ProLiant DL380p Gen8 • Intel(R) Xeon(R) CPU E5-2650 @ 2.00GHz Dual CPU, 8 Core/CPU • 64GB RAM DDR3, Synchronous Registered (Buffered), 1600 MHz • 1.6 TB, RAID 1+0, 6 SAS 600GB, 15K RPM • Gigabit Ethernet PCIe
Events • Events from captured from an existing ESM installation • Replayed at 10K, 15K , 20K, 25K, 30K, 35K and 40K EPS*
* EPS used for benchmarking under idealized conditions
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8
All quiet on the waterfront!
Idle resource utilization
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9
Events, no ArcSight foundation packages, no user activity
Resource utilization
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10
Events, Cisco monitoring, no user activity
Resource utilization
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11
Events, all foundation packages, PCI, no user activity
Resource utilization
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12
Installed packages
Resource utilization at 40K EPS
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13
Events, all foundation packages, PCI, no user activity
System metrics @ 40K EPS
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14
Events, all foundation packages, PCI, active channels
Active channel render times
Simulated user activity • Attach to 10 active channels at a time • Record channel count and time to render 100%
Resource utilization • CPU: 59% • Disk IOPs: 109 • Blocks transferred/s: 16,800 • Disk utilization: 16.1% • Network utilization: 22,300KB/s
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15
Events, all foundation packages, PCI, active channels
System metrics @ 40K EPS
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16
Events, all foundation packages, PCI, reports
Report execution
Simulated user activity • Four virtual report users • Running distinct subsets of reports
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17
Maximizing EPS
Packages / rules • Disable packages you don’t need • Consider turning off rules you don’t need • Dashboards/Shared/All Dashboards/ArcSight/Administration/ESM/System Health/Rules/Rules Status
– Partial Matches per Rule counts – Top Firing Rules – Rule Error Logs
• Custom Content • Performance implications of custom rules
Sizing • Assess your CPU, Memory and IO utilization and size them accordingly
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18
Optimizing your ESM deployment
Approaches to performance • Characterize your workload
– Event rate – Asset size – Solution packs or custom content – Typical usage
• Define your expectations – Response times – Spare capacity or headroom
• Size – Hardware to match
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19
For more information
Attend these sessions
• BOF3554, Tuning your ArcSight correlation engine
• TB3248, Syslog connector performance tuning
Visit these demos
• HP ArcSight ESM, DEMO3525
After the event
• Contact your sales rep
Your feedback is important to us. Please take a few minutes to complete the session survey.
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20
Please fill out a survey. Hand it to the door monitor on your way out.
Thank you for providing your feedback, which helps us enhance content for future events.
Session TT3129 Speaker Praki Prakash
Please give me your feedback
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you