Lars Ejskjaer Greg Ferguson TR 1-470 Performance Analysis for Dummies.

Lars EjskjaerGreg Ferguson

TR 1-470

Performance Analysis for Dummies

2

Understanding Performance Analysis Can Help You Sell

− Gain better understanding of your customers environment

− Grow awareness to possible system bottleneck areas

− Understand changes that can enhance your customers overall satisfaction

Getting Started…

Simple Analysis (1, 2, 3)

Valuable Tools and Resources

Agenda

3

”My storage is slow…”

The Customer Problem

4

Guide the customer to best practices

Recommend solutions

− Administrative

− Product

Provide advice in the context of their environment

Your Goal

5

Understanding of how the customer’s environment was set up

Ability to identify missed best practices

− Administrative

− Performance

Ability to identify common performance issues

What You Will Need To Know

6

Getting Started

7

Easily Collect Performance Data - Perfstathttps://communities.netapp.com/docs/DOC-16209

Pro Tip:

Perform multiple smaller iterations versus one larger iteration for better visibility

8

Loading Data into LatX for Analysis https://latx.netapp.com/

9

Find and View the Perfstat

NetApp Confidential – Limited Use

10

Finding Specific Data

Pro Tip

Use PRESTATS iteration 1 for configuration information

Use POSTSTATS for performance measurements

11

Understand Configuration

− Aggregates

− Volumes

− Look for configuration errors

Analyze Performance

− System

− Disk

− Flash

Make Recommendations

Analysis Strategy

12

Disk Configuration

The Simple Analysis Part 1

13

Disk Types on the Controller

Pro Tip

Common disk types today: SAS, BSAS, MSATA

Prestats - sysconfig -r

14

RAID Groups

Pro Tip

Avoid unbalanced raid-groups in aggregates!

Poststats – statit

15

32-bit vs 64-bit Aggregates

Pro Tip

New features work best (require) with 64-bits aggregates Using one aggregate type makes the customer operations easier

Prestats – aggr status -v

16

Aggregate UtilizationPoststats - df –A -h

Pro Tip

An aggregate which is 80% full, should be monitored Aggregates over +90% full could impact performance

17

Aggregate Snapshot Copies Prestats - df –A –h, snap status -A & aggr status –v

NetApp Confidential – Limited Use

Pro Tip

Only SyncMirror and MetroCluster use aggregate snapshots

If not in use: - Remove schedule- Release space reservation

18

Volumes

The Simple Analysis Part 2

19

Volume Space UtilizationPoststats - df & lun stat –v all (or vol status –v)

Pro Tip

Databases like Oracle initialize their data files or data could be static so it could be fine that the volume is almost full

20

DeduplicationPoststats- stats perfstat_sis

Pro Tip

Savings less than 6-8% are just using resources without a real effect, unless data is static, turn it off and save resources!

21

Deduplication RuntimesPoststats - sis status -l

Pro Tip

Deduplication is a low priority process but it still occupies system resourcesWork to understand run times to smooth system scheduling for better performance

22

MisalignmentPoststats - nfsstat –d & lun stat –v all

Pro Tip

Understand what is in these files/LUNsLogfiles are often ”misaligned”Verify that virtual machines are aligned

23

System Performance

The Simple Analysis Part 3

24

ONTAP breaks work into groups of processes called domains

ONTAP schedules work across CPU cores as IT sees best

This can be seen in Sysstat –M

Detailed analysis of this is an advanced topic

What Are Domains In Data ONTAP?

25

CPU UtilizationPoststats - sysstatM.out

Pro Tip:

Average CPU utilization >70% depicts a very busy controllerCPU Utilization is a generally poor indicator of performanceLook at AVG CPU – Single CPU (thread) utilization is very informative

26

Writing Data to Disk - CP Type & TimePoststats – sysstat_1sec.out

Pro Tip

Deferred Back-to-back CP’s are performance killers (type #)- Data can’t get to disk fast enoughInvestigate (ignoring the CPU utilization):- Mis-alignment- Disk over commitedSolutions:- Move load to another controller/aggregate- Add disks/Flash Cache

27

Aggregate Disk UtilizationPoststats - statit

Pro Tip

Analyze performance impact based on drive utilization (by drive type)- SATA drives > 50% = busy- SAS drives > 60% = busyStatit will give clues about where to move load to

28

Active VolumesPerfsys Report

Pro Tip

Map volumes to aggregates (aggr status –v) Identify workloads to move

29

FlashCachePerfsys Report

Pro Tip

Verify that the system is benefiting from the use of FlashCacheUse PCS and perfstat to verify a customers gains by use of FlashCache

30

Latency in the EnvironmentPoststats - stats perfstat_cifs/_nfs/_fcp

Pro Tip

High latency = performance impactLatency requirements vary by applicatonAnalyze the workload and- Add Disk- Add FlashCache- Upgrade the Controllers

31

We reviewed

− High disk I/O’s

− Busy volumes

− Disk configuration issues

− Average CPU utilization

− Potential Flash Cache benefits

− Mis-alignments

Review of Findings

32

Resolve mis-alignments

Consider moving busy volumes

Add drives and reallocate to even out raid groups

Add Flash Cache

Upgrade Controllers

Add more Controllers within Clustered Data ONTAP

Recommendations

33

Don’t be afraid – performance is no longer such a mystery!

Now YOU Can Better Understand System Performance

Have FUN!

Most importantly, monitor disk utilization!

34

The Community site

− http://communities.netapp.com

ONTAP documentation (particularly ONTAP Command ref)

− http://support.netapp.com

Latx – your analysis tool

− http://latx.netapp.com

Important Resources

35

Complimentary Sessions

TR-1-123 Sizing, Designing, and Presenting a NetApp Solution

TR-2-717 Using NetApp Tech Tools to Create Winning Proposals and Tech Refreshes

TR-2-315 A Field Guide to Sizing - Part 1

TR-3-317 A Field Guide to Sizing – Part 2

36

Take an Insight Survey!

1) Click on the session number in your agenda. 2) Click on the Surveys Button. 3) Follow the prompts, complete the survey and submit!

Complete this survey by 7PM and be entered to win one of the following prizes: 1 iPad Mini 16GB Wifi 1 Bose SoundLink Mini Bluetooth 2 Jawbone Up Wristbands (Activity Tracker) 4 NetApp Signature Dry Zone Caps

NEW!

Went to a different session? Need a translated survey? Visit the main survey page in the mobile app to take a daily survey – available in English, Chinese, Japanese and Korean.

37

Facebook

www.facebook.com /NetAppInsightAmericas

www.facebook.com /NetAppInsightEMEA

Twitter

www.twitter.com / InsightAmericas

www.twitter.com / InsightEMEA

Tweet friends with #NTAPInsight

38

© 2013 NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, and Go further, faster, are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such.