The Five Problems Facing Business- Critical NFS Deployments In this webinar you will learn how to detect and overcome: ● Metadata Bottlenecks ● Rogue Clients & Noisy Neighbor issues ● Server/VM Latency issues ● Poor Write Performance ● Cluster Node Bottlenecks On Demand Webinar For audio playback and Q&A go to: http://bit.ly/5NFSProblems
30
Embed
Webinar: Five Problems Facing Business-Critical NFS Deployments
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
The Five Problems Facing Business-Critical NFS Deployments
In this webinar you will learn how to detect and overcome:● Metadata Bottlenecks● Rogue Clients & Noisy Neighbor issues● Server/VM Latency issues● Poor Write Performance● Cluster Node Bottlenecks
On Demand Webinar
For audio playback and Q&A go to:
http://bit.ly/5NFSProblems
● Analyst firm focused on storage, cloud and virtualization
● Knowledge of these markets is gained through product testing and interaction with end users and suppliers
● The results of this research can be found in the articles, videos, webinars, product analysis and case studies on our web site:http://storageswiss.com
Who Is Storage Switzerland?
Our SpeakersGeorge Crump is the founder of Storage Switzerland, the leading storage analyst focused on the subjects of big data, solid state storage, virtualization, cloud computing and data protection.
He is widely recognized for his articles, white papers, and videos on such current approaches as all-flash arrays, deduplication, SSDs, software-defined storage, backup appliances, and storage networking. He has over 25 years of experience designing storage solutions for data centers across the US.
Our SpeakersCTO John Gentry is responsible for being the voice of the customer and understanding the key IT infrastructure industry trends that affect product strategy and strategic alliances. With Virtual Instruments since 2009, John brings over 20 years of IT industry experience and has held a number of senior level sales, sales engineering and product marketing positions at industry leaders such as Qlogic, Borland, McData, and CNT. John earned his BS degree from the University of California at Santa Cruz.
Who Is Virtual Instruments?
Global Leader in Infrastructure Performance Analytics
• Founded in 2008
• HQ in San Jose, CA
• Global 2000 Customers
• Every Major Vertical
• 45 of the Fortune 100
• Merged with Load DynamiX in April 2016
ChangeImpactAnalysis
TechnologyEvaluation
ProductEvaluation
InfrastructureOptimization
ProductionPerformanceAnd AvailabilityManagement
Mission Critical NFS Use Cases
• Databases on NFS
• Virtualization on NFS
• NFS as a Front End to Object/Cloud Storage
• NFS for Performance Sensitive Unstructured Data
Polling Question
What % of Your Workloads are Running on NFS?
The Advantages Of Mission Critical NFS
• Simple to Manage
• More Granular control over Data (file vs. volume)
• In many cases performance neutral
Metadata Bottlenecks
• Metadata operations are at least 50% of NFS traffic, often 80-90%
• Several NFS solutions have the ability to move metadata traffic to flash, but this is often still insufficient
• Identifying a metadata performance issue is very difficult
• Typical workarounds
Scale
• As NFS inherits more workloads, workload variability becomes a problem, especially with virtualization
• More workloads means greater capacity consumption and more metadata
• Typical workaround is scale-out NAS
Scale-Out NAS Challenges
• Scale-out NAS is an interconnected set of servers called storage nodes
• The file system is typically striped across those nodes
• Most metadata handling is performed by one node in the cluster (bottleneck)
• The nodes must stay in sync - Extra network I/O
Rogue Clients & Noisy Neighbor Issues
• Rogue Clients/Noisy Neighbors can starve other resources
• Particularly difficult to identify IP based rogue clients/noisy neighbors
• Compounded by virtualization
• Typical workarounds
• Need real-time monitoring to assess latency issues
• Ability to measure SLA adherence is practically non-existent
Server/VM Latency Issues & SLA Adherence
• Large sequential files xfer rate a challenge
• One fix is to use ‘async’ at cost of potential data loss
• Another is to go from RAID 5 or 6 to RAID 10
• Or play with datasync and writesync, and transfer settings
Poor Write Performance
●Most IP Infrastructures Are not Optimized for Mission Critical NAS
○ Storage Traffic is Different
○ Typical Workarounds
■ Buy more hardware
■ Faster Network
■ Faster or Scale-Out NAS
■ Flash, Flash and more Flash
●Infrastructure Optimization Should Happen First!
Summary
MAXIMIZE AVAILABILITY
Identify & resolve problems before users are affected
Prevent & eliminate unplanned outages and slowdowns
OPTIMIZE COST
Match purchasing & deployment decisions to your application
workload I/O profiles
Maximize utilization of existing IT assets
GUARANTEE PERFORMANCE
Monitor and optimize infrastructure & workload performance
Accelerate & de-risk IT infrastructure changes and
transformations
The 3 Pillars of Virtual Instruments Value for IT
Making Applications & Infrastructure Perform Better Together
The VI Solution Architecture
Production storage Lab storage
SAN and NAS Performance
Probes
Virtual Server Probe
NTAP Storage Software
Probe
Network Switch Probe
Workload Generation Appliance
Workload Data Importer
TAP
Switch
Servers and VMs
VirtualWisdom Management Platform
WorkloadSensor
18
VirtualWisdom Entity Centric Model
Intelligent Topology
Case-based Alarms
Live Reports
Applied Analytics
Entity Centric view of Application Infrastructure
The New Virtual Wisdom NAS Performance Probe
• Full 10G line rate monitoring of NAS protocols for 16 concurrent ports in a single 2U device
• Initial support for NFSv3; software upgradability to SMBv3 and NFSv4 in 2017
• Workload and response time metrics captured for every read and write operation
• Provides performance, capacity and health info for every attached client and server
• Out-of-band, vendor-agnostic on the wire approach
• Enables unprecedented visibility to incoming requests by client to identify rogue clients
• Compatible with existing optical TAP / TAP Patch Panels
Metrics Analyzed by VW NAS Performance Probe
• Link metrics• Health, Utilization, SFP Diagnostics
• Flow Metrics for Commands• Procedure rates/counts, pending procedures• Response times, Avg payload, sum of
payload, …• RPC Statistics
• RPC counts, NLM counts, …• Hot file metrics
• Reported for top X files per interval• File size/path attributes, …
New NAS Performance Probe
• Released with VirtualWisdom 5.0
• New wide-screen dashboard with improved navigation and dark background
• Extremely customizable dashboard – here showing addition of VM probe metrics
ProbeNAS NFSv3 Performance & Flow Analysis
The VW NAS Probe allows you to understand NFS overall performance
ProbeNAS NFSv3 Performance & Flow Analysis
and by Client/Server flows.
Using VirtualWisdom Applied Analytics to find Root Causes
Balance Finder Trend Matcher
Event Advisor
Balance Finder automatically determines if the environment is balanced or imbalanced, and tracks indicators of any
change in the balance of an environment
Trend Matcher enables you to identify the probable source of a recognized event and the
other entities that might also be affected
Event Advisor lets you quickly determine if there are any trends or events that should be
investigated or noted across the entire environment
Data-informed prediction of resource needs. Learns from “seasonal” business patterns—whether a
season is hourly, daily, weekly, monthly, quarterly, yearly, etc.
Seasonal Trend Advisor
Using VirtualWisdom Applied Analytics to Suggest Changes
Queue Solver
Queue Solver examines actual historical host configuration settings (HBA queue depths) and performance data to provide recommendations to optimize the system-wide
performance
VM Coordinator
VM Coordinator allows you to see the optimal placement of your virtual machines across your cluster to eliminate over-
provisioning and unnecessary re-balancing
VM Deployment Advisor
Identifies the optimal cluster and host to deploy a VM, based on available capacity and expected VM workload across CPU, Memory, I/O and Network.
ProbeNAS Advanced Analytics Example
Event Advisor:
Lets you quickly determine if there are any trends or events that should be investigated or noted across the entire environment
Trend Matcher:
Automatically correlates events across entire set of relevant metrics to quickly determine root cause.
Customer Case Study: NAS Performance Probe (Financial Services Beta user)
Challenge NAS Performance Probe Solution1. On-going performance
problems for months that were not resolved even after a storage upgrade.
With VirtualWisdom, in a few hours, customer discovered the issue was a single rogue client issuing ~30,000 requests/sec, doing file based replication and scanning the file system. The VW Trend Matcher analytics found the client issuing the thousands of GetAttr procedures. Change in scheduling and frequency solved the problem
2. New users could not access the NFS storage once a max number of client sessions were reached.
The NFS storage would not accept additional client sessions once it reached its maximum. The customer used VirtualWisdom metric ‘Maximum Concurrent Total NFS Procedures’ with a time-based comparison; set up an alarm threshold for proactive notification at 80% of limit to avoid the problem.
3. Customer did not have a consistent way of resolving NFS issues.
With VirtualWisdom, customer was able to develop a report that any admin could use to investigate when a user complains. Type in IP address and see which mount point they’re having problems with.
Summary: The industry’s 1st real-time NAS Monitoring Solution