Netapp Certified Storage Associate (NCSA)
Post on 15-Apr-2017
501 Views
Preview:
Transcript
ENG. Ahmed Gamil
NETAPP
4/13/2016
2016NCSA
Table of ContentsCourse Goal:................................................................................................................................................7
Introduction to Data Storage.......................................................................................................................7
Storage solution:.....................................................................................................................................7
Disk......................................................................................................................................................7
Disk array.............................................................................................................................................8
Just a bunch of disks (J-B-O-D) solution...............................................................................................8
Intelligent storage systems..................................................................................................................8
Storage Types:...........................................................................................................................................10
DAS........................................................................................................................................................10
Protocols............................................................................................................................................10
Advantages........................................................................................................................................10
Disadvantages....................................................................................................................................10
NAS........................................................................................................................................................11
Protocols............................................................................................................................................11
Advantages........................................................................................................................................11
SAN........................................................................................................................................................11
Protocols............................................................................................................................................11
Advantages........................................................................................................................................11
Disadvantages....................................................................................................................................11
Virtualization.............................................................................................................................................12
Server virtualization...............................................................................................................................12
Server virtualization benefits.............................................................................................................12
Storage virtualization.............................................................................................................................12
Storage virtualization benefits:..........................................................................................................12
Data Networks Fundamentals...................................................................................................................13
Small Computer System Interface (SCSI)...............................................................................................13
Internet SCSI (ISCSI)...............................................................................................................................13
Serial-Attached SCSI (SAS).....................................................................................................................13
Fiber Channel (FC).................................................................................................................................14
ENG. Ahmed Gamil1
FC topological implementations........................................................................................................14
Fiber Channel switched infrastructure..................................................................................................15
Zoning Types:.....................................................................................................................................15
Storage types.........................................................................................................................................16
Block storage.....................................................................................................................................16
File storage........................................................................................................................................16
Object storage...................................................................................................................................16
Some acronyms.........................................................................................................................................16
Traditional Storage................................................................................................................................16
Storage terms........................................................................................................................................16
Flash storage.............................................................................................................................................17
Flash storage contains:..........................................................................................................................18
Types of flash storage:...........................................................................................................................18
Solid State Technologies............................................................................................................................18
SSD Types:.............................................................................................................................................18
Flash Endurance........................................................................................................................................19
Endurance metrics:...............................................................................................................................19
Endurance levels (categories)................................................................................................................20
Flash translation layer...........................................................................................................................20
Write amplification (WA).......................................................................................................................20
Garbage collection (GC).........................................................................................................................21
Performance curve................................................................................................................................22
Fresh out of the box (FOB).................................................................................................................22
Write Cliff...........................................................................................................................................22
Steady State.......................................................................................................................................22
Over-provisioning..................................................................................................................................22
Major Industry Trends per NetApp............................................................................................................23
OnCommand Management Console.........................................................................................................24
System Manager GUI.............................................................................................................................24
Unified Manager GUI.............................................................................................................................25
Workflow Automation...........................................................................................................................25
Performance Manager...........................................................................................................................26
ENG. Ahmed Gamil2
OnCommand Insight..............................................................................................................................26
ONTAP OS..................................................................................................................................................26
Clustered Data ONTAP...........................................................................................................................27
ONTAP OS support.............................................................................................................................27
Data ONTAP 7-Mode.................................................................................................................................28
NetApp Hardware Basics...........................................................................................................................28
Slots and ports.......................................................................................................................................29
Slots...................................................................................................................................................29
Ports..................................................................................................................................................29
Disk shelves...........................................................................................................................................30
Disk classification...............................................................................................................................30
Disk identification system..................................................................................................................31
Disk ownership..................................................................................................................................31
Internal components.............................................................................................................................32
Flash Cache 2.....................................................................................................................................32
FAS 2552 model.........................................................................................................................................32
Another Model......................................................................................................................................33
Using the CLI..............................................................................................................................................33
Date.......................................................................................................................................................33
Version..................................................................................................................................................33
Help “?”.................................................................................................................................................33
System console......................................................................................................................................34
Man page...............................................................................................................................................34
Create volume.......................................................................................................................................34
NETAPP technology...................................................................................................................................34
Disks......................................................................................................................................................36
Aggregates (RAID options).....................................................................................................................36
Aggregates structure:........................................................................................................................36
Snaplock............................................................................................................................................37
Creating Aggregates...........................................................................................................................37
Flash Pools.............................................................................................................................................40
Activate flash pools............................................................................................................................40
ENG. Ahmed Gamil3
Flash Pools support............................................................................................................................40
Flash pool Limitation.........................................................................................................................41
FlexArray................................................................................................................................................41
Volume..................................................................................................................................................42
Volume modes...................................................................................................................................42
Volume types:....................................................................................................................................42
Qtrees........................................................................................................................................................47
Create Qtree..........................................................................................................................................48
Calculating Usable Disk Space...................................................................................................................49
Review.......................................................................................................................................................50
SnapShot...................................................................................................................................................50
SnapRestore..............................................................................................................................................52
SnapMirror................................................................................................................................................52
SnapVault..................................................................................................................................................53
Network Administration............................................................................................................................53
Name resolution....................................................................................................................................53
Create the VLANs...................................................................................................................................55
Create VIF..............................................................................................................................................57
Interface groups / aggregation..........................................................................................................57
Using CLI............................................................................................................................................60
Creating LUNs............................................................................................................................................62
Create LUN GUI......................................................................................................................................62
Delete LUN GUI......................................................................................................................................66
Create LUN CLI.......................................................................................................................................67
Using SnapDrive........................................................................................................................................68
NFS Exports and CIFS Shares.....................................................................................................................68
Delete Export.........................................................................................................................................69
Create Export.........................................................................................................................................69
Create CIFS share...................................................................................................................................71
Stop Sharing..........................................................................................................................................73
Using BranchCache....................................................................................................................................73
Distributed caching................................................................................................................................74
ENG. Ahmed Gamil4
Hosted Casing........................................................................................................................................74
Branch cache configurations.................................................................................................................74
Using Quotas.............................................................................................................................................75
Create Qtree..........................................................................................................................................76
Set Quota...............................................................................................................................................76
RBAC..........................................................................................................................................................80
Create User............................................................................................................................................80
Review...................................................................................................................................................81
Storage System Maintenance....................................................................................................................81
Auto support.........................................................................................................................................81
Logs.......................................................................................................................................................82
Degraded disk........................................................................................................................................82
Disk scrubbing.......................................................................................................................................82
sysstat....................................................................................................................................................82
Storage Space Management......................................................................................................................83
Provisioning...........................................................................................................................................83
Space allocation.....................................................................................................................................84
CLI..........................................................................................................................................................84
Storage Efficiency (Deduplication).........................................................................................................85
Clustered Data ONTAP Concepts...............................................................................................................86
Clustered Data ONTAP User Interfaces......................................................................................................87
Clustered Volumes and Namespaces.........................................................................................................87
Namespace............................................................................................................................................88
Junction.............................................................................................................................................88
Infinite volume......................................................................................................................................88
Clustered File Access.................................................................................................................................89
NFSv4.....................................................................................................................................................89
PNFS (Parallel NFS)................................................................................................................................89
LIF......................................................................................................................................................89
Common Internet File System (CIFS).....................................................................................................90
Netapp Networks......................................................................................................................................90
Clustered Load Balancing..........................................................................................................................91
ENG. Ahmed Gamil5
Load Sharing Mirrors (LSM)...................................................................................................................91
SAN Infrastructure.....................................................................................................................................91
Unified storage......................................................................................................................................92
Steps for the establishment of the SAN:................................................................................................92
FC Connectivity..........................................................................................................................................92
Fibre Channel connection:.....................................................................................................................93
Commands.............................................................................................................................................93
sysconfig command...........................................................................................................................93
fcadmin command.............................................................................................................................93
License command..............................................................................................................................94
To view and add license................................................................................................................94
References.................................................................................................................................................95
NCSA
NetApp has a lot of great products:
unified storage
high performance san storage
enterprise all-flash storage
Course Goal:Understand key storage topics
Understand the NetApp product portfolio
Describe and configure key NetApp elements
ENG. Ahmed Gamil6
Introduction to Data StorageStorage solution:Disk
Is a hard disk that would be a spinning platter
Manufacturers would make the platter spin faster and faster and faster “RPM speeds”
It cause latency due to mechanical parts.
Solid state drive is even more efficient than the mechanical spinning disk but it is still a traditional type of disks
Disk array
Can do redundant array of inexpensive disks inside the array
It Contains:
Firmware of the array
RAID
Power supply
Fans for cooling
ENG. Ahmed Gamil7
Just a bunch of disks (J-B-O-D) solution
Where we have disks out there that we just keep adding to the network and then the operating system
Each disk works as an independent disk in the array
Intelligent storage systems
A server that needs to access the data. And it does so through a storage area network (SAN)
Then we connect to this storage system
Storage system component:
front end
cache : for fast access to data since data that's frequently accessed will be cached
back end
actual disks
Factors before purchase:
Applications :
Like Email, DB, etc.
How much data do these application store
How quickly does the application able to retrieve data from the store
ENG. Ahmed Gamil8
What about the requirements like latency?
Data protection :
Like back up and disaster recovery
Availability
How much downtime can we survive with?
How much is acceptable? 99.999 is acceptable
Availability could be accomplished by redundancy and disaster recovery systems
Security
Scalability
How fast, how easily your storage infrastructure can grow as you need it to
Performance
The throughput of a system, the response time, the capacity, the reliability of a system
Storage Types:DAS : direct-attached storage
NAS : network-attached storage
SAN : storage area networks
ENG. Ahmed Gamil9
DASIs an external disk system or an internal disk system “Blade Servers”
Storage appeared as a block storage which can be formatted by OS
ProtocolsATA, SATA, eSATA, SCSI, SAS, USB, USB 3.0, and Fiber Channel.
Advantages Not complicated (storage network between the device and the storage itself)
Affordable (good price)
DisadvantagesLack of scalability
NASIt is storage attached to the network (Server must connect through a network)
NAS contains filesystem, CPU and resources
Storage appeared on the remote server as a shared folder
ProtocolsNetwork file system (NFS), or common internet file system (CIFS)
ENG. Ahmed Gamil10
Advantages More scalable (easy to build out the amount of storage that's connected to the LAN)
Servers worry only about its applications only not storing data to NAS
Easy to install, deploy and then manage.
Cost effective solution
Nice utilization rate
SANIt is storage attached to the network (Server must connect through a network)
Storage appeared as a block storage which can be formatted by OS
ProtocolsFiber Channel, Fiber Channel over Ethernet, or iSCSI
FCOE: 10-gigabit-per-second Ethernet
AdvantagesRedundancy “redundant NICs and RAID”
Highest performance
Most reliable infrastructure
DisadvantagesThe most complex
Virtualization Server virtualizationVMware make server virtualization and what it is called “Server sprawl”
Server sprawl: the need of more and more storage “server expansion”
Server virtualization benefits buy fewer servers
better utilization with high flexibility
ENG. Ahmed Gamil11
lower operating costs (like power, human resource)
lower management costs (easy to administer)
support more technologies like “ VMotion, backup and snapshotting technology ”
Storage virtualizationWe can add as much physical storage as we want and it can be located anywhere.
Then we will represent it to the end users as one logical entity.
End user don’t know the location of the physical storage
NetApp tries to do for storage what VMware has done for servers (Storage sprawl)
Storage sprawl: the need of more and more storage “storage expansion”
Storage virtualization benefits:Resources are pooled
Support multiple protocols
High storage utilization
Create products that lower the total cost of ownership (TCO) when it comes to provisioning and maintaining storage for the data center
RAID-DP which is It is a RAID technology that is efficient like RAID-10 but cheap like RAID-5
Snapshot copies which is like a backup with smallest window
reduced space for the backup process and near instantaneous restoration of data
deduplication : in case of many VM uses same files, it will use only one copy for all VMs instead of create a copy for each VM
Data Networks FundamentalsWe will talking about cabling and protocols that are going to run over that cabling inside SAN
SCSI : Small Computer System Interface
ISCSI : internet SCSI
SAS : Serial-Attached SCSI
FC : fiber channel
ENG. Ahmed Gamil12
Small Computer System Interface (SCSI)
Hook up like printers and scanners to devices.
The SCSI system is a client server type of system (initiator - client)
Internet SCSI (ISCSI)Used to run SCSI protocol commands for moving data to and from storage over IP networks
Uses Ethernet switches
Less expensive than FC
Support longer distances
Client server type of system (initiator - client)
Serial-Attached SCSI (SAS)Used to run SCSI protocol commands over a serial cable
Simple and inexpensive
Very limited (distance = 8-meter, number of devices that we can attach to the SAS network)
Fiber Channel (FC)Flexible
Reliable
Scalable
Optical or copper: copper and optics can be utilized for Fiber Channel as well
Supercomputers
ENG. Ahmed Gamil13
FC topological implementations
Fiber Channel point-to-pointConnect host to storage directly using HBA
Fiber Channel Arbitrated LoopDevices being connected in like a ring
Fiber Channel switched infrastructureDevice connects to two independent storage area networks
If one of the SAN completely fails, the other redundant fabric can still carry the information.
ENG. Ahmed Gamil14
Fiber Channel switched infrastructure
What is zoning?
Zoning means what devices in the fabric can communicate with what other devices.
H1 can access LUN2 of S1 and LUN0 of S2
H2 can access LUN1 of S2 and LUN1 and LUN 3 of S1
Zoning Types:Hard zoning
Soft zoning
Hard zoning Assign a certain switch port number to a particular HBA. If the port is down, so we need to assign another port
Soft zoningIf this HBA with WWPN is connected to any switch port, it will add this HBA to a certain zone
Storage typesBlock storage
File storage Store things in terms of files on these devices
Object storageIdentify each thing stored as an object
It allows object re-usability (the object could be available for many server)
ENG. Ahmed Gamil15
Some acronyms
High Performance Computing (HPC): amounts of memory “99.999%” and storage capacity
Traditional StorageCombination between flash and mechanical storage
It provide low latency and low cost
Storage terms
Seek time (rotational disk)
The amount of time it takes to move that mechanical head into the correct position over the platter.
Rotational latency (rotational disk)
The amount of time that it takes to spin the correct sector on the platter into the right position
Transfer time (flash / rotational disk)
The amount of time it takes to actually read information from the platter or write information to the platter
Overhead (flash / rotational disk)
How much disk logic stuff has to go on? And what does that do to our latency?
Longevity/Endurance (flash disk)
A lifetime of the flash media that it can endure this process of block writes over the top of other data.
Pages: (flash disk)
It could be small container of data (512 bytes) or large (8 kilobytes)
Erase Block (flash disk)
Container of memory pages (32 to 128 pages) that can be buffered
ENG. Ahmed Gamil16
Flash storageIs non-volatile storage and it is re-writable memory.
Require erasing blocks of data before they can be written to
Supports only a finite number of rights and that number is vary by the technology we use
SSD drive mean flash drive which provide
High performance and better than mechanical disk drive due to there is no moving parts
lower write performance than a read performance
Electricity is utilized to store and retrieve data storage very quickly
Low latency
High cost/GB
Flash storage contains:Memory unit (DRAM): Dynamic holding location for the data
Access controller (Flash controller):
Manage the placement and access of data into the flash storage system
Move data between the DRAM and flash memory chip
Flash memory chip: The actual storage of information
ENG. Ahmed Gamil17
Types of flash storage:
Flash memory
SD card
SSD drive
Flash based PCI card
Solid State TechnologiesSolid state drive, or SSD is often used in enterprises today
Uses non-moving flash memory technology
Increase the reliability of the particular storage media. (Eliminating the moving mechanical parts)
Flash memory is non-volatile and it's re-writable memory
SSD Types:Negative AND (NAND) SSDs:
Commonly used because it's more durable, it's less expensive.
Has denser cells, and the write erase operations are faster than NOR SSD
Negative OR (NOR) SSDs:
Designed to store the binary code of programs
Has high performance in read operations
Single level cell Flash technology (SLC)
ENG. Ahmed Gamil18
Store one bit per cell
Has high endurance
Multi-level cell Flash technology (MLC)
Store two bits per cell.
Provide higher capacity
Approximately 1/10 the endurance all of single level cell Flash memory.
3-bit per cell and even 4-bit per cell technologies
High capacity
Very limited number of writes to the memory.
Flash EnduranceQuality of the flash storage depending on:
How much data we are able to write to that flash media over its lifetime?
How long will the flash storage last us (life time)?
Endurance metrics:Total bytes written (TBW)/ data written per day.
Program/erase cycles that are capable for the particular flash media (P/E value)
We can categorize endurance based on:
What you need to store?
How much manipulation of that data needs to be done on an ongoing basis?
E.g.
Write intensive storage:
High endurance: 25 plus data writes per day
Medium endurance: 10 data rights per day
Read intensive storage:
One to three data rights per day.
ENG. Ahmed Gamil19
Endurance levels (categories)We can categorize the endurance in order to balance between the flash technology and its cost
Single level cell NAND technology very high endurance very cost
Multi-level cell NAND + wear leveling provide balance between affordability and endurance
Over-provisioning there is a wasted capacity old technique (we will discuss)
Flash translation layer
FTL is a layer that introduce wear leveling techniques in order to improve the endurance of the media
FTL is a layer is located in the SW layer between the file system part and HW level
Write amplification (WA) Flash memory must be erased before it can store new data
Data cannot be overwritten directly as it is in a hard disk drive,
In order to rewrite the data directly to the flash disk, Flash make a process called Write amplification
Write amplification (WA) move user data and metadata more than once to the buffer then to the media then to the buffer then to the media again (read, updated and written to a new location)
ENG. Ahmed Gamil20
SSDs use a process called garbage collection (GC) to overcome this behavior
Garbage collection (GC)Garbage collection is a big part of write amplification on the SSD
Data is written to the flash memory in units called pages (Multiple cells)
The memory can be erased block by block, each block is made up of multiple pages
SSDs use garbage collection to reclaim the space by the flowing steps
You write a data to pages (A-D) in Block “X”
Then write new data in new pages (E-G)
If you need to replace old (A-D) pages with new (A’-D’) pages, you will write new pages normally to the same block “X”
Then read and rewrite pages (E-D’) from block “X” to Block “Y”
Then Block “X” is considered as empty and available to use
All SSDs include some level of garbage collection, but they differ in when and how fast they perform the process
ENG. Ahmed Gamil21
Performance curveIt’s a graph of performance over your gigabits that are being stored
Fresh out of the box (FOB)Describe a flash memory device that has few or no program/erase (P/E) cycles since the device was manufactured
New data written to the flash directly before program/erase (P/E) cycles
Write CliffPerformance of an SSD drive drops dramatically
It occurs when all of the cells within an individual drive have been written to at least once.
So any incoming data is buffered until the cell “flashed” and the new data re-written back to the cell ((P/E) cycles)
Steady StateIt is the state of the SSD over its life time
Over-provisioningIt is a technique that's used with a solid state drive.
It you bought SSD with reported capacity 1TB, the actual capacity is more than 1TB (over-provisioned)
This additional space is used as over-provisioned storage space for wear leveling techniques, buffering and garbage collection and any other techniques that can improve the endurance performance
ENG. Ahmed Gamil22
Major Industry Trends per NetAppFlash-based storage technologies
If you're implementing flash smartly, it will lead to lower costs.
Cloud-based technologies
Reduce energy cost, space saving, enhance services
Software-defined data centers
An extension of cloud-based technologies
It is based on “ONTAP software” which creates a storage virtual machine (SVM) from fabric-attached storage (FAS) and provide logical views of that hardware for our end users
Converged infrastructures
Easier to manage
Cost savings
Example for converged infrastructure technology:
Fiber Channel over Ethernet (FCOE) to carry 10 gbps Ethernet.
LAN technology that's utilizing Ethernet (eth) + SAN technology using Fiber Channel (FC)
Mobility
Is to be able to access the data regardless its location or size or the device we're using
big data
Term for these huge data sets that is difficult to process using traditional applications
ENG. Ahmed Gamil23
OnCommand Management ConsoleNetApp has its own operating system which called ONTAP operating system
ONTAP operating system can be managed by GUI or CLI
NetApp named their graphical user interface stuff OnCommand
OnCommand is the umbrella software for a bunch of packages
System Manager GUI
This is a day-to-day management and monitoring tool
Interacting with the clustered Data ONTAP operating system.
Manage your particular system
Manipulating the volumes
Manipulating the shares
Getting a snapshot of the usage of a particular device.
ENG. Ahmed Gamil24
Unified Manager GUI
Monitoring and alerting tool
Give a dashboard to get any required information about the system
Support 3rd party APIs to provide multi-vendor dashboard
Workflow Automation
Allows to create a workflow to automate a process like (create a basic cluster mode volume)
Workflow Automation permits
ENG. Ahmed Gamil25
Performance Manager
Biggest level of monitoring in detail
OnCommand Insight
It is a tool that designed for hybrid cloud environments
Monitor and manage hybrid cloud
ONTAP OS ONTAP OS has 2 different versions:
Clustered Data ONTAP
Data ONTAP 7-mode
ENG. Ahmed Gamil26
Clustered Data ONTAPOpen Network Technology for Appliance Products (ONTAP)
It is an operating system from NetApp, for the management, maintenance, initial set-up, and configuration
ONTAP use RAID Dual Parity (RAID-DP) with 2 parity disks
ONTAP operating system goal:
to be able to do high availability (in case of maintenance or upgrade)
to be efficient (apply compression to save capacity, thin replication of data, data deduplication to get rid of redundant copies )
scalability (Through a clustered solution that's virtualized with our storage virtual machines)
ONTAP OS supportUnified Storage Architecture (USA):
NFS, CIFS, Fiber Channel, iSCSI, Fiber Channel over Ethernet,
Unify Fabric Attached Storage:
SAS, SATA, Fiber Channel, SSD
Storage Virtual Machines (SVM)
It is a logical representations of the storage that might have a lot of physical hardware.
Its own Logical interfaces, logical volumes and LUNs
Non-volatile RAM (NV-RAM):
There's a battery connected to this RAM so that we can preserve the data in case of a power failure
Write Anywhere File Layout File System (WAFL)
Is a file layout (like file system) that supports large, high-performance RAID arrays
ENG. Ahmed Gamil27
In case of a crash or power failure, it will restart without long consistency checks
Snapshots
Is a pointer for the original data
Data ONTAP 7-ModeData ONTAP 7-Mode Support
Multi-protocol capabilities (NFS, CIFS, Fiber Channel, iSCSI, Fiber Channel over Ethernet)
NAS/SAN
RAID-DP
Snapshot
Improve Compression
Improve Encryption
64-bit aggregation
Move the running 32-bit aggregates, to a 64-bit aggregation non-disruptively.
IPv6
Flash Pools: we can add Flash to the system, and the Data ONTAP will use it for caching
BranchCache: provide caching features to company branches which has slow WAN
Data ONTAP 7-Mode contains:
Disk Blade (D-Blade):
to manage WAFL, RAID and Storage (SAS, NFS, FC)
Management host (M-host) :
Manage nodes by using
Command line interface “ng shell” or “ngsh” to manage the nodes
Element Manager User interface from the browser
NetApp Hardware BasicsLet’s talk about some of the basic hardware components we should be familiar with
ENG. Ahmed Gamil28
All hardware components can be break down into three general categories.
slots and ports
disk shelves
internal components
Slots and ports
Slots Used to plug stuff, taking ports out, putting new ports in or configuring what the port will do in the software
PortsManagement Port “e0M” Connected to a separate network (management network)
Remote LAN Module (RLM) interface
We can use this port to manage the node remotely, even if the NetApp device itself is down or ONTAP OS is down
It provides remote access, monitoring, troubleshooting, logging, and alerting features
We can interact with RLM using:
ENG. Ahmed Gamil29
SSH (default)
Telnet (disabled by default)
Remote Shell (disabled by default)
Data ports “e0a, e0b,…..” Connected to data network
Data interfaces support 802.3ad technology to aggregate links together
So you can aggregate “e0a” and “e0b” to “group1” to take advantage of link aggregation (high performance and availability)
Unified port/Unified target adapter (UTA2)Is the port that can act as Fiber Channel or 10-gigabit Ethernet port
Disk shelvesThe physical disks where we're storing information
We can put different drives in, and going from SSD technology to mechanical back to SSD technology again
Clients see volumes or aggregates instead of physical disks
Physical disks support Fiber Channel, ATA, Serial ATA, SAS, and SSD
FC and ATA use the Fiber Channel Arbitrated Loop (FC-AL) connection-type
SATA, SAS, and SSD use a SAS connection-type
Disk aggregation and RAID improve the performance and availability of the disk
ENG. Ahmed Gamil30
Disk classification Data disk: contain all data
Hot Spare disk: used in case of a disk failure
Parity disk: to store the parity information
Disk identification systemThe disk identification name is based on its location in the NetApp appliance
The disk location is based on “slot no.”, “port no.” and “device_ID”
For disks that used FC-AL: the location is based on “shelf identifier”, “bay number” and “device ID”
Disk ownershipDisk ownership is controlled by a controller and disk pool
For single system set up
There is one controller that is in charge of that disk,
There is only one pool (pool 0) that the disk is a part of
For high availability set up
We have 2 controllers and 2 pools
Disk ownership is controlled by software settings (system manager)
ENG. Ahmed Gamil31
Internal componentsLike CPU, RAM, non-volatile RAM and Flash Cache 2.
Flash Cache 2
It is a technology that allow the mechanical drives that can act as SSD performance by caching the frequently information in a Flash Cache on the device
It provide lower cost per TB, less power, more capacity storage, more IOPS and improve response time
FAS 2552 model
ENG. Ahmed Gamil32
Another Model
Using the CLIDate
VersionFor firmware version
Help “?”To get help bout the available commands
To get help about a certain command
ENG. Ahmed Gamil33
System consoleTo go to the system console
CTRL + D to exit system console
Man page
Then “q” to quit
Create volumeTo create new volume with name “my_cli_test_ncsa” and size “2g” on aggregate “aggr0”
NETAPP technology
ENG. Ahmed Gamil34
Storage
Disks
Aggregates (RAID options)
Volumes (FlexVol and Traditional)
FlexCache
FlexClone
Deduplication
QTrees
CIFS Oplocks
Security styles
Quotas
aggregate snapshotting
data compression
sync mirroring
snap blocking
metro clustering
FlexArray
ENG. Ahmed Gamil35
DisksThis is the physical disk reside in a disk enclosure
Disk has a pathname like 2a.17, where 2a = SCSI adapter and 17 = disk SCSI ID
Spare will be used in any group or any aggregate to replace failed disks.
There are 4 types of disks:
Data : holds data stored within the RAID group
Spare: Does not hold usable data but is available to be added to a RAID group in an aggregate, also known as a hot spare
Parity: Store data reconstruction information within the RAID group
dParity: Stores double-parity information within the RAID group, if RAID-DP is enabled
Aggregates (RAID options)An aggregate is a component in NetApp that contains disks, RAID groups, etc.
Disks are grouped together in aggregates to provide storage volumes
All disks in the same aggregate have to be same type
Each aggregate has its own RAID configuration, and disks and luns
When using RAID4 or RAID-DP the largest disks will be used as the parity disk/s
You can create traditional volumes or NetApp's FlexVol volumes
We are using aggregate to create volumes, specifically “FlexVols” which can be used by the end user
To create the aggregate we will:
give aggregate a name
type of RAID
define a size
select no. of disks
Aggregates structure: 32-bit: low performance, limited size (16 terabytes)
64-bit: support up to 100 terabytes
ENG. Ahmed Gamil36
32-bit and 64-bit aggregates can co-exist now on the same device.
Fewer aggregates reduces the overall management workload
SnaplockIt is a technology to write once, read many" (WORM) data
Administer non-rewritable storage of data
Creating AggregatesUsing system manager:
ENG. Ahmed Gamil37
ENG. Ahmed Gamil38
ENG. Ahmed Gamil39
Flash PoolsIt is a cash memory storage for NETAPP appliances
Flash pools used to create a flash cash which contains the frequently accessed data
The controller is ensuring that frequently accessed data is put on the flash, and less frequently accessed data put on the mechanical disk drive
Flash Pools are differ from flash cache,
Flash cache is used for read operation
Flash pools are used for read, write, metadata operations
ENG. Ahmed Gamil40
Fusion Drive is a type of drives that is contains a mechanical drive included with flash
Activate flash pools From CLI:
select our aggregate
run command “aggregate options hybrid enabled on”
Flash Pools supportSnapMirror
aggregate snapshotting
data compression
sync mirroring
snap blocking
metro clustering,
Flash pool LimitationFlash Pool aggregates cannot be used in the following configurations:
flash pool is not supported on ONTAP7.2 and earlier
32-bit aggregates
Aggregates composed of array LUNs
SnapLock aggregates
Traditional volumes
A storage system that uses Storage Encryption
FlexShare is not supported for volumes associated with Flash Pool aggregates.
Read-only volumes, such as SnapMirror destinations, are not cached in the Flash Pool cache.
FlexArrayV-Series runs behind FAS arrays and other storage from other major vendors.
FlexArray is a software that will replace NetApp's V-Series (virtualization gateways).
FlexArray is built into Ontap. The first version supports NetApp FAS, EMC and Hitachi storage.
ENG. Ahmed Gamil41
FlexArray also supports NetApp's E-Series high-performance arrays
Aggrigate will use RAID type “Hyprid”
Volume Volumes contain file systems that hold user data
User can access the data using NFS, CIFS, HTTP, FTP, FC, and iSCSI.
Volume modesOnline Read and write access to this volume is allowed.
Restricted Some operations, such as parity reconstruction, are allowed, but data access is not allowed.
Offline No access to the volume is allowed.
Volume types:Traditional volume
FlexVol
Flex clone
Flex cache
Root volume
Traditional volumeTraditional volume tightly tied to the aggregate
ENG. Ahmed Gamil42
No other volumes can get their storage from this aggregate.
To increase the traditional volume, we have to increase the size of the aggregate by adding physical disks.
MAX traditional volume size is 16 TB
Traditional volume cannot be decreased/shrink
The smallest traditional volume contains two disks (for RAID4) or three disks (for RAID-DP)
It is available in 32bit only
Traditional Volumes cannot be cloned, but we can create a copy of the volume with space equivalent to the amount of the original storage space
FlexVol Volume (flexible volume)It is a volume that is not depend on the aggregate:
We can decoupling the flex volume from the aggregate
Every FlexVol can be administer independently, grow independently, set up particular features independently
Choose independently the flexible volumes that are sharing the aggregate.
You can increase or decrease the size of FlexVol volumes online
We can clone the volume (FlexClone)
As mentioned in the image,
Each set of Hard Disks grouped in a RAID groups
All RAID groups aggregated to one aggregate.
Then, we can divide the aggregate to “flex volumes”,
Each flex volume can be used as a lun or NFS volume,
5% of the aggregate capacity reserved for snapshot
ENG. Ahmed Gamil43
20% of the flexvol size reserved for snapshot
0.5% of the flexvol size reserved for volume metadata (space guarantee)
There are two types of FlexVolume
32-bit
64-bit
To use Data ONTAP to move data between a 32-bit volume and a 64-bit volume, you must use “ndmpcopy” or “qtree SnapMirror”. (vol copy command or volume SnapMirror cannot be used)
Create FlixVol volumeCreating FlexVol requires:
Name
Type of FlexVol (NAS/SAN)
Related aggregate
The size of the FlexVol
Snapshot reserve: what will be reserved in that FlexVol for snapshotting technology? 20%
Space guarantee: free space in the aggregate for creating FlexVol, if it set to none, then at a certain time, users couldn’t write to the FlexVol, and the aggregate could run out of space.
ENG. Ahmed Gamil44
FlexClone volumesCopies (point-in-time copies) of a parent flex volume.
FlexVol use its snapshot copy to create FlexClone,
ENG. Ahmed Gamil45
FlexVol is Writable volumes
We can manage them separately as a regular FlexVol volume
After create FlexClone, any changes in the parent FlexVol will not reflected in the FlexClone volume.
Any LUNs present in the parent volume are present in the FlexClone volume but are unmapped and offline.
FlexClone volumes always exist in the same aggregate as their parent volumes.
FlexClone volumes and parent volumes share the same disk space for any common data (no additional disk space) until changes are made to the FlexClone volume or its parent
We can Split the FlexClone volume connection with the parent volume, so the FlexClone volume will use its own disk space instead of sharing it with its parent
FlexClone volumes also can be cloned
The space guarantee and space reservation of the FlexClone volume is created as its parent if the aggregate has enough space
Root volumeEvery aggregate have root volume
Root Volume contains system files, log files, core files, home directory and /etc directory
We can use system manager and the CLI in order to interact with those files
Root volume contain files that can be edit in order to change NetApp system behaviors.
FlexCache VolumesIt is a local volume (in our storage) that is used to cache data for other remote volume (remote storage) to speeds up data access to remote data
When a client requests data from the FlexCache volume, the data is read from the origin system and cached on the FlexCache volume, this data is then served directly from the local FlexCache volume.
ENG. Ahmed Gamil46
FlexShare:Is a features that applied to the volumes to provide:
Volume Priority: A higher priority gives a volume a greater percentage of available resources when a system is fully loaded.
User versus system priority : Prioritize user workloads (application and end-user traffic) versus system work (backup, replication, and so on)
Cache utilization : Optimizing cache usage by Configuring the cache to retain data in cache or reuse the cache depending on workload characteristics
QtreesQtrees enable you to partition your volumes into smaller segments that you can manage individually
Qtrees used to organize the data and apply the following:
Quotas: to limit the size of the data
Backup: to keep backups more modular, flexible backup schedules
Security style: for Windows files and applications, you can group the data in a qtree and set its security style to NTFS
CIFS oplocks (opportunistic locks): opportunistic locks is a database concept that may require to be “OFF” or “ON”, So you can assign Qtree to a certain type
We can't do snapshot copies of a Qtree,
ENG. Ahmed Gamil47
We can't do space reservations or guarantees within a Qtree structure.
Create Qtree
ENG. Ahmed Gamil48
Calculating Usable Disk SpaceIn earlier versions of the ONTAP software,
When NetApp prepare a physical disk, it reserve:
10% for the WAFL file system and 90% for aggregate
5% of the aggregate space was reserved for the snapshot copy
We create our FlexVol inside the aggregate, 20% of the FlexVol is reserved for snapshot
In new versions
10% for the WAFL file system and 90% for aggregate
0% of the aggregate space was reserved for the snapshot copy but it is configurable
ENG. Ahmed Gamil49
We create our FlexVol inside the aggregate, 20% of the FlexVol is reserved for snapshot
ReviewAll physical disks in NetApp are part of what type of grouping?
RAID
Aggregate
FlexVol
Qtree
Which statement is false?
You can increase the size of a FlexVol.
You can decrease the size of a FlexVol.
A FlexVol is tied to a physical disk.
A FlexVol permits the reduction in administrative scope
SnapShot Snapshot copies are frozen read-only views of our particular volume
It is the first line of defense for backup and recovery of software
We can have up to 255 snapshot per volume
We can do automatic snapshot scheduling, or manually using CLI or GUI (system manager)
We can see
ENG. Ahmed Gamil50
We can take snapshot in hourly basis
We can reserve a space in “%” for snapshot copies
We can check the space allocation for snapshot
ENG. Ahmed Gamil51
SnapRestoreVery easy automated restoration of data
SnapMirrorTechnology for replication and disaster recovery
SnapMirror is a DR application that make a replication from primary controller and secondary controller
Then use the SnapUpdate to performing Data Migrations to apply any changes
ENG. Ahmed Gamil52
SnapVaultSnapVault is a backup application
Moving SnapShot copy data to other locations
SV can be scheduled at multiple intervals
Network AdministrationHow you can set up that NetApp device to easily interact with the network around it.
Name resolutionAs Linux machines, network setting contains hosts file, resolve.conf and NIS which contains IP-Name resolution or DNS settings
ENG. Ahmed Gamil53
And
ENG. Ahmed Gamil54
And we can configure the storage to use ordered setting from “nsswitch.conf” file
Create the VLANs Network interface tap contains all NIC types and configuration (Physical, Virtual, VLANs)
ENG. Ahmed Gamil55
ENG. Ahmed Gamil56
Create VIFVIF is a virtual interface that linked to multible physical interfaces to provide FT
Interface groups / aggregationIf one physical interface is down, no effect on the group
Aggregation of links is part of the 802.3ad standard
Single mode:
One interface is active
Multimode static config:
All interfaces are active
Provide LB for the traffic
Multimode dynamic config:
A dynamic protocol (LACP) is used to determine which interface will be used
Take advantage of link aggregation control protocol (LACP) to support LB
Connected switches should support (LACP)
ENG. Ahmed Gamil57
ENG. Ahmed Gamil58
ENG. Ahmed Gamil59
Using CLIversionGet the firmware on this particular service processor
systemSee all of the system commands
ENG. Ahmed Gamil60
system console connect to the system console
IfconfigConfigure and monitor interfaces
IfgrpConfigure and monitor Interface group
ENG. Ahmed Gamil61
setupWill reconfigure basic configuration files like “/etc/rc”, “/etc/exports”, “/etc/nsswitch.conf”, etc…….
Creating LUNsThere are many ways to create LUNs
Create LUN GUI
Or
ENG. Ahmed Gamil62
Then
ENG. Ahmed Gamil63
Then choose the related volume that will create the LUN inside
ENG. Ahmed Gamil64
Then create the igroup: the initiator group that's allowed to access this LUN (just a table of who can access the particular LUN)
ENG. Ahmed Gamil65
Delete LUN GUIWe cannot delete online LUN, we have to make it offline first
ENG. Ahmed Gamil66
Create LUN CLIGo to system console
ENG. Ahmed Gamil67
Using SnapDriveSnapdrive is a software addons for WIN and Linux desktop
Used to expand the storage, manage snapshot copies, cloning disks
NFS Exports and CIFS SharesIn System Manager:
Create Share for a Windows based CIFS Share
Create Export for NFS export
ENG. Ahmed Gamil68
For CIFS, WIN users has to Map network drive to share the location for all users
For NFS export, Linux host has to create directory and mount the exported NFS in order to use the storage
If we go to export TAP, we will find our previous FlixVol we had created “East”
In order to create new export, lets delete “/vol/East” and create it again
Delete Export
Create ExportChoose the volume you need to export
ENG. Ahmed Gamil69
Then set permissions
ENG. Ahmed Gamil70
Create CIFS shareNo CIFS share automatically created, System Manager contains only hidden shares in windows env.
We will create new CIFS share with same steps as Export
ENG. Ahmed Gamil71
Also you can “Edit” and change permissions and other options
ENG. Ahmed Gamil72
Stop Sharing
ENG. Ahmed Gamil73
Using BranchCacheBranchCache is a feature used when your remote office is connected with a low-bandwidth WAN connection, and you have many devices in remote site needs the same file, so using BranchCache will send the file to only one remote device then cache it to others
Distributed cachingAny remote device can cache the file and provide it to other devices like “P2P”
Hosted CasingUse a server instead of normal devices to cache files and provide them to devices like “client-server”
Branch cache configurationsMost branch cache configurations done by CLI
Branch cache should be enabled per service
Go to system Console
Check Branch cache for CIFS share
ENG. Ahmed Gamil74
Create Branch cache for CIFS share
cifs shares -add share_name path –branchcache
cifs shares -add projects /vol/East –branchcache
Using QuotasWe have many types of quotas: Qtree quota, user quota, group quota
Qtree is seems to be as flexvol volumes but there are some differences
You can enable/disable snapshot feature
Space guarantee
Support quotas
Support backup
Security styles (NTFS, UNIX, MIXED)
CIFS oplock
ENG. Ahmed Gamil75
Create Qtree
Set QuotaYou can set quota based on the used space or no. of files
ENG. Ahmed Gamil76
ENG. Ahmed Gamil77
Then set hard limits, soft limit “warning” and threshold
ENG. Ahmed Gamil78
To get information about the quota:
/etc/quotas to see raw storage information about the quota
ENG. Ahmed Gamil79
RBACRole-Based Access Control used to enhance the security of your organization by:
Provide differing levels of access for different users or groups
Apply Least Privilege concept: create account that has the least level of privileges
Able to log what one administrator is doing from another
Provide
Capabilities, Roles,
Create UserCreate new user “gamil” and assign it to Administrator group
ENG. Ahmed Gamil80
ReviewWhich statement regarding RBAC in NetApp is incorrect?
Capabilities refer to specific actions
User accounts are placed in groups
Capabilities are assigned users
NetApp includes an Admin and Power Users group
Storage System MaintenanceAuto support
ENG. Ahmed Gamil81
LogsThrough Event Management System (EMS) that generate the logs, you can filter the logs
Degraded diskIt is a situation that we have RAID-DP with 5 disks, then 2 disks failed, and no other spare disks
Disks will stay in this degraded mode for 24 hours by default, then shutdown
So data is still exist but there's no spares, this situation called degraded disk
If the situation occurred, you will receive auto support e-mails and
Disk scrubbingNetApp check media for errors, check the parity information
Start checking Manually CLI:
disk scrub start
disk scrub stop
We can start checking automatically
sysstat sysstat is a CLI tool that doing a certain number of queries of the system statistics, or set the interval
ENG. Ahmed Gamil82
Storage Space ManagementHow you are doing space guarantee or disk provisioning
ProvisioningFull provisioning: full space guarantee
Thin provisioning: allocate space as it is stored
File provisioning: need space guarantee for a certain file
ENG. Ahmed Gamil83
Space allocation You can check space summary from “space allocation tap”
CLI df –h ncsa_class
aggr show_space –h aggr0
ENG. Ahmed Gamil84
Storage Efficiency (Deduplication)Take fingerprint of the block, then looking for blocks with identical fingerprints, then erase that blocks and add pointer that points to the other block that has the identical fingerprint.
So that there's no identical storage.
ENG. Ahmed Gamil85
Clustered Data ONTAP Concepts In 2011, we had ONTAP 8.1 referring to two different modes of operation:
The 7-mode
Clustered mode
For NAS, we can have 24 nodes inside of our clustered environment.
For SAN, we can have 8 nodes inside of our clustered environment
ENG. Ahmed Gamil86
Clustering = HA pair = multiple node is working together with 10 GB connection between each other
ONTAP 8.2 support single node cluster, 2 node switchless cluster
Virtual storage servers (VSS)
Data Vserver is presenting a particular namespace to clients
Clustered Data ONTAP User InterfacesThere are 2 user interfaces:
OnCommand system manager (GUI)
Cluster shell (CLI)
For CLI, you can use “tab” to auto complete
“..” go back one step
“top” go to the top of hierarchy
st get statistics or storage
ENG. Ahmed Gamil87
Clustered Volumes and NamespacesIn a clustered environment we deal with only FlexVol
FlexVol can move through the cluster
FlexVol owned by Vservers
NamespaceVolumes from all over the cluster become a Namespace
Access to the root of the Namespace allow access to volumes information that exists below it
JunctionBind the volumes together in the Namespace.
In order to create a namespace, we have to create a junction for all volumes
Junction looks like a directory path “/eng/p7/source”
Infinite volume Starting from clustered data ontap 8.1.1
Provides a single mount point that can scale to 20PB and 2 billion files
Infinite volume integrates with deduplication, compression, and NetApp SnapMirror replication technology
From clustered data ontap 8.1.1, they started to add disaster recovery features
Infinite volume doesn’t support Qtrees or its quota
To create Infinite volume, you have to activate “IS repository switch”
ENG. Ahmed Gamil88
Clustered File Access Enabled NFS on the Storage Virtual Machine (SVM) to manage file access using NFS.
NFS v4 and later are stateful technology
Clustered Data ONTAP 8.1 support NFS4 and 4.1, PNFS (Parallel NFS) and CIFS
NFS4 use referrals concept that allow users to obtain the information from the storage system
If the user request for information comes into node1, but the information is actually stored in node2.or, if volume is moved from one node to another, SO:
NFSv4 Clients have to unmount and remount the file system manually
PNFS (Parallel NFS)Does not have to unmount or remount to access the new location of the volume
PNFS uses logical interface (LIF) to communicate with the cluster
LIF There is LIF Inside clustered Data ONTAP, which is logical interface owned by storage virtual machine (SVM)
ENG. Ahmed Gamil89
LIF Is an IP address associated with a physical port. If there is any component failure, a LIF can fail over to or be migrated to a different physical port, thereby continuing to communicate with the cluster.
PNFS allows us to separate the Metadata Path from the actual Data Path, so the client can reach the data wherever it is stored in any node
PNFS does not support FlexCache volumes, coral volumes, and load-sharing volumes.
Common Internet File System (CIFS)Used SMP protocol version 2, 2.1 and 3, it is stateful technology since SMB v2
SMB v2.1 Support Oplock leasing
SMB v3 support BranchCache 2, Witness Protocol and Hyper-V over Server Message Block
Netapp NetworksManagement Network
Admin use LIF to access the management network and manage the infrastructure
Cluster Interconnect
There's logical interfaces for the cluster interconnecting for the inter-cluster communications
Data network
There's LIF for end user who is accessing the data that's stored in the cluster
ENG. Ahmed Gamil90
Clustered Load BalancingThe ability to smooth out load, to load balance in your storage enterprise.
Natapp allow LB through DNS
You can create volumes across NetApp cluster in a round-robin fashion, and we can move volumes within the vserver between nodes or aggregates
In a CIFS environment, the CIFS clients might see a slight degradation in performance
Load Sharing Mirrors (LSM)We can take a particular FlexVol, then create a load-sharing mirror and copy a read-only copy of that FlexVol.
The read-only copy exist in the same location in the namespace, but it's help with our load-balancing implementation.
There is automatic and manual sync between original FlexVol and the read-only copy
SAN InfrastructureThere are 2 operating systems:
data ontap
SANtricity used in big data environments (E-Series NetApp devices)
ENG. Ahmed Gamil91
Unified storage NetApp can easily address both NAS and SAN and at the same time
NAS : file-based access to storage using CIFS or NFS
SAN : Block Level access using iSCSI or FC or FCOE
For FC, the connection terms are:
WWNN: Worldwide Node Names, a unique identifier for the device itself.
WWPN: Worldwide Port Names that are identifiers for the particular portal
For ISCSI:
IQN style EUI style
Steps for the establishment of the SAN:Disc system use as our Target device
Create a session from the Initiator to this Target
Create an igroup controls access to the LUNs. (Like Zoning)
Create a LUN
Map a LUN to igroup
Find a LUN on the OS by the user
Prepare the LUN
FC ConnectivityWhat is NetApp going to plug in to?
ENG. Ahmed Gamil92
What is end host systems going to plug in to?
How's that all going to work?
The parallel SCSI has scalability issues with limited speed
Fibre Channel connectivity is faster and can accommodate 12 million addresses
Fibre Channel connection: P2P : legacy Arbitrated Loop (AL): used when connecting Fibre Channel disks inside of the NetApp Fabric Config : used today
FC support Dual Fabric (switches) to provide redundancy to prevent single point of failure
We can do dual path with single fabric and dual path with dual fabrics
The initiator for the fabric can be Linux, WIN, and VMware and the endpoint for the Initiators is a Host Bus Adapter (HBA).
The target device is like Brocade, Cisco
So the initiator connect to HBA of the FC switch and the switch is connected to the NETAPP target FC interface (fc0b)
Commandssysconfig commandAllow you to identify your onboard Fibre Channel adapters on the particular NetApp device
Allow you to do the Fibre Channel Admin command
fcadmin commandLet you view the current configuration in Fibre Channel config,
configure particular Fibre Channel
ENG. Ahmed Gamil93
License commandTo view and add license
FCOE is not a Host Bus Adapter “HBA”, it is Converged Network Adapter
FCOE converge our network traffic, it take FC traffic and lay it through the LAN (encapsulating it in Ethernet frames)
FCOE requires 10 gigabit per second connectivity
ENG. Ahmed Gamil94
Referenceshttps://www.cbtnuggets.com/it-training/netapp-certified-storage-associate-ncsa-ns0-145
http://www.datadisk.co.uk/html_docs/netapp/netapp_disk.htm
ENG. Ahmed Gamil95
top related