Storage Virtualization and VDI Storage
Storage Virtualization and VDI Storage
Sisimon Soman
Agenda
• Datacenter Storage in General
• DAS, NAS and SAN
• Storage Virtualization
• VDI Storage– Issues with VDI Storage– Solutions for Performance issues
Study by IDC
• In 2011 roughly 1.8 trillion GBs of data created.
• Total data doubled in last two years.
• The prediction is total data will grow by 50x in next decade.
• Too much of unstructured data.
• Disks are still in stone-age with motor, spindle and head.
Numbers collected from EMC.com which is available for public
Types of Storage
• Classify into three according to the access mechanism between server and storage.– Direct Attached Storage (DAS)– Network Attached Storage (NAS)– Storage Area Network (SAN)
Direct Attached Storage (DAS)
• Disk(s) directly connected to the machine.• Most simple/common storage.• Example: Our Laptops, Desk tops etc
SCSI
Network Attached Storage
• Uses CIFS/NFS protocol to access files.• Similar to a remote shared folder.• Client side redirector forward the file requests to
NAS box.• Example: EMC Celerra, NetApp FAS
LAN LAN
Servers
NAS box
NAS - Module diagramApplication
File System Redirector
CIFS Protocol Layer
TCP/IP Stack
NIC Driver
User Space
Kernel Space
LAN LAN
CIFS Protocol Layer
TCP/IP Stack
NIC Driver
File System
Volume Manager
Disk Driver
NAS Box
Server
What is a SAN
Picture taken from wikipedia
EMC Symmetrix DMX 2000 EMC Symmetrix DMX 1000
SAN Continued..
• Example : EMC Symmetrix, CLARiiON
Fibre Channel Switch
Servers
SAN
SAN Architecture
Fibre Channel Switch
Controllers/Storage Processors (It can be ACTIVE-ACTIVE or ACTIVE-PASSIVE)
Disks
GBs of memory in each controller
Difference between SAN and NAS
• In NAS ‘file streams’ are transferred thru wire.
• In SAN ‘disk blocks’ are read from storage.• In SAN Fibre Channel is the common
communication mechanism. (SAN support SCSI over TCP/IP also, which is called iSCSI)
• In NAS file streams transfer over TCP/IP stack.
Common features of SAN
• Backup
• Replication
• Snapshot
• Features specific to SAN like SRDF for Symmetrix
SAN Specific data collected from wikipedia
Storage Virtualization - Advantages
• Hide the internal complexity of storage system.
• Better Disk block usage – Study shows only 30-40% of disk space is used effectively.
• Better Performance.
• Scalability.
Taxonomy of Storage Virtualization
• Virtualization at Host Operating system storage stack.
• Switch/Appliance based Virtualization.
• Virtualization at external storage array (SAN).
Storage Stack
File System
Volume Manager
Disk Class Driver
Hardware Driver
ApplicationUser Space
Kernel Space
Host Operating system Storage stack
Fibre channel adapter
Fibre channel Switch
SAN
Fibre Channel cable
Virtualization at Operating Storage Stack
File System
Volume Manager
Disk Class Driver
Hardware Driver
• A typical Windows Storage stack
IO Request flow thru each layer
Virtualization at File System Layer
File System
Volume Manager
Disk Class Driver
Hardware Driver
File1.doc
Disk blocks
Virtual Cluster Number (VCN)
Logical Cluster Number (LCN)
• NTFS expose IOCTL FSCTL_GET_RETRIEVAL_POINTERS, so that any app can query VCN-LCN mapping. Usually the disk defragment app uses this IOCTL.
HSM File System (Tiered Storage)
RAM ($50/MB)
SAN ($.5/MB)
TAPE($.05/MB)
Only 20-30% of actively used data
• Ex: EMC DiskXtender
Virtualization at Volume Manager - RAID
• RAID 5 - Also know as Striped Volume with parity.• Fault tolerance is achieved by reserving an equivalent of
one disk for keeping parity information.• Rotate the parity stripe to all disks - Avoid the possibility
of parity disk become busy all time.
RAID
Virtualization at Disk Controller
• Disk Controller convert Logical Block Address to Cylinder Head Sector address.
• Disk controller take care of damaged sectors also.
LBA 1
LBA 2
LBA 3
LBA 4
LBA 5
LBA n
Putting it all together
Windows Storage Stack
Switch based Virtualization
• Combine LUNs from one or more Arrays to a single virtual LUN and pass to host OS.
• Take one big LUN from one Array, divide it and give to different host OS.
• Security – One host can see certain LUNs only.• Vendors are adding more intelligence in switch
level like advanced volume management, caching, QoS functions…
• EMC Invista, IBM SAN Volume Controller
Switch level Virtualization
IBM EMC HP HDX
Intelligent switch – Virtualize LUNs from different array
Virtualization at SAN
Virtual Disk
Disk in the storage array
VDI Storage
Problems with VDI Storage
• Boot/Login Storm• App Storm• Virus scanning.• Many PoC fails or end up with more cost
because of storage array cost.• Some queries in XD/VDI-in-a box forum
– Bootup time of approx ~170 min.– Slow logon.– Do I need to put dedicated LUN to each server.
Windows partition alignment issue
• Data stored in disk as blocks. Block size vary, usually 64K (multiple of OS page size).
• Windows XP/2003 write signature starting of partition and actual partition start at 63 sectors to make it aligned with disk cylinder boundaries.
• This can result in extra IOs.• The partition should aligned with SSD
cache/storage block.
Windows 2003 partition
Block 1 Block 2 Block 3 Block 4 Block 5 Block 6
NTFS Volume 1Signature
Partition starting at less SSD friendly location
IO Blender problem
• Sequential and Random IO.
• OS try to make the IO sequential – Windows Cache Mgr and SCSI/Storport driver, Linux Buffer cache and IO Scheduler.
• Hypervisor screw-up this optimization.
• OS think data from block storage, hypervisor convert it to VHD file (.vmdk)
Read/Write IOPs
• Write IOPs are costly – Cache flushing, RAID cost..
• Windows Paging IO – Paging IOs are sensitive and slowing down it reduce system performance.
• Memory intensive App may increase Paging IO. Number of Paging writes may go higher than write IOs from App.
What is new in VDI Storage
• More than a dozen Storage Startups
• Driver at guest OS stack to profile IOs.
• Module at Hypervisor storage stack which do actual IO scheduling.
• Merge random IOs, Dedupe and compression(make ssd cache effective).
• SSD cache
• Algorithm to analyze App IOPs(profiling) and scheduling is proprietary to vendors.
What is new in VDI Storage
IO Profiler
OS 1
IO Profiler
OS 2
IO Profiler
OS 3
IO Scheduler
Hypervisor
SSD Cache/Storage
Hardware
Questions ?Questions ?