Storage Tiering for Microsoft Exchange Server and EMC ... · PDF fileEMC Symmetrix VMAX with Enginuity 5875 ... Thin pool configuration ... directly or remotely to a Symmetrix VMAX

White Paper

Abstract

As the need for more information continues to explode, businesses are forced to deal with an ever-increasing demand for storage and at the same time, a requirement to keep cost to a minimum. One way to satisfy both needs is to effectively use low-cost SATA drives, high-performance Fibre Channel drives, and Enterprise Flash Drives to provide high performance for mission-critical applications and cost-effective storage for non-critical applications. Matching business needs with various drive types is known as “tiering,” and this white paper describes how to implement storage tiering using EMC® Symmetrix VMAX™ Enhanced Virtual LUN Technology and Fully Automated Storage Tiering (FAST). December 2010

STORAGE TIERING FOR MICROSOFT EXCHANGE SERVER AND EMC SYMMETRIX VMAX WITH ENGINUITY 5875

2 Storage Tiering for Microsoft Exchange Server and EMC Symmetrix VMAX with Enginuity 5875

Copyright © 2009, 2010 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate of its publication date. The information is subject to change without notice. The information in this publication is provided “as is”. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are the property of their respective owners. Part Number h6733.3


Table of Contents

Executive summary.................................................................................................. 5

Audience ............................................................................................................................ 6

Products and features overview ............................................................................... 6

Symmetrix VMAX ................................................................................................................ 6

Symmetrix Management Console ....................................................................................... 7

Symmetrix Enterprise Flash Drives ...................................................................................... 8

Symmetrix Enhanced Virtual LUN (VLUN) migration ............................................................ 9

Virtual Provisioning .......................................................................................................... 10

Virtual Provisioning enhancements .............................................................................. 11

Large volume support ....................................................................................................... 12

Evolution of storage tiering .................................................................................... 13

Symmetrix Fully Automated Storage Tiering ...................................................................... 13

How FAST is configured ................................................................................................ 15

FAST device movement ................................................................................................. 16

Symmetrix Fully Automated Storage Tiering for Virtual Provisioning (FAST VP) .................. 16

How FAST VP is configured ........................................................................................... 18

FAST VP data movement ............................................................................................... 19

Exchange Server basic storage requirements .......................................................... 20

Exchange Server Mailbox Server Role disk sizing .............................................................. 20

“Skewed” Exchange LUN access....................................................................................... 21

Exchange Server 2010 and the advent of Virtual Provisioning ........................................... 22

Exchange Server 2010 database maintenance ................................................................. 22

Database maintenance configuration options .............................................................. 22

Online defragmentation ............................................................................................... 23

Online database scanning (Background Database Maintenance) ................................. 23

Storage tiering mechanics ..................................................................................... 23

Storage tier definitions ..................................................................................................... 23

Disk group provisioning tiers ........................................................................................ 23

Virtual provisioning tiers .............................................................................................. 23

FAST ................................................................................................................................. 24

FAST VP ............................................................................................................................ 24

FAST applied ......................................................................................................... 24

FAST test methodology and configuration ......................................................................... 25

Disk configuration – RAID 1 .......................................................................................... 26

Disk configuration – RAID 5 (7+1) ................................................................................. 26

Configuring FAST for the environment ............................................................................... 26

Monitoring workload performance .................................................................................... 30

Identification of candidate volumes ................................................................................. 30


Microsoft Exchange Server Jetstress prior to migration ................................................. 32

Selection of migration targets ........................................................................................... 33

Scheduling migrations ..................................................................................................... 34

Assessing the benefits of FAST ......................................................................................... 35

Microsoft Exchange Server Jetstress post-migration ..................................................... 36

FAST VP applied .................................................................................................... 37

FAST VP test methodology and configuration .................................................................... 37

Thin pool configuration ................................................................................................ 39

Configuring FAST VP for the environment .......................................................................... 40

Monitoring workload performance .................................................................................... 44

FAST VP pool allocation .................................................................................................... 45

Thin data movement without BDM ................................................................................ 46

Thin data movement with BDM ..................................................................................... 48

FAST VP Jetstress results................................................................................................... 49

Conclusion ............................................................................................................ 50


Executive summary EMC® Symmetrix VMAX™ is the newest addition to the Symmetrix® product family. Built on the strategy of simple, intelligent, modular storage, it incorporates a new scalable Virtual Matrix™ interconnect that connects all shared resources across all VMAX Engines, allowing the storage array to grow seamlessly and cost-effectively from an entry-level configuration into the world’s largest storage system. Symmetrix VMAX provides improved performance and scalability for demanding enterprise storage environments while maintaining support for EMC’s broad portfolio of platform software offerings.

EMC Symmetrix VMAX delivers enhanced capability and flexibility for deploying Microsoft Exchange Server and Microsoft Exchange Server 2010 mailbox databases, from a mission-critical email facility to test and development. In order to support this wide range of performance and reliability at minimum cost, Symmetrix VMAX arrays support multiple drive technologies including Enterprise Flash Drives (EFD), Fibre Channel (FC) drives, both 10k rpm and 15k rpm, and 7,200 rpm SATA drives. In addition, various RAID protection mechanisms are allowed that affect the performance, availability, and economic impact of given Exchange Server 2007 and Exchange Server 2010 environments deployed on a Symmetrix VMAX array.

Since customers are under increasing pressure to improve performance as well as the return on investment (ROI) for storage technologies, ensuring that data is placed on the most appropriate storage type is a key requirement. It is common to find that the Exchange Server environment has differing workloads between different users located within the Exchange mailbox databases. It is also common to balance out over a number of Exchange mailbox stores the users of different workload profiles so that no one mailbox store is overworked while others remain underutilized. The imbalance of I/O load across the mailbox database causes much higher utilization of the LUNs holding the active objects and therefore it is referred to as LUN access “skewing.” By using a storage classification there is a case for categorizing mail users dependent on their I/O profile and keeping these categories of users on a common storage type that matches their needs best. This will help increase performance, reduce the overall number of storage types, and improve the total cost of ownership (TCO) and ROI for storage deployments.

Companies increase deployment of multiple drive and protection types in their high-end storage arrays. This gives the Exchange administrator and architect working with the storage administrator the challenge of selecting the correct storage configuration for each class of email user. Often, a single storage type is selected for all data in a given Exchange environment, effectively placing both active and idle mailbox databases on fast FC drives. This approach is expensive and inefficient, because often most users’ email data is not very active and yet it resides on unnecessarily high-performance drives.

Alternatively, making use of high-density low-cost SATA drives for the less active email data, FC drives for the medium active, and EFDs for the very active enables efficient use of storage resources, and reduces the overall cost and the number of


drives necessary. This, in turn, also helps to reduce energy requirements and floor space, allowing the business to grow more rapidly.

To achieve this “tiered” approach with Microsoft Exchange Server 2007/2010 it is necessary to understand the workload characteristics of the volumes involved and to use Symmetrix Enhanced Virtual LUN Technology to move the data between drive types and RAID protections seamlessly inside the storage array. Symmetrix Enhanced Virtual LUN Technology doesn’t require application downtime. It preserves the device IDs, which means there is no need to change file system mount points, volume manager settings, database file locations, or scripts. It also maintains any TimeFinder® or SRDF® business continuity operations even as the data migration takes place.

This manual and often time-consuming approach to storage tiering can be automated using Fully Automated Storage Tiering (FAST). FAST uses policies to manage sets of volumes and available storage types. Based on the policy guidance and the actual workload profile over time, the FAST controller will recommend and even execute automatically the movement of the managed devices among storage types.

This white paper describes a tiered storage architecture for both the Exchange Server 2007 and Exchange Server 2010 Mailbox Server Role and the way in which volumes can be moved nondisruptively using either Enhanced Virtual LUN, FAST, or Fully Automated Storage Tiering with Virtual Pools (FAST VP) in order to be able to put the right data on the right storage at the right time.

Audience

This white paper is intended for Microsoft Exchange Server administrators and architects, storage administrators and architects, customers, and EMC field personnel who want to understand the implementation of storage tiering in a Symmetrix VMAX.

Products and features overview

Symmetrix VMAX

Symmetrix VMAX, the newest member of the Symmetrix family, is a revolutionary new storage system purpose-built to meet all data center requirements as seen in Figure 1. Based on the Virtual Matrix Architecture™ and new Enginuity™ capabilities, Symmetrix VMAX scales performance and capacity to unprecedented levels, delivers nondisruptive operations, and greatly simplifies and automates the management and protection of information.


Figure 1. The Symmetrix VMAX platform

The Symmetrix VMAX design is based on individual engines with redundant CPU, memory, and connectivity on two directors for fault tolerance. VMAX Engines connect to and scale out through the Virtual Matrix Architecture, which allows resources to be shared within and across VMAX Engines. To meet growth requirements, additional VMAX Engines can be added nondisruptively for efficient and dynamic scaling of capacity and performance that is available to any application on demand.

Symmetrix Management Console

Many large enterprise data centers have stringent change control processes that ensure reliable execution of any modification to their IT infrastructure. Often changes are implemented using scripts that are fully documented, have been thoroughly reviewed, and can be consistently executed by all storage administrators. An alternative to using scripts is to use the Symmetrix Management Console (SMC).

The SMC, as shown in Figure 2, is a GUI that allows storage administrators to easily manage a Symmetrix. SMC can be run on many open systems hosts connected directly or remotely to a Symmetrix VMAX that requires management.

1 – 8 redundant VMAX Engines

Up to 2.1 PB usable capacity

Up to 128 FC FE ports

Up to 64 FICON FE ports

Up to 64 GigE / iSCSI FE ports

Up to 1 TB global memory (512 GB usable)

48 – 2,400 drives

Enterprise Flash Drives 200/400 GB

FC drives 146/300/450 GB 15k rpm

FC drives 300/450/600 GB 10k rpm

SATA drives 1/2 TB 7.2k rpm


Figure 2. Symmetrix Management Console

Symmetrix Enterprise Flash Drives

With the Symmetrix DMX™ and Symmetrix VMAX support of Enterprise Flash Drives (shown in Figure 3), EMC has created an ultra-performance storage capability that removes previous performance limitations imposed by magnetic disk drives.

Figure 3. Enterprise Flash Drive

Enterprise Flash Drives (EFDs) dramatically increase performance for latency-sensitive databases like Exchange Server Mailbox Server Role servers. EFDs, also known as solid state drives (SSD), contain no moving parts, which remove much of the storage latency delay associated with traditional magnetic disk drives. A Symmetrix VMAX with EFDs can deliver single-millisecond application response times and up to 30


times more I/O operations per second (IOPS) than traditional Fibre Channel hard disk drives (HDD). Additionally, because there are no mechanical components, EFDs consume significantly less energy than hard disk drives. When replacing a larger number of HDDs with a lesser number of EFDs, energy consumption can be reduced by up to 98 percent for a given IOPS workload.

The high-performance characteristics of EFDs eliminate the need for organizations to purchase large numbers of traditional hard disk drives, while only utilizing a small portion of their capacity to satisfy the IOPS requirements of database. The practice of underutilizing a hard disk drive for increased performance is commonly referred to as short-stroking. EFDs can increase database performance and eliminate the need to short-stroke drives, thus keeping storage footprint and power consumption to a minimum and reducing TCO.

Symmetrix Enhanced Virtual LUN (VLUN) migration

The Enhanced Virtual LUN migration feature introduced with Symmetrix VMAX with Enginuity 5874 and later offers Exchange Server administrators the ability to transparently migrate LUNs used by Exchange Server mailbox databases between differing storage tiers (as shown in Figure 4). The storage types are defined based on a number of attributes that may include a particular storage device technology such as high-performance EFDs, Fibre Channel drives, or high-capacity low-cost SATA drives. Migrations also allow for a change in RAID protection, which may also be an attribute of a type of storage.

Symmetrix VMAX with Enginuity 5875 introduces enhanced functionality that allows for migration of Virtual Provisioning™ (VP) extents between different thin pools. This new functionality now tracks extent allocation for a thin device, and provides the facility to migrate all the extents comprising the storage allocations to a user-specified target thin pool. Thin device migrations allow for changes in RAID protection, and technology type as provided by migrations of non-thin devices.

Figure 4. Enhanced Virtual LUN


Enhanced Virtual LUN is fully integrated with Symmetrix replication technology and maintains consistency and source/target device relationships in replications such as SRDF, TimeFinder/Clone, TimeFinder/Snap, or Open Replicator. The migration does not require swap or Dynamic Relocation Volumes (DRV) space, and is nondisruptive to the attached Exchange Server environment and other internal Symmetrix applications such as TimeFinder and SRDF. All migration combinations of drive types and protection types are valid except for unprotected volumes.

Virtual LUN migration occurs independent of host operating systems or applications, and during the migration the devices remain fully accessible to both the Microsoft Exchange Server mailbox server and the underlying Microsoft Windows Server operating system. While the back-end device characteristics change (RAID protection and/or physical drive type) the migrated devices’ identities remain the same to the host, allowing seamless online migration. Virtual LUN migration is also fully integrated with Symmetrix replication technology and maintains consistency of source/target device relationships in replications such as SRDF, TimeFinder/Clone, TimeFinder/Snap, or Open Replicator.

The advantages of migrating data using storage technology are ease of use, efficiency, and simplicity. Data is migrated in the Symmetrix back end without needing any SAN or host resources increasing migration efficiency. The migration is a safe operation as the target is treated internally as just another “mirror” of the logical device, although with its own RAID protection and drive type. At the end of the migration the data on the original “mirror” is formatted to preserve security. Finally, since the identity of source devices doesn’t change, moving between storage tiers is easy and doesn’t require additional host change control, backup script updates, or changes in filesystem mount points, volume manager, or others. The migration pace can be controlled using Symmetrix quality of service (symqos) commands.

Enhanced Virtual LUN migration helps customers to implement an Information Lifecycle Management (ILM) strategy for their databases, such as the movement of the entire mailbox databases between storage types, including thin pools. It also allows adjustments in service levels and performance requirements to application data. For example, customers often provision storage for a particular application before clear performance requirements are known. LUN migration may be utilized at a later time, once the requirements are better understood, to implement any adjustment to increase user experience and ROI using the correct storage type.

Virtual Provisioning

Virtual Provisioning™, generally known in the industry as “thin provisioning,” enables organizations to enhance performance and increase capacity utilization for Exchange Server environments. The Symmetrix VMAX system includes Virtual Provisioning support native to Enginuity version 5874 and later.

Symmetrix thin devices are host-accessible devices that can be used in many of the same ways that Symmetrix devices have traditionally been used. Unlike regular host-accessible Symmetrix devices, thin devices do not need to have physical storage completely allocated at the time the device is created and presented to a host. The


physical storage that is used to supply disk space to thin devices comes from a shared storage pool called a thin pool. The thin pool is comprised of devices called data devices that provide the actual physical storage to support the thin device allocations.

When a write is performed to a part of the thin device for which physical storage has not yet been allocated, the Symmetrix allocates physical storage from the thin pool for that portion of the thin device only. The Symmetrix operating environment, Enginuity, satisfies the requirement by providing a block of storage from the thin pool called a thin device extent. This approach allows for on-demand allocation from the thin pool and reduces the amount of storage that is consumed or otherwise dedicated to an Exchange instance.

The architecture of Virtual Provisioning creates a naturally striped environment where the thin extents are allocated across all volumes in the assigned storage pool. By striping the data across all devices within a thin storage pool, a widely striped environment is created. The larger the storage pool for the allocations is, then the greater the number of devices that can be leveraged for a given Exchange server instance is. It is this wide and evenly balanced striping across a large number of devices in a pool that allows for optimized performance in the environment.

For additional information, please see the white paper Implementing Virtual Provisioning on EMC Symmetrix with Microsoft Exchange Server available on EMC Powerlink®.

Virtual Provisioning enhancements

With the Symmetrix VMAX array in conjunction with Enginuity version 5874 and later, thin pools now can be reduced in size nondisruptively, helping reuse space to improve efficiency. When a data device is disabled, it will first move data elsewhere in the pool by draining any active extents to other, enabled data devices in the thin storage pool. Once the draining is complete, that data device can then be removed from the pool.

In addition to reusing space more efficiently, benefits of this capability include the ability to:

Adjust the subscription ratio of a thin pool – that is, the total amount of host perceived capacity divided by the underlying total physical capacity

Adjust the utilization or “percent full” of a thin pool

Remove all data volumes from one or more physical drives, possibly in preparation for removal of physical drives with a plan to replace them with higher-capacity drives

Customers can virtually provision all tiers and RAID levels, and support local and remote replication for thin volumes and pools using any RAID type. This includes support for data devices of RAID 1, RAID 5 (3+1), RAID 5 (7+1), RAID 6 (6+2), and RAID 6 (14+2), as well as support for TimeFinder/Clone, TimeFinder/Snap, SRDF/A, SRDF/S, SRDF/DM, Open Replicator, and Open Migrator.


Large volume support

Prior to Enginuity 5874, the largest single logical volume that could be created on a Symmetrix was 65,520 cylinders, approximately 59.99 GB. With Enginuity version 5874 and later, a logical volume can be configured up to a maximum capacity of 262,668 cylinders, or approximately 240.48 GB, about four times as large as with Enginuity version 577x. This simplifies storage management and provides additional flexibility by reducing the required number of volumes to meet a given space requirement.

In general, fewer larger logical volumes perform well in a Symmetrix environment; however, to ensure the best possible performance experience, large volumes should be carefully considered in a traditional, fully provisioned environment with Microsoft Exchange. For example, to assign a single large logical volume that is RAID 1 protected would only allow for two physical spindles to support the workload intended for that LUN. Should the RAID protection be RAID 5 (7+1), for instance, then this concern is lessened as eight disks would be available to service the workload.

Large volume support provides most value in Virtual Provisioning environments. In these environments, customers may strive to overprovision the thin pool as a means to improve storage utilization. Furthermore, Virtual Provisioning deals with the performance needs by utilizing a striping mechanism across all data devices allocated to the thin pool. Performance limits can be mitigated by the total number of spindles allocated to the thin pool.

The implication of utilizing a smaller number of large volumes extends to features such as Symmetrix Remote Data Facility (SRDF). Using a smaller number of larger volumes, as compared to a larger number of smaller volumes, would limit the synchronous write workload capable of running in parallel to the remote site. Such configurations may create additional latency during concurrent database file writes, for example.

With these considerations in mind, it is recommended to use striped metavolumes for Exchange database file LUNs in traditional, full provisioned environments. The number of members in the meta will be dependent on the specific configuration; however, the goal should be to spread the metavolume and subsequently the host workload against the assigned disk group as evenly as possible. For the Exchange log volume, when considering the I/O to that LUN will be serial and sequential, a striped metavolume with a large number of metamembers is not as critical. However, a striped metavolume that meets the required space requirements is still recommended. Should Virtual Provisioning be used in the environment, the wide striping inherent to this technology mitigates the need to use striped metavolumes in most Exchange environments.


Evolution of storage tiering

Symmetrix Fully Automated Storage Tiering

Introduced in the Enginuity 5874 Q4 service release, EMC Symmetrix VMAX Fully Automated Storage Tiering (FAST) is Symmetrix software that utilizes intelligent algorithms to continuously analyze device I/O activity and generate plans for moving and swapping devices for the purposes of allocating or re-allocating application data across different performance storage types within a Symmetrix array. FAST proactively monitors workloads at the Symmetrix device (LUN) level in order to identify “busy” devices that would benefit from being moved to higher-performing drives such as EFD. FAST will also identify less “busy” devices that could be relocated to higher-capacity, more cost-effective storage such as SATA drives without altering performance.

Time windows can be defined to specify when FAST should collect performance statistics (upon which the analysis to determine the appropriate storage type for a device is based), and when FAST should perform the configuration changes necessary to move devices between storage types. Movement is based on user-defined storage types and FAST policies.

The primary benefits of FAST include:

Automating the process of identifying volumes that can benefit from EFD and/or that can be kept on higher-capacity, less-expensive drives without impacting performance

Improving application performance at the same cost, or providing the same application performance at lower cost. Cost is defined as space, energy, acquisition, management, and operational expense

Optimizing and prioritizing business applications, allowing customers to dynamically allocate resources within a single array

Delivering greater flexibility in meeting different price/performance ratios throughout the lifecycle of the information stored

The management and operation of FAST can be conducted using either the Symmetrix Management Console (SMC) or the Solutions Enabler Command Line Interface (SYMCLI). Additionally, detailed performance trending, forecasting, alerts, and resource utilization are provided through the Symmetrix Performance Analyzer (SPA). And if so desired, EMC Ionix™ ControlCenter® provides the capability for advanced reporting and analysis that can be used for chargeback and capacity planning.

From a business perspective, “storage tiering” generally means that policies coupled with storage resources having distinct performance, availability, and other characteristics are used to meet the service level objective (SLO) for a given application. (By SLO we mean a targeted I/O service goal, that is, performance for an application.) This remains the case with FAST.


For administrators, the definition of storage tiering is evolving. Initially, different storage platforms met different SLOs. For example:

Gold Tier – Symmetrix; Silver Tier – CLARiiON®; Bronze Tier – Tape

More recently, storage tiering meant that multiple SLOs are achievable in the same array:

Gold Tier – 15k, FC RAID 1; Silver Tier – 10k, FC RAID 5; Bronze Tier – 7.2k, SATA, RAID 6

FAST changes the categories further. Because multiple storage types can support the same application, “tier” is not used to describe a category of storage in the context of FAST. Rather, EMC is using new terms:

Storage group – logical grouping of volumes (often by application) for common management

Storage class – combination of storage types and FAST policies to meet SLOs for storage groups

FAST policies - policies that manage data placement and movement across storage types to achieve service levels for one or more storage groups

Storage type – a shared storage resource with common technologies, namely drive type and RAID scheme

For example, users might establish a Gold Storage Class as follows:

Service level objective

Storage class FAST policy Storage type

Read/write response time objective

Gold 10%

40%

50%

15k rpm RAID 1

10k rpm RAID 5

7.2k SATA RAID 6


Figure 5 provides a quick summary of the relationship between storage types, FAST policies, and storage groups.

Figure 5. FAST relationships

In the rest of this paper, storage type at times will be referred to as a Symmetrix tier, in order to map clearly to specific commands in tools such as SMC. "FAST policies" and "storage groups" will remain the same throughout this paper.

How FAST is configured

FAST is configured by defining three distinct objects:

Storage groups are a logical grouping of up to 4,096 Symmetrix devices. Storage groups are shared between FAST and Auto-provisioning Groups; however, a Symmetrix device may only belong to one storage group that is under FAST control.

Storage types are a combination of a drive technology (for example: EFD, FC 15k rpm, or SATA) and a RAID protection type (for example: RAID 1, RAID 5 3+1, or RAID 6 14+2). There are two types of storage types – static and dynamic. A static type contains explicitly specified Symmetrix disk groups, while a dynamic type will automatically contain all Symmetrix disk groups of the same drive technology. A storage type will contain at least one Symmetrix disk group but can contain more than one. If more than one disk group is contained in a storage type, the disk groups must be of a single drive technology type.

FAST policies associate a set of storage groups with up to three storage types. FAST policy includes the maximum percentage that storage group devices can occupy in each of the storage types. The percentage of storage specified for each type in the policy when aggregated must total at least 100 percent and may total more than 100 percent. For example, if the storage groups associated with the policy are allowed 100 percent in any of the types, FAST can recommend for all the storage devices to be together on any one type (capacity limit on the tiers is not forced). In another example, to force the storage group to one of the storage types


simply set the policy to 100 percent on that type and 0 percent on all other types. At the time of association, a storage group may also be given a priority (between 1 and 3) with a policy. If a conflict arises between multiple active FAST policies, the FAST Policy priority will help determine which policy gets precedence.

Refer to Figure 5 for the relationship between storage groups, storage types, and FAST policies.

FAST can be configured to operate in a “set and forget” mode (Automatic) where the system will continually gather statistics, analyze, recommend, and execute moves and swaps to maintain optimal configuration based on policy, or in a “user approval” mode (User Approved) where all configuration change plans recommended by FAST must be approved to be executed.

FAST device movement

There are two methods by which a device will be relocated to another type: move or swap.

A move occurs when unconfigured (free) space exists in the target storage type. Only one device is involved in a move, and a DRV (a special Symmetrix device used for device swapping) is not required. Moves are performed by creating new devices in unconfigured space on the appropriate storage type, moving the data to the new devices, and deleting the old device.

A swap occurs when there is no unconfigured space in the target type, and results in a corresponding device being moved out of the target storage type. In order to preserve data on both devices involved in the swap, a single DRV is used (DRV should therefore be sized to fit the largest FAST controlled devices).

Moves and swaps are completely transparent to the host and applications and can be performed nondisruptively, without affecting business continuity and data availability. Symmetrix metadevices are moved as a complete entity; therefore, metadevice members cannot exist in different Symmetrix disk groups.

FAST optimizes application performance in Symmetrix VMAX arrays that contain drives of different technologies. It is expected that customers will have their arrays configured with Flash, Fibre Channel, and/or SATA drives, resulting in storage tiers with different performance levels. Rather than leave applications and data statically configured to reside on the same storage type, FAST will allow customers to establish the definitions and parameters necessary for automating data movement from one type to another according to current data usage. The first version of FAST was released with Enginuity 5874 and will move data at the full device granularity.

Symmetrix Fully Automated Storage Tiering for Virtual Provisioning (FAST VP)

FAST VP builds incrementally upon the functionality of FAST v1, adding support for thin devices and sub-LUN data movement. FAST v1 moved application data only at the volume (LUN) level. Entire devices were promoted or demoted between tiers based on overall device performance. FAST VP adds finer granularities of performance measurement and data movement. The data from a single thin device under FAST


control can be spread across multiple tiers. FAST is free to relocate individual chunks of a thin device, based on performance data gathered at the sub-extent level.

FAST VP is released with Enginuity 5875 and requires Solutions Enabler 7.2.

FAST VP will maximize the benefits of in-the-box tiered storage by optimizing cost vs. performance requirements by placing the right thin data extents, on the right tier, at the right time. The FAST VP system allows a storage administrator to decide how much SATA/FC/Flash capacity is given to a particular application and then automatically place the appropriate busiest thin data extents on the desired performance tier and the least busy thin data extents on a capacity tier. The administrator’s input criteria are assembled into FAST policies. The FAST VP system uses policy information to perform extent data movement operations within two or three disk tiers in the VMAX array. Because the unit of analysis and movement is measured in thin extents this sub-LUN optimization is extremely powerful and efficient. FAST VP made available in Enginuity 5875 is an evolution of the existing FAST and EMC Optimizer technology.

There are two components of FAST VP – the FAST controller and the Enginuity 5875 microcode. The microcode is responsible for collection of performance statistics, at both the LUN and sub-LUN level. The FAST controller is responsible for analyzing performance data collected by the microcode.

The resulting data analysis generates a data movement policy that contains promotion and demotion thresholds for each tier included in a FAST policy.

The microcode applies this movement policy to all thin devices under FAST VP control to determine the appropriate tier for the data and will generate and execute movement requests to relocate thin extents to the appropriate tier.

Figure 6. FAST VP components


How FAST VP is configured

FAST VP requires three control objects —storage groups, FAST policies, and thin tiers.

Storage groups are a logical collection of Symmetrix volumes, typically associated with an application, that are to be managed together.

FAST policies contain a set of tier usage rules that can be applied on one or more storage groups.

Thin tiers contain between one and four thin storage pools of a matching drive technology (EFD, FC, or SATA) and a RAID protection type.

The storage group definitions are shared between FAST and Auto-provisioning Groups. However, a Symmetrix device may only belong to one storage group that is under FAST control.

A FAST VP policy groups between one and three tiers and assigns an upper usage limit for each thin tier. The upper limit specifies how much allocated capacity of the thin devices in an associated storage group can reside on that particular thin tier.

The upper capacity usage limit for each thin tier is specified as a percentage of the allocated capacity of the thin devices in the associated storage group. The usage limit for each tier must be between 1 percent and 100 percent. When combined, the upper usage limit for all Symmetrix tiers in the policy must total at least 100 percent, but may be greater than 100 percent up to 300 percent.

A thin tier will contain at least one thin storage pool from the Symmetrix but can include up to four. If more than one thin pool is contained in a thin tier, the thin pools must be of the same drive technology type and RAID protection.

Figure 7 shows two storage groups, Thin_ProdApp1 and Thin_Development. Each storage group is associated with one policy, Platinum and Bronze, respectively. These policies associate the storage groups with up to three thin tiers.


Figure 7. FAST policy association

Based on the Platinum policy, FAST VP will place 25 percent of the allocated capacity of the Symmetrix thin devices in the Thin_ProdApp1 storage group, in one or more thin pools configured as RAID 5 (3+1) on EFD, 50 percent on RAID 5 (7+1) thin pools on FC, with the remaining 25 percent in RAID 6 (14+2) thin pools on SATA drives.

In the case of the Thin_Development storage group, all of the allocated capacity of the thin devices can exist in RAID 6 (14+2) thin pools on SATA drives. However, depending on performance needs and utilization, up to 25 percent of the allocated thin device capacity may be relocated by FAST to RAID 5 (7+1) thin pools on FC.

Performance time windows can be defined to specify when the FAST VP controller should collect performance data, upon which analysis is performed to determine the appropriate tier for devices. Also, defined data movement windows will determine when to execute the data movements necessary to move data between tiers.

FAST VP has two modes of operation, Automatic or Off. When operating in Automatic mode, data analysis and data movements will occur continuously during the defined data movement windows. In Off mode, performance statistics will continue to be collected, but no data analysis or data movements will take place.

FAST VP data movement

Each FAST VP extent, for which sub-LUN-level statistics are collected, is made up of 48 FAST VP sub-extents – 120 tracks (7680 KB) of contiguous space on each thin device. Activity bitmaps are maintained at this sub-extent level to track data contributing to the performance metrics.

When promoting data to a higher tier, only active sub-extents will be relocated. However, when a FAST VP extent is considered to be idle enough to be demoted to a


lower tier, the entire extent will be relocated. The result is both focused efficient promotion of data and large efficient demotion of data.

Data movements take place by moving the allocated track groups of each FAST VP sub-extent from one thin pool to another. When the relocation completes, the track groups from the original thin pool are deallocated. The first version of FAST VP is released with Enginuity 5875 and will move data at the sub-LUN level of granularity.

Exchange Server basic storage requirements For any Exchange Server Mailbox Server Role design, three basic storage requirements need to be balanced in order to achieve success. These are:

Transactional I/O. This I/O must have a measured performance in latency for each I/O being satisfied by the system.

Always invest in high-performance drives

Always consider performance above capacity

Dedicated drives will give measured performance

Backup and restore throughput must be of an acceptable level to achieve the necessary service level agreements required by the business.

Ensure that there is sufficient capacity to satisfy the business needs on both the primary storage subsystem and the backup medium. Always consider performance above capacity

Exchange Server Mailbox Server Role disk sizing

A common practice for Exchange administrators and architects is to categorize users and their related mailbox activities into profiles. Such categorization is important when considering user activity and has a direct impact on storage sizing, including IOPS, as previously discussed. Many standard guidelines and sizing calculators will break users down into three such profiles: Heavy, Medium, or Light. Additionally, some environments may need to include “Very Heavy” or “BlackBerry” profiles to represent even higher user activity. By using this method of categorizing users by I/O profile it is possible to place the correct users on the correct type of storage.

Disk sizing for Exchange Server has now been assisted by using Microsoft Office Excel spreadsheets, one for Exchange Server 2007 and one for Exchange Server 2010, developed by Microsoft, to give Exchange and storage architects the ability to size a disk subsystem for the Exchange Server Mailbox Server Role. The Exchange Mailbox Server Role Storage Requirements Calculator, which can be downloaded from Microsoft.com, is usually the first point of call when calculating how much storage is required for any given Exchange Server Mailbox Server Role environment. While this tool is an easy first pass, it is recommended that it is used in conjunction with the Exchange Storage Review Program (ESRP) testing results. This will enable the Exchange and storage architects to make a comparison for any given number of users

http://www.microsoft.com/

http://www.emc.com/esrp


for an Exchange Server Mailbox Server Role. The use of both tools will enable a more accurate disk sizing exercise to be achieved.

Although the Microsoft tool allows for different tiers of users to be used to calculate the disk numbers, it does not cater for the ability to have different storage types within the same storage array to service the different tier of users. The number of disks calculated, to house the Exchange databases, will be of the same disk type (15k, 10k, or 7.2k), same disk size, and RAID type for every database.

When using the Microsoft calculator and comparing the ESRP results for any given set of user IOPS and capacity requirements, it is very difficult to get a precise disk sizing for the storage subsystem. Also if using performance monitoring on an existing Exchange environment, unless it is of the same version, there is no direct tool to compare the output from an Exchange Server 2003 mailbox server and use those numbers to calculate the disk requirements for either an Exchange Server 2007 or Exchange Server 2010 mailbox server. It will always be an estimation of the number of disk drives that are required. It is only when the Exchange Server mailbox server is in production, with users carrying out their daily email functions and effective performance monitoring is carried out, can the real workload be verified.

“Skewed” Exchange LUN access

Skewed LUN access is a phenomenon that is present in most databases. Skewed LUN access is when a small number of database LUNs receive a large percentage of the I/O from the applications. This skewed type of activity can be transient, that is to say, lasting only a short period of time, or persistent, lasting much longer periods of time. Skewed, persistent access to volumes makes those volumes good candidates to place on a higher-performing storage type, if they are not already on the highest-performing storage type.

Skewed LUN access in Exchange Server can still be present by the fact that 100 percent of all users are not always doing email operations at the same time or in the same manner. In most Exchange Server configurations the storage design is sized on a 100 percent concurrency of users. This high level of concurrency is rarely realized so the storage subsystem may be running at a lower utilization than was designed for. Skewed LUN access in Exchange can also occur when users are placed, for example, into a specific mailbox database dependent on their name and not necessarily by their I/O usage profile. This can lead to mailbox database stores having completely different I/O characteristics than was originally designed. It is always best practice for Exchange administrators to monitor the I/O characteristics of the different mailbox databases so that they can be balanced, first across different mailbox databases on a single mailbox server, or second between Exchange servers in the Exchange Organization. However the balancing of these users to give the best usage of the storage resources will lead to some downtime for user access to their mailbox. This may be unacceptable in some environments. This differing of I/O profiles across the storage platform can mean that some LUNs are overutilized, with others being the opposite and receiving lower activity than designed. If these volumes are persistently receiving fewer requests than others they may be good candidates to move to a lower-


performing storage type, while the overutilized ones may be moved to a higher-performing storage type.

Exchange Server 2010 and the advent of Virtual Provisioning

With the advent of Exchange Server 2010 and the further reduction in I/O to the disks, in a normal, standard user environment, the necessity to use a single drive technology is further reduced. Exchange Server 2010 also brings with it the advent of very large mailboxes supported by the latest version of Microsoft Office, currently Office 2010. A mailbox size of 10 GB may be common in some organizations but the cost of providing this huge storage capacity at day 1 of the implementation would also have a very high capital cost. The use of Virtual Provisioning for Exchange Server 2010 is a very flexible way of provisioning the server with a LUN of the correct capacity, but with an actual smaller LUN on the storage array being used and the reduced capital costs involved.

As previously stated, when calculating the number of disks required for an Exchange Server 2010 environment it is always paramount to get the correct number of disks to meet the performance criteria. If the whole of the storage was provisioned at the initial implementation to meet the very large mailboxes it is likely that there would be enough disks servicing the number of IOPS to the Exchange servers, and as already stated this could be a huge capital cost. If the number of drives needed to meet the initial implementation capacity requirements is lower than required to meet the total IOPS performance requirement, and as not everyone will fill a 10 GB mailbox from day 1, a more cost-effective solution would be to use a mix of drive technologies. Thus, using a mix of Fibre Channel and SATA drives would meet both the required capacity and performance criteria. The use of FAST VP to control the placement of data extents on the LUNs for the Exchange database files, thereby putting the extents on the appropriate storage tier, is an ideal solution to meet these needs. By using Virtual Provisioning and growing the storage environment according to the growth of the users’ mailboxes, the cost of the storage requirement will spread evenly over time. In some cases where the users’ mailboxes do not actually grow to the notional “10 GB,” as designed for, the storage environment may never require additional disk upgrades.

Exchange Server 2010 database maintenance

Database maintenance configuration options

Exchange 2010 database maintenance is now comprised of several tasks that maintain the integrity of mailbox databases. Database maintenance is divided into two parts:

Store mailbox maintenance

Extensible storage engine (ESE) database maintenance

In previous Exchange versions (Exchange 2007 SP1) ESE database maintenance was disk-intensive. In Exchange 2010, improvements have been made to increase performance in order to support both large mailboxes as well as to support different storage types.


Online defragmentation

The architecture for online defragmentation has changed in Exchange Server 2010. Online defragmentation was moved out of the mailbox database maintenance process, and now runs in the background 24×7. No settings need to be configured for this feature. Exchange monitors the database as it's being used, and small changes are made over time to keep it defragged for space and contiguity. Online defragmentation is also throttled so that it doesn't have a negative impact on client performance

Online database scanning (Background Database Maintenance)

Online database scanning runs a checksum against the database and carries out post-Exchange Server 2010 store crash operations. Online database scanning will find and recover lost space due to store crashes. In Exchange Server 2010, there are now two options to run online database scanning (or BDM) on active database copies

Run BDM in the background 24×7 as the default behavior. This option is recommended for databases larger than 1 TB, where more time is needed to checksum the databases. Exchange scans the database no more than once per day, and again will generate a warning event if it can't finish scanning within a seven-day period.

Run BDM by changing the Mailbox Database Maintenance schedule. This option is recommended for databases that are less than 1 TB in size.

So while BDM is now an ongoing, rather than scheduled if turned on, process and as such will have an impact on the I/O that is being carried out at the disk, many of the operations are sequential and so are far more disk-friendly.

Storage tiering mechanics

Storage tier definitions

There are two types of storage tiers – disk group provisioning (DP) and virtual provisioning (VP).

Disk group provisioning tiers

The storage tier used by FAST is called a DP tier. It is defined by combining one or more physical disk groups, of the same technology type, and a RAID protection type.

Virtual provisioning tiers

For FAST VP, the storage tier is called a VP tier. When defined, VP tiers contain between one and four thin storage pools – each pool must contain data devices of the same RAID protection type and be configured on the same drive technology type.


FAST

The goal of an effective storage tiering approach in a multi-tiered storage configuration is to place the right data on the right storage at the right time. A given Exchange Server LUN may be highly active and highly accessed when the mailbox database is populated in the first instance. But over time, its usage may drop to where it could be deployed on a tier of storage that has lower-cost and lower-performance characteristics. Alternatively, co-located storage devices within a given storage type (also known as a tier) may be contending for I/O resources between Exchange servers, or between Exchange Storage Groups located on the same Exchange server. One workload may represent a much more demanding I/O profile. Identification of this demand, and mapping this to available storage tiers within the array, may not only improve the performance of the higher workload, but in so doing, may provide benefits to the remaining mailbox databases as the available resources are able to satisfy the reduced I/O demand.

A typical storage tiering approach uses the following steps:

1. Monitor volume performance

2. Identify candidate volumes for migration

3. Find space to move volumes

4. Schedule the movement

5. Repeat the process at a later time

Fully Automated Storage Tiering is the automated processing for these five steps.

FAST VP

The way that FAST VP works is that only those extents needing to be moved to another tier of storage will actually be moved, either up a tier or down a tier accordingly.

Storage tiering using FAST VP uses the following steps:

1. Monitor volume performance

2. Identify candidate LUN extents for migration

3. Find space within another appropriate thin pool to move extents

4. Schedule the movement

5. Constantly monitor LUN extents so that the process may be repeated at a later time

FAST VP is the automated processing for these five steps.

FAST applied To validate the mechanisms and show the efficacy of the FAST solution for a Microsoft Exchange Server environment, the following testing was conducted. A user workload was generated against an Exchange Server mailbox database. Storage tiers were


defined within the environment, and a FAST policy was defined for the Exchange Server mailbox database. FAST monitoring was implemented, and a user approval mode was utilized to identify the actions the FAST controller determined. These suggested migrations were subsequently approved, and the system monitored for the resulting performance changes.

FAST test methodology and configuration

For FAST testing, Microsoft Windows Server 2008 SP2 and Jetstress for Microsoft Exchange Server 2007 were utilized. The mailbox database was implemented to a single Exchange Storage Group and comprised a single LUN. An additional LUN was utilized as the location for the transaction log device. Logically, the configuration was as shown in Table 1.

Table 1. Test configuration

Host device Style SymmDev Allocation (KB) Volume name

\\.\PHYSICALDRIVE7

M(4) 0646 2515968000 SG1_Data

\\.\PHYSICALDRIVE6

M(4) 064A 2515968000 SG2_Data

\\.\PHYSICALDRIVE2

M(4) 0652 125829120 Logs

The simulated environment used was Jetstress for Exchange Server 2007. As such, it represents the workload of an Exchange Server 2007 mailbox server environment. Simulated Exchange database operations are carried out, to perform a range of processing tasks. The workload generates a significant read/write requirement against the mailbox database. It was expected to achieve ~1200 IOPS for the testing. The Jetstress parameters are shown in the following table.

Table 2. Jetstress parameters

Jetstress Parameter Value selected

Define Test Scenario Test Category

Test disk subsystem throughput

Select Capacity and Throughput Size database storage capacity %age = 100

Suppress tuning and use thread count (per storage group) = 16

Select Test Type Performance

Test Duration 4 hours

Define Storage Groups Number of Storage Groups = 2

Number of databases = 2


Disk configuration – RAID 1

Each metavolume presented to the host as a LUN was created as a striped configuration using two members, where each member hypervolume was 120 GB, giving a 240 GB volume presented to Exchange Server. Each hypervolume was of a RAID 1 configuration. Thus each hypervolume was located on a set of four physical spindles. These drives were 450 GB 15k rpm.

Disk configuration – RAID 5 (7+1)

Each RAID 5 volume presented to the host as a LUN was created as a striped configuration using eight members, where each member hypervolume was 30 GB, giving a 240 GB volume presented to Exchange Server. Thus each volume was located on a set of eight physical spindles. These drives were 450 GB 15k rpm for Fibre Channel and 1 TB 7.2k for SATA.

The tests outlined for FAST in this white paper were performed on the following configuration:

Configuration aspect Description

Storage controller Symmetrix VMAX 1194

Microcode 5874.xx.xx

Microsoft Windows Microsoft Windows Server 2008 SP2

Jetstress Executable 08.02.0060.000

Jetstress Core Engine 08.02.0060.000

Tier 1 4 x 450 GB 15k Fibre Channel

Volumes 2 x 2-member striped metavolume

Tier 2 16 x 450 GB 15k Fibre Channel

Volumes 2 x 8-member RAID 5 (7+1) volumes

Tier 3 16 * 1 TB SATA

Volumes 2 x 8-member RAID 5 (7+1) volumes

Configuring FAST for the environment

When initially configured, the storage allocations for all LUNs were made from a single pool of physical spindles with a single RAID protection scheme. This is synonymous with a storage type or tier, as previously defined. The Symmetrix VMAX array contained a number of different technologies including 450 GB 15k rpm drives and 1 TB SATA drives. Using these differing technologies and available RAID levels, it is possible to construct the various storage tiers that may be applied.


Figure 8. SMC FAST Configuration Wizard

In the tested environment, multiple storage tiers were defined, as shown in Figure 9. A type (or tier) of storage may differ on the technology implemented (the physical drive characteristics) or by the RAID protection scheme implemented. The tiers created in this configuration varied both in terms of the underlying technology, where the tier “FC_R1_DG0” was defined as RAID 1 protection on 450 GB 15k rpm devices, FC_R57_DG3 was defined as being a RAID 5 (7+1) protection scheme on Fibre Channel 450 GB 15k rpm drives, and SATA_R57_DG1 was defined as being a RAID 5 (7+1) protection scheme on 1 TB SATA drives. Other entries also exist, but these three tiers were subsequently used to define a policy for the Exchange Server environment.

Figure 9. Defining storage types (also known as storage tiers) within SMC


The storage tiers were applied to create a policy named “FAST_EX2007”. This policy defined the applicable storage tiers, and applied capacity percentages to the various tiers. The percentages define how much of the storage space used by the storage group defined in the policy can be utilized from each of the tiers. In an Exchange Server environment the movement between storage tiers will always be dependent on the performance and not capacity, therefore the percentage allocation for each storage type must be 100 percent as shown in Figure 10.

Figure 10. Tier definition within SMC

The allocation of a storage group to a defined tier binds the policy to the devices contained with the storage group. The storage group is defined when implementing Auto-provisioning Groups, and defines the storage devices that will be available to a host when bound to a view. For the tested workload the storage group name was “FAST_EX2007” and it defined all storage devices that were presented to the Exchange Server mailbox server. In Figure 11 the storage group is displayed with its component devices.


Figure 11. Allocating a storage group to a policy in SMC

To complete the implementation of the FAST mechanisms, it is necessary to set appropriate time periods to collect statistics for workload analysis. Statistics collection is defined within the Symmetrix Optimizer environment, and may also be set through Symmetrix Management Console, as shown in Figure 12.

Figure 12. Symmetrix Optimizer collection and swap/move windows

It is possible to configure Symmetrix Optimizer to execute swaps automatically or require user approval for swaps. If automatic mode is set for Symmetrix Optimizer, FAST suggested movements will be executed, as Optimizer is used for affecting


movement. In the tested configuration, Optimizer was left in user approval mode, to show the planned device migrations.

Having defined the storage group association to the storage types through the policy, and scheduling the application time periods within Symmetrix Optimizer, the environment was configured appropriately. To validate the selection of devices, and planned movement, the operating system and Exchange Server Jetstress monitoring were also configured.

Monitoring workload performance

Microsoft Exchange Server and Windows Server administrators will typically utilize Windows performance counters to monitor overall system performance. The instrumentation provided by the Windows performance counters represents a valid performance profile of the behavior of the storage devices presented to the host. The expected total IOPS throughput was ~1200 for both database volumes. Windows performance counters, utilized by Jetstress, will be used in the following sections to monitor and qualify migrations.

Identification of candidate volumes

A Jetstress workload was executed for a period of 4 hours to ensure that the workload reached a constant state. Figure 13 shows the total Jetstress workload generated prior to any volume movements. The graphic shows the workload broken down by volume, database, and log. In this test, the load was run against two RAID 5 (7+1) 240 GB volumes and it can be seen that the IOPS peaks at 2,486 for SG1_Data volume and 2,447 for SG2_Data volume. The load for the Log volume was a single RAID 1 Log volume comprising of four mirrored hypervolumes of 30 GB, a total of 120 GB, and achieved 946 IOPS. Log volumes were not included within the FAST policy.


Figure 13. Jetstress IOPS for Database and Log volumes before FAST migration

The Jetstress results are detailed in the following report and show that the disk subsystem is performing extremely well with a database latency average well below the Microsoft-recommended 20 ms. This indicates that these particular Exchange databases could be moved onto a different RAID type or even a different tier of storage. It can also be seen that the total IOPS throughput was 4982, well in excess of the 1200 IOPS expected.


Microsoft Exchange Server Jetstress prior to migration

Performance Test Result Report Test Summary

Overall Test Result Pass

Machine Name LICOH013

Test Description

Test Start Time 10/31/2009 4:26:28 AM

Test End Time 10/31/2009 8:28:26 AM

Jetstress Version 08.02.0060.000

Ese Version 08.01.0240.005

Operating System Windows Server (R) 2008 Enterprise Service Pack 2 (6.0.6002.131072)

Performance Log C:\Program Files\Exchange Jetstress\RAID5_FC\Performance_2009_10_31_4_26_33.blg C:\Program Files\Exchange Jetstress\RAID5_FC\DBChecksum_2009_10_31_8_28_26.blg

Database Sizing and Throughput

Achieved I/O per Second 4982.775

Capacity Percentage 100%

Throughput Percentage 100%

Initial database size 412214919168

Final database size 435038486528

Database files (count) 2

Jetstress System Parameters

Thread count 16 (per-storage group)

Log buffers 9000

Minimum database cache 64.0 MB

Maximum database cache 512.0 MB

Insert operations 40%

Delete operations 30%

Replace operations 5%

Read operations 25%

Lazy commits 55%

Disk Subsystem Performance

LogicalDisk Avg. Disk sec/Read

Avg. Disk sec/Write

Disk Reads/sec Disk Writes/sec Avg. Disk

Bytes/Write

Database (J:) 0.005 0.006 1292.641 1203.415 (n/a)

Database (I:) 0.005 0.006 1286.673 1200.046 (n/a)

Log (L:) 0.000 0.000 0.000 948.949 5739.808

Host System Performance

Counter Average Minimum Maximum

% Processor Time 9.162 5.322 16.379

Available Mbytes 6561.663 6495.000 6647.000

Free System Page Table Entries 33558329.421 33557708.000 33558608.000

Transition Pages RePurposed/sec 0.000 0.000 0.000

Pool Nonpaged Bytes 62459200.000 60534784.000 63799296.000



Pool Paged Bytes 135726899.200 133464064.000 136257536.000

Database Page Fault Stalls/sec 0.000 0.000 0.000

Selection of migration targets

Selection of the migration targets is based on the relevant policy being applied. In the tested configuration, all devices were initially created within a single storage tier. All devices were from the one pool of 32 drives. All hypervolumes were configured with RAID 5 7+1 protection. This style of design is acceptable for Exchange Server mailbox database volumes where low IOPS and high user mailbox limits are used. Certainly latencies for the environment were within general best practice guidance.

As the monitoring time interval was set to analyze all devices within the system during the course of the workload run, and the policy engine was utilizing these statistics, a plan for movement was automatically generated by the FAST controller. The plan can either be viewed through Symmetrix Management Console through the Swap/Move list, or via the symfast CLI as shown in Figure 14. The suggested plan is uniquely identified by the Plan Identifier, and details the reason for the plan (Performance) and the suggested movement. Again, as the setting on the system was defined to be user approved, the plan shows a Plan State of NotApproved.

Figure 14. FAST generated performance movement plan

Plan Identifier


The targeted devices include the storage for both Exchange databases. The plan suggests that the devices be relocated to the SATA storage tier, and displays the associated disk group number and name. Additionally, the protection type is displayed, in this case R5 (7+1), as this was the protection type defined for this storage tier.

This workload is being migrated to a lower-performing storage tier based on the relative performance of all storage devices mapped to the policy – noted that the Group Attributes include the status that the migration is based on performance.

Scheduling migrations

Migrations that are either based on a user approval mode, or automatically approved, are subject to a further scheduling policy defined within the Optimizer environment. The migration time period, especially for up-tiering events, which are generally trying to address underperforming configurations, should be scheduled during hours when heavy production utilization is not typical. QoS mechanisms within the Symmetrix VMAX environment will ensure that user workloads are not significantly affected when they occur, but scheduling movements such that they complete outside production periods is recommended.

User approval of a given FAST plan may be either approved through Symmetrix Management Console in the Symmetrix Optimizer section, or by utilizing the symfast CLI, as shown in Figure 15. Note that the plan identifier needs to be supplied, and is the same plan identifier that was identified in Figure 14.

Figure 15. User approval of the suggested FAST plan

Approved plans will wait for the defined swap window to arrive before execution. Once executing, a migration cannot be terminated, and will run until concluded. Depending on volume sizes, and the number of migrations in process, the time for migration completion will vary. Symmetrix QoS mechanisms will prevent the migration from adversely affecting production workloads should the migration continue into normal work hours. It is also possible to apply manual priority settings to further limit the copy process by utilizing the symqos CLI and lowering the Mirror Pace setting.

While the migration is executing, it is possible to query the progress of the migration by again using SMC or the symfast CLI. In Figure 16 additional information is displayed for a plan that is executing a migration, including the time that the actual migration began.


Figure 16. FAST migration in process

Once all devices have fully migrated, the migration will automatically terminate, and the targeted devices will exist on the selected storage tier. The performance attributes of the storage tier will automatically apply to the devices.

Assessing the benefits of FAST

The migrations implemented by FAST moved the underutilized volumes onto a lower-performing storage tier. In the example configuration, two LUNs were migrated to SATA RAID 5 (7+1). The Jetstress test was rerun, and the results are detailed in Figure 17 and the following performance report.

Execution

Status


Figure 17. Jetstress IOPS for Database and Log volumes after FAST migration

Microsoft Exchange Server Jetstress post-migration

Performance Test Result Report Test Summary

Overall Test Result Pass

Machine Name LICOH013

Test Description

Test Start Time 11/1/2009 3:43:04 AM

Test End Time 11/1/2009 7:47:34 AM

Jetstress Version 08.02.0060.000

Ese Version 08.01.0240.005

Operating System Windows Server (R) 2008 Enterprise Service Pack 2 (6.0.6002.131072)

Performance Log C:\Program Files\Exchange Jetstress\RAID5_FC\Performance_2009_11_1_3_43_8.blg C:\Program Files\Exchange Jetstress\RAID5_FC\DBChecksum_2009_11_1_7_47_34.blg

Database Sizing and Throughput

Achieved I/O per Second 1265.255

Capacity Percentage 100%

Throughput Percentage 100%

Initial database size 435040583680

Final database size 441078284288

Database files (count) 2


Jetstress System Parameters

Thread count 16 (per-storage group)

Log buffers 9000

Minimum database cache 64.0 MB

Maximum database cache 512.0 MB

Insert operations 40%

Delete operations 30%

Replace operations 5%

Read operations 25%

Lazy commits 55%

Disk Subsystem Performance

LogicalDisk Avg. Disk sec/Read

Avg. Disk sec/Write

Disk Reads/sec Disk Writes/sec Avg. Disk

Bytes/Write

Database (J:) 0.016 0.007 327.230 306.453 (n/a)

Database (I:) 0.015 0.007 326.707 304.866 (n/a)

Log (L:) 0.000 0.000 0.000 349.099 4472.561

Host System Performance


% Processor Time 2.102 1.099 7.835

Available MBytes 6446.060 6401.000 6476.000

Free System Page Table Entries 33558552.924 33558141.000 33558672.000

Transition Pages RePurposed/sec 0.000 0.000 0.000

Pool Nonpaged Bytes 63622314.667 63483904.000 63787008.000

Pool Paged Bytes 138968900.267 138657792.000 139091968.000

Database Page Fault Stalls/sec 0.000 0.000 0.000

FAST VP applied FAST VP can only be configured in an Automatic mode in which the system will continually gather statistics, analyze, and execute moves and swaps to maintain optimal configuration based on policy. To validate the mechanisms and show the efficacy of the FAST VP solution for a Microsoft Exchange Server environment, the following testing was conducted. A user workload was generated against an Exchange Server mailbox database. Storage tiers were defined within the environment, and a FAST VP policy was defined for the Exchange Server mailbox databases’ LUNs. FAST VP monitoring was implemented. The extents identified by the FAST VP control to move were moved between the appropriate thin pools. The system continued monitoring for the period of the test and extents moved if and when necessary, dependent on the I/O load.

FAST VP test methodology and configuration

For all testing, Microsoft Windows Server 2008 R2 and Jetstress for Microsoft Exchange Server 2010 were used.


A Hyper-V environment was built comprising of four virtual servers. The Hyper-V server was allocated storage from a single VMAX storage group, comprising of 20 LUNs. The layout of the LUNs presented to the Hyper-V server, and subsequently to the virtual machines, is shown in Table 3.

Table 3. Hyper-V server disk configuration

Volume Host device

Sym Dev

Drive type RAID

Allocation (KB)

Jetstress Threads Server

DB1 1 00D4 SATA 6 (14+2) 245813 1 EX2010-VMBX1

DB2 2 00D5 SATA 6 (14+2) 245813 1 EX2010-VMBX1

DB3 3 00D6 SATA 6 (14+2) 245813 1 EX2010-VMBX1

DB4 4 00D7 SATA 6 (14+2) 245813 5 EX2010-VMBX2

DB5 5 00D8 SATA 6 (14+2) 245813 5 EX2010-VMBX2

Log 1 6 00DE SATA 6 (14+2) 245813 1 EX2010-VMBX1

Log 2 7 00DF SATA 6 (14+2) 61454 1 EX2010-VMBX1

Log 3 8 00E0 SATA 6 (14+2) 61454 1 EX2010-VMBX1

Log 4 9 00E1 SATA 6 (14+2) 61454 5 EX2010-VMBX2

Log 5 10 00E2 SATA 6 (14+2) 61454 5 EX2010-VMBX2

DB6 11 00E8 FC 10K 5 (3+1) 245813 2 EX2010-VMBX4

DB7 12 00E9 FC 10K 5 (3+1) 245813 2 EX2010-VMBX4

DB8 13 00EA FC 10K 5 (3+1) 245813 2 EX2010-VMBX4

DB9 14 00EB FC 10K 5 (3+1) 245813 5 EX2010-VMBX3

DB10 15 00EC FC 10K 5 (3+1) 245813 5 EX2010-VMBX3

Log 6 16 00F2 FC 10K 5 (3+1) 61454 2 EX2010-VMBX4

Log 7 17 00F3 FC 10K 5 (3+1) 61454 2 EX2010-VMBX4

Log 8 18 00F4 FC 10K 5 (3+1) 61454 2 EX2010-VMBX4

Log 9 19 00F5 FC 10K 5 (3+1) 61454 5 EX2010-VMBX3

Log 10 20 00F6 FC 10K 5 (3+1) 61454 5 EX2010-VMBX3

The simulated environment was Jetstress for Exchange Server 2010. As such, it represents the workload of an Exchange Server 2010 Mailbox Server Role environment. Simulated Exchange database operations are carried out to perform a range of processing tasks. The workload generates a significant read/write requirement against the mailbox database. The Jetstress parameters were as shown in the following table.


Table 4. Jetstress parameters

Jetstress Parameter Value selected

Define Test Scenario Test disk subsystem throughput

Select Capacity and Throughput Size database storage capacity %age = 50

Suppress tuning and use thread count (per database) = see Table 3

Select Test Type Performance

Run Background Database Maintenance1

Test Duration 12 hours

Define Database Configuration Number of Databases = 3/2 (dependent on server)

Number of copies per databases = 2

Thin pool configuration

Two thin pools were created, one consisting of DATA devices taken from the 300 GB 10k Fibre Channel disk group and one consisting of DATA devices taken from the 1 TB SATA disk group. The thin pools were created using the following shown in the table.

Table 5. Thin pool configuration

Pool name Disk type RAID type Number of DATA devices

Bound thin devices

EX2010_FC_R5 300GB FC 10k 5 (3+1) 80 10

Ex2010_SATA 1TB SATA 6 (14+2) 200 20

All bound thin devices were added to a single VMAX storage group, EX2010_FAST_VP_TEST, and presented to the Hyper-V server.

The tests outlined for FAST VP in this white paper were performed on the following configuration:

Configuration aspect Description

Storage controller Symmetrix VMAX 1194

Microcode 5875.xx.xx

Microsoft Windows Microsoft Windows Server 2008 R2

Jetstress Executable 14.01.0180.003

Jetstress Core Engine 14.01.0180.003

Tier 1 300 GB 10k Fibre Channel

Tier 2 1 TB SATA

1 Two sets of tests were carried out with and without Background Database Maintenance (BDM) selected to determine the effect of the constant monitoring BDM imposes on the disk.


Configuring FAST VP for the environment

When initially configured, the storage allocations for all LUNs were made from a single pool of physical spindles with a single RAID protection scheme. This is synonymous with a storage type or tier, as previously defined. The Symmetrix VMAX array contained a number of different technologies including 300 GB 10k rpm drives and 1 TB SATA drives. Using these differing technologies and available RAID levels, it is possible to construct the various storage tiers that may be applied.

Figure 18. SMC FAST Configuration Wizard

In the tested environment, two Virtual Pool Tiers were defined, as shown in Figure 19. A type (or tier) of storage may differ on the technology implemented (the physical drive characteristics) or by the RAID protection scheme implemented. The tiers created in this configuration varied both in terms of the underlying technology, where the tier “EX2010_FAST_VP_FCR5” was defined as RAID 5 (3+1) protection on 300 GB 10k rpm devices, and EX2010-FAST_SATA was defined as being a RAID 6 (14+2) protection scheme on 1 TB SATA drives. These two tiers were subsequently used to define a policy for the Exchange Server environment.


Figure 19. Defining Virtual Pool Tiers within storage types (also known as storage tiers) within SMC

The storage tiers were applied to create a policy named “EX2010”. This policy defined the applicable storage tiers, and applied capacity percentages to the various tiers. The percentages define how much of the storage space used by the storage group defined in the policy can be utilized from each of the tiers. In an Exchange Server environment the movement between storage tiers will always be dependent on the performance and not capacity, therefore the percentage allocation for each storage type must be 100 percent and is shown in Figure 20.


Figure 20. FAST VP policy definition within SMC

The allocation of a storage group to a defined tier binds the policy to the devices contained with the storage group. The storage group is defined when implementing Auto-provisioning Groups, and defines the storage devices that will be available to a host when bound to a view. For the tested workload the storage group name was “EX2010_FAST_VP_TEST” and it defined all storage devices that were presented to the Exchange Server mailbox servers mentioned in Table 3. In Figure 21 the storage group is displayed within its FAST VP policy and with its component devices.


Figure 21. Allocating a storage group to a policy in SMC

To complete the implementation of the FAST mechanisms, it is necessary to set appropriate time periods to collect statistics for workload analysis. Statistics collection is defined within the Symmetrix Optimizer environment or the FAST Settings, and may also be set through Symmetrix Management Console, as shown in Figure 22.

Figure 22. FAST VP Move Time Window settings in SMC

It is only possible to configure FAST VP to execute thin data movement automatically; the other option is set to Off. If Automatic mode is set, FAST suggested movements will be executed. In the tested configuration, FAST VP Specific Settings for Thin Data Movement Mode was left in automatic mode, as shown in Figure 23.


Figure 23. FAST VP Thin Data Movement Mode setting

Having defined the storage group association to the storage types through the policy, and scheduling the application time periods within Symmetrix Optimizer/FAST, the environment was configured appropriately. To validate the selection of devices, and planned movement, the Windows operating system and Exchange Server Jetstress monitoring were also configured.

Monitoring workload performance

Microsoft Exchange Server administrators and Windows Server administrators will typically utilize Jetstress to measure overall disk subsystem and system performance. In the tests run to stimulate movement between storage tiers a series of tests were run, using differing thread counts for each disk type, on the disks available to determine the point where a Jetstress run would pass or fail. The initial pass/fail results were:

300 GB FC 10k RAID 5 (3+1): 5 threads – Pass

6 threads – Fail

1 TB SATA RAID 6 (14+2): 2 threads – Pass

3 Threads – Fail


These limits were then used to run the next set of Jetstress parameters, used when monitoring the workload, and are detailed in Table 3 and Table 4. Two sets of tests were run to determine whether or not the BDM process had a significant effect on the movement of the extents due to the continuous maintenance processes on the database that BDM imposes.

FAST VP pool allocation

A Jetstress workload was executed for a period of 12 hours to ensure that the workload reached a constant state. A script, using the CLI command

symcfg –sid 1194 show –pool EX2010_FC_R5 –thin –detail –mb

symcfg –sid 1194 show –pool EX2010_SATA –thin –detail –mb

was used in a loop (next figure), to capture device pool allocation. This was run at 20-minute intervals to show the movement of data between the two thin pools over the period of the Jetstress test run.

Figure 24. SYMCLI command to show thin pool allocation


Thin data movement without BDM

The outputs from these scripts at the start and end of the testing without BDM were used to calculate the amount of data movement between the storage tiers and are summarized for specific database Symmetrix Devices (Symm Dev) in the following charts:

Figure 25. Allocation of Sym Dev 00D7 within the EX2010_SATA thin pool without BDM

Figure 25 highlights the difference in pool allocation of the database DB4 residing on Sym Dev 00D7, bound to the EX2010_SATA thin pool. As a result of the FAST VP movement it can be seen that 41 percent of the data has moved up to the EX2010_FC_R5 thin pool. This is expected as this database was running at five threads, three threads above the baseline.


Figure 26. Allocation of Sym Dev 00EA within the EX2010_FC_R5 thin pool without BDM

Figure 26 highlights the difference in allocation for the DB8 database on Symm Dev 00EA within the EX2010_FC_R5 thin pool as a result of the FAST VP movement. It can be seen that 59 percent of the data residing on Symm Dev 00EA has been moved down a storage tier to the EX2010_SATA thin pool. This is expected as this database was running at two threads, three threads below the baseline.


Thin data movement with BDM

Figure 27. Allocation of Sym Dev 00D7 within the EX2010_SATA thin pool with BDM

Figure 27 highlights the difference in pool allocation of the database DB4 residing on Sym Dev 00D7, bound to the EX2010_SATA thin pool. As a result of the FAST VP movement it can be seen that 40 percent of the data has moved up to the EX2010_FC_R5 thin pool. This is expected as this database was running at five threads, three threads above the baseline.


Figure 28. Allocation of Sym Dev 00Ea within the EX2010_FC_R5 thin pool with BDM

Figure 28 highlights the difference in allocation for the DB8 database on Symm Dev 00EA within the EX2010_FC_R5 thin pool as a result of the FAST VP movement. It can be seen that 58 percent of the data residing on Symm Dev 00EA has been moved down a storage tier to the EX2010_SATA thin pool. This is expected as this database was running at two threads, three threads below the baseline.

It can be seen by the data that there is little significant difference in the amount of data extents moved by FAST VP when running or not running the BDM process.

FAST VP Jetstress results

All of the Jetstress tests completed successfully and summaries of the relevant Jetstress test results are detailed in Table 6 and Table 7.

Table 6. Summary of Jetstress test results when running without BDM

Tests without BDM EX2010_VMBX1 EX2010_VMBX2 EX2010_VMBX3 EX2010_VMBX4 DB1 DB2 DB3 DB4 DB5 DB6 DB7 DB8 DB9 DB10 Transactional I/O Performance I/O Database Reads Average Latency (msec)

8.27 8.67 8.41 12.14 16.69 11.41 11.40 8.42 8.42 8.41

I/O Database Writes Average Latency (msec)

3.19 3.16 3.138 6.16 6.05 6.66 6.53 6.40 6.31 6.29

I/O Log Reads Average Latency (msec)

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

I/O Log Writes Average Latency (msec)

1.55 1.56 1.55 1.07 1.08 1.45 1.45 1.06 1.06 1.06

Background Database Maintenance I/O Performance Database Maintenance IO Reads/sec

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00


Total I/O Performance I/O Database Reads Average Latency (msec)

8.27 8.67 8.41 12.14 16.69 11.41 11.40 8.42 8.42 8.41


3.19 3.16 3.13 6.16 6.05 6.66 6.53 6.40 6.31 6.29


0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00


1.55 1.56 1.55 1.07 1.08 1.45 1.45 1.06 1.06 1.06

Table 7. Summary of Jetstress test results when running with BDM

Tests with BDM EX2010_VMBX1 EX2010_VMBX2

EX2010_VMBX3 EX2010_VMBX4

DB1 DB2 DB3 DB4 DB5 DB6 DB7 DB8 DB9 DB10 Transactional I/O Performance I/O Database Reads Average Latency (msec)

8.02 7.99 7.90 15.76 19.41 13.95 10.01 8.78 8.81 8.80


3.28 3.18 3.29 5.07 5.02 5.52 5.48 4.24 4.26 4.26


0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00


1.73 1.71 1.71 1.19 1.21 1.56 1.57 1.20 1.20 1.20

Background Database Maintenance I/O Performance Database Maintenance IO Reads/sec 29.16 29.16 29.19 25.68 24.74 26.18 26.16 29.05 29.03 29.03

Total I/O Performance I/O Database Reads Average Latency (msec)

8.02 7.99 7.90 15.76 19.41 13.95 10.01 8.78 8.81 8.80


3.28 3.18 3.29 5.07 5.02 5.52 5.48 4.24 4.26 4.26


2.15 2.11 2.20 2.53 2.54 2.72 2.74 2.55 2.46 2.54


1.73 1.71 1.71 1.19 1.23 1.56 1.57 1.20 1.20 1.20

The Jetstress results tabled above show that even when running FAST VP there is little impact upon the performance of the Exchange Server 2010 mailbox server. It also shows that by using FAST VP more sustained I/O throughput can be achieved. In this case it is highlighted by increasing the thread count on the SATA drives from a pass on two threads and fail on three threads to being able to achieve a pass running with five threads on server EX2010-VMBX2.

Conclusion A Microsoft Exchange Server (2007 or 2010) mailbox database environment will often exhibit a perceived skewed workload across any given Exchange server due to the nature of the users’ email workloads, I/O profiles, and concurrency. The storage design for any given Exchange Server Mailbox Server Role server is not always an exact science and while the number of physical disk drives can be derived from


various sources, the exact nature of the workload cannot be fully analyzed until the server has been commissioned into production and detailed monitoring carried out. There can be one of three outcomes to this analysis:

The I/O performance and latency statistics for the given number of physical drives and the layout are accurate and are performing to expectations and the storage within the array is utilized to the maximum.

The I/O performance is low and the drive latency is too high, resulting in users’ email performance experience being compromised.

The I/O performance and latency statistics for the given number of physical drives and the layout are accurate and are performing to expectations; however, after detailed analysis after commissioning into production it is found that the workload could have been achieved by using fewer physical drives.

For the two scenarios where the storage subsystem is either performing below par or has been overconfigured, the use of FAST, FAST VP, and Symmetrix VMAX can enable the system to be better balanced to meet overall storage I/O requirements. FAST and FAST VP will also enable Exchange and storage administrators to use storage types within a storage array to allow the use of the correct class of storage for the correct class of email users.

For the current (Exchange Server 2010) and future versions of Exchange Server the amount of storage capacity needed to fulfill users’ mailbox requirements of greater than 10 GB lends itself to the use of Virtual Provisioning for the initial deployment and to grow, in staged increments, when the need arises. Whilst the figure of a “10 GB” user mailbox is currently the “standard” there will be many users within organizations that will never reach that limit. Thus having to provision the total capacity upon initial deployment is less cost-effective and Symmetrix VMAX Virtual Provisioning in conjunction with FAST VP is the ideal fit.

Storage Tiering for Microsoft Exchange Server and EMC ... · PDF fileEMC Symmetrix VMAX with Enginuity 5875 ... Thin pool configuration ... directly or remotely to a Symmetrix VMAX

Documents