DESIGN GUIDE EMC VSPEX FOR VIRTUALIZED ORACLE DATABASE 12c OLTP Enabled by EMC VNXe and EMC Data Protection VMware vSphere 5.5 Red Hat Enterprise Linux 6.4 EMC VSPEX Abstract This Design Guide describes how to design virtualized Oracle database resources in an EMC ® VSPEX ® Proven Infrastructure for VMware vSphere that uses EMC VNXe ® and EMC Data Protection storage. This document also describes how to size Oracle Database 12c on VSPEX, allocate resources following best practices, and use all the benefits that VSPEX offers. February 2015
68
Embed
EMC VSPEX FOR VIRTUALIZED ORACLE DATABASE 12c OLTP - Dell EMC · Oracle Database 12c ... EMC VSPEX for Virtualized Oracle Database 12c OLTP 9 ... Qualified by EMC to sell, install,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DESIGN GUIDE
EMC VSPEX FOR VIRTUALIZED ORACLE DATABASE 12c OLTP Enabled by EMC VNXe and EMC Data Protection
VMware vSphere 5.5 Red Hat Enterprise Linux 6.4
EMC VSPEX
Abstract
This Design Guide describes how to design virtualized Oracle database resources in an EMC® VSPEX® Proven Infrastructure for VMware vSphere that uses EMC VNXe® and EMC Data Protection storage. This document also describes how to size Oracle Database 12c on VSPEX, allocate resources following best practices, and use all the benefits that VSPEX offers.
February 2015
2 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled By VNXe and EMC Data Protection Design Guide
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.
The information in this publication is provided “as is”. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
EMC VSPEX for Virtualized Oracle Database 12c Enabled by VNXe and EMC Data Protection Design Guide
Figure 2. Architecture of the validated infrastructure .......................................... 18
Figure 3. Example: Storage layout of Oracle 12c Database on VMware ............... 34
Figure 4. Storage layout example: Oracle Server for VNX series with FAST VP/Cache enabled ............................................................................... 36
Figure 5. init.ora Parameters from the AWR Report ............................................. 55
Figure 6. IOStat by function summary from the AWR Report ................................ 56
Figure 7. Foreground Wait Event from the AWR report ......................................... 56
Figure 8. Transactions in a load profile from the AWR report ............................... 57
Table 13. Recommendations for the storage layout example: Large-sized Oracle 12c server ........................................................................................... 30
Table 14. Oracle Sever storage pools example .................................................... 34
Table 15. Database layout for a consolidated Oracle environment example ........ 42
Table 16. High-level steps for application verification ......................................... 46
8 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Purpose of this guide
EMC® VSPEX® Proven Infrastructures are optimized for virtualizing business-critical applications. VSPEX provides modular solutions built with technologies that enable faster deployment, greater simplicity, greater choice, higher efficiency, and lower risk.
VSPEX provides partners with the ability to design and implement the virtual assets that are required to support a fully integrated virtualization solution for an Oracle relational database management system (RDBMS) on a VSPEX private cloud infrastructure.
The VSPEX for virtualized Oracle infrastructure provides customers with a modern system capable of hosting a virtualized database solution that is scalable and delivers a constant performance level. This solution uses VMware vSphere backed by the EMC VNXe® storage array. The compute and network components, while vendor-definable, are designed to provide redundancy and sufficient power to handle the processing and data needs of the virtual machine environment.
This Design Guide describes how to design the resources necessary to deploy Oracle Database 12c in a VSPEX Proven Infrastructure that uses EMC VNXe storage arrays. This guide also describes using the EMC VSPEX Sizing Tool to select the correct VSPEX Proven Infrastructure. This guide applies to Oracle online transaction processing (OLTP) workloads but does not cover data warehousing workloads.
Business value
Software for database management systems is used by many different types of businesses. Despite the increasing market share of other data management tools, growth in sales is expected to continue and will accelerate as customers continue to diversify their infrastructures and supporting technologies and use more hardware and software appliances and configurations.
This VSPEX Proven Infrastructure is focused on helping EMC partners understand the value that the VNXe series and Oracle bring to customers who often have growing, isolated IT environments running server-centric applications, facing increased Oracle backup and recovery issues.
This VSPEX solution is designed to meet Oracle database challenges and enable customers to increase performance, scalability, reliability, and automation. By consolidating database applications on EMC VNXe, customers can use a single centralized storage platform to manage exploding data growth more effectively. This solution has been sized and proven by EMC to:
Deploy systems faster, saving time and effort with EMC Proven Solutions
Increase performance and scalability
Minimize storage requirements and reduce costs
Chapter 1: Introduction
9 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Scope
This Design Guide describes how to plan and design a VSPEX Proven Infrastructure for VMware vSphere virtualized Oracle 12c databases. The guide assumes that a VSPEX Private Cloud already exists in the customer environment.
This guide also shows how to use the VSPEX Sizing Tool to size Oracle 12c Release 1 (12.1) on the VSPEX Proven Infrastructures, how to use best practices to efficiently allocate resources, and how to use all the benefits that VSPEX offers.
The EMC Data Protection solutions for Oracle Server 12c data protection are described in a separate document, EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2 Design and Implementation Guide.
Audience
This guide is intended for internal EMC personnel and qualified EMC VSPEX partners. It assumes that VSPEX partners who intend to deploy this solution are:
Qualified by EMC to sell, install, and configure the EMC VNX family of storage systems
Qualified to sell, install, and configure the network and server products required for VSPEX Proven Infrastructures
Certified for selling VSPEX Proven Infrastructure
Partners who plan to deploy the solution must also have the necessary technical training and background to install and configure:
VMware vSphere 5.5
Redhat Enterprises Linux 6.4
Oracle Database 12c
This guide provides external references where applicable. EMC recommends that partners who implement this solution are familiar with these documents. See Essential reading and Chapter 7: Reference Documentation for details.
Terminology
Table 1 lists the terminology used in the guide.
Table 1. Terminology
Term Definition
AWR Automatic Workload Repository is a powerful monitoring utility bundled with Oracle Database 10g and later releases
dNFS Direct Network File System client can be used to provide access to Network File System (NFS) storage devices
Chapter 1: Introduction
10 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Term Definition
FAST™ Cache Fully Automated Storage Tiering Cache is an EMC storage feature that uses flash drives as an extension to system cache
FAST VP Fully Automated Storage Tiering for Virtual Pools, is an EMC storage feature that provides automatic storage tiering at the sub-LUN level
OLTP Online transaction processing, a class of systems that manage applications for data entry and retrieval transaction processing
PowerCLI A Windows PowerShell interface to the VMware vSphere and vCloud APIs
SGA System Global Area is a group of shared memory structures that contains data and control information from one Oracle database instance
PGA Process Global Area is memory specific to an operating process or thread that is not shared by other processes or threads on the system
VMDK VMware Virtual Machine Disk is an open file format that is a container for virtual hard disk drives for virtual machines
VMFS VMware Virtual Machine File System is a cluster file system that uses storage virtualization for multiple installations of VMware ESX server
Chapter 2: Before You Start
11 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
12 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Deployment workflow
Refer to the process flow in Table 2 to design and implement a VSPEX for virtualized Oracle Database 12c OLTP solution.
Table 2. VSPEX workflow for virtualized Oracle Database deployment
Step Action
1 Use the VSPEX for Oracle Database 12c OLTP qualification worksheet to collect user requirements. The one-page qualification worksheet is in Appendix A of this Design Guide.
2 Use the EMC VSPEX Sizing Tool to determine the recommended VSPEX Proven Infrastructure for your Oracle Database 12c OLTP solution, based on the user requirements collected in Step 1.
For more information about the Sizing Tool, refer to the VSPEX Sizing Tool on the EMC Business Value Portal.
Note: You need to register the first time you access the tool. If the VSPEX Sizing Tool is not available, you can manually size the application using the guidelines in Appendix B High-level Oracle Database Server Sizing Logic and Methodology.
3 Use this Design Guide to determine the final design for your VSPEX solution.
Note: Consider all application requirements and not only the requirements for the Oracle database.
4 Select and order the correct VSPEX Proven Infrastructure. Refer to the appropriate VSPEX Proven Infrastructure document in Essential reading for guidance.
5 Deploy and test your VSPEX solution. Refer to the appropriate VSPEX Implementation Guide in Essential reading for guidance.
13 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Essential reading
EMC recommends that you read the following documents before implementing the solution described in this document. The documents are available from the VSPEX space in the EMC Community Network or from EMC.com and the VSPEX Partner Portal. If you do not have access to a document, contact your EMC representative.
Refer to the following VSPEX Implementation Guide:
EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide
Refer to the following VSPEX Proven Infrastructure Guide:
EMC VSPEX Private Cloud VMware vSphere 5.5 for up to 200 virtual machines Enabled by Microsoft Windows Server 2012 R2, EMC VNXe3200, and EMC Powered Backup Proven Infrastructure Guide
Refer to the following EMC Data Protection for VSPEX guide:
EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2 Design and Implementation Guide
16 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Overview
This solution offers users a powerful approach to deploying Oracle databases on EMC VNXe systems. By using advanced VNXe storage functionality such as FAST Cache or FAST VP, users can achieve increased performance for their Oracle deployments. With these advanced data features, the VNXe series not only reduces the initial cost of the Oracle database deployment, but also reduces the complexity associated with day-to-day data management by automating the complex and time-consuming storage tiering process.
EMC VNXe with Oracle:
Automatically delivers the highest input/output operations per second (IOPS) and lowest response times at the lowest cost
Provides automated storage tiering, requiring no manual tuning
Supports separate workloads with different I/O profiles
Supports Network File System(NFS), Fibre Channel(FC), and other protocols depending on a customer’s specific needs
Provides greater virtualization integration and lower Oracle licensing costs through server and database consolidation
The solution enables customers to quickly and consistently deploy a virtualized Oracle Database 12c in the VSPEX Proven Infrastructure. This solution was validated using NFS and FC for data storage.
This Design Guide can help EMC personnel and qualified EMC VSPEX Partners to deploy a simple, effective, and flexible Oracle Database 12c solution on a VSPEX Proven Infrastructure for their customers.
EMC VSPEX Proven Infrastructure
EMC has joined forces with IT infrastructure providers to create a complete virtualization solution that accelerates the deployment of the private cloud. VSPEX enables faster deployment, greater simplicity and choice, greater efficiency, and lower risk. Validation by EMC ensures predictable performance and enables customers to select technology that uses their existing IT infrastructure while eliminating planning, sizing, and configuration burdens. VSPEX provides a virtual infrastructure for customers who want the simplicity that is characteristic of truly converged infrastructures, while simultaneously gaining more choice for individual stack components.
VSPEX solutions are proven by EMC and packaged and sold exclusively by EMC channel partners. VSPEX provides channel partners with more opportunity, a faster sales cycle, and end-to-end enablement. By working together, EMC and its channel partners can deliver proven infrastructure that accelerates the journey to the cloud for even more customers.
VSPEX Proven Infrastructure, as shown in Figure 1, is a modular, virtualized infrastructure that is validated by EMC and delivered by EMC’s VSPEX partners. VSPEX
Chapter 3: Solution Overview
17 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
includes a virtualization layer , server, network, and EMC storage and data protection, designed by EMC to deliver reliable and predictable performance.
Figure 1. VSPEX Proven Infrastructure
VSPEX provides the flexibility to select network, server, and virtualization technologies that fit a customer’s environment to create a complete virtualization solution. VSPEX delivers faster deployment for EMC partner customers, with greater simplicity and efficiency, more choice, and lower risk to a customer’s business.
Chapter 3: Solution Overview
18 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
For more information about VSPEX Proven Infrastructures, refer to EMC VSPEX Private Cloud VMware vSphere 5.5 for up to 200 virtual machines Enabled by Microsoft Windows Server 2012 R2, EMC VNXe3200, and EMC Powered Backup Proven Infrastructure Guide
Solution architecture
Figure 2 shows the architecture that characterizes the infrastructure validated for an Oracle database 12c overlay on a VSPEX infrastructure. The data protection components of the solution provide data protection to the Oracle databases.
Figure 2. Architecture of the validated infrastructure
To validate this solution, we1 completed the following tasks:
Deployed all Oracle Database 12c servers as virtual machines on VMware vSphere 5.5.
Used the VSPEX sizing tool for Oracle Database 12c to determine the number of detailed compute resources for each Oracle Database 12c database. We provided three Oracle sizing options (small, medium, and large) to help size the customers’ environment and select the options that best suit their needs.
1 In this guide, “we” refers to the EMC Solutions engineering team that validated the solution.
Chapter 3: Solution Overview
19 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Determined the recommended storage layout for Oracle Database 12c and the virtual infrastructure pool in the VNXe series storage arrays.
Note: The minimum Oracle version for this solution is Release 1 (12.1). We refer to this as 12c throughout this document.
Key components
This section provides an overview of the key technologies used in this solution:
Oracle Database 12c
VMware vSphere 5.5
EMC VNXe3200™
Red Hat Enterprise Linux 6.4
EMC Data Protection solutions
Oracle Database 12c is the latest version of Oracle database technology. Oracle 12c is available in a variety of editions that are tailored to meet the business and IT needs of an organization. In this solution we will consider:
Oracle Database 12c Release 1 Standard Edition (SE)
Oracle Standard Edition (Oracle SE) is an affordable, full-featured data management solution that is ideal for all companies. It is available on single or clustered servers and can be licensed on a maximum capacity of four processor sockets, regardless of core count. The SE license includes Oracle Real Application Clusters (RAC) as a standard feature with no additional cost.
Oracle Enterprise Edition (Oracle EE) delivers industry-leading performance, scalability, security, and reliability on a choice of clustered or single servers running Windows, Linux, or UNIX. Oracle Database 12c EE supports advanced features, either included or as extra-cost options, that are not available with Oracle Database 12c SE. For example, security features such as Virtual Private Database are included with Oracle Database 12c EE, as well as data warehousing options such as partitioning and advanced analytics.
Note: The Oracle Database edition can affect the licensing cost and the size and number of VMware ESXi clusters that you can configure. How to Find the Oracle Processor Core Factor Multipliers (Doc ID 1330016.1) on My Oracle Support provides more information about Oracle processor licensing.
Oracle Multitenant is a new feature for the Oracle Database 12c Enterprise Edition that helps to reduce IT costs by consolidation, provisioning, upgrades, and more. Oracle Multitenant is supported by a new architecture that enables a single super database to hold many sub-databases. Oracle Multitenant is fully interoperable with Oracle Real Application Cluster (RAC).
Introduction
Oracle Database 12c
Chapter 3: Solution Overview
20 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
VMware vSphere 5.5 transforms a computer’s physical resources by virtualizing the CPU, RAM, hard disk, and network controller. This transformation creates fully functional virtual machines that run isolated and encapsulated operating systems and applications in the same way as physical computers.
VMware High Availability (HA) provides easy-to-use and cost-effective high availability for applications running on virtual machines. The VMware vSphere vMotion and VMware vSphere Storage vMotion features of vSphere 5.5 enable the seamless migration of virtual machines and stored files from one vSphere server to another, with minimal or no performance impact. Coupled with VMware vSphere Distributed Resource Scheduler (DRS) and VMware vSphere Storage DRS, virtual machines have access to the appropriate resources at any point in time through load balancing of compute and storage resources.
The VMware Native Multipathing Plug-In (NMP) is the default module in vSphere that is used for multipathing. NMP provides a default path selection algorithm based on the array type and it associates the physical paths with a specific storage device or logical unit number (LUN). The specific details for handling path failover for a particular storage array are delegated to a Storage Array Type Plug-In (SATP). The specific details for determining which physical path is used to issue an I/O request to a storage device are handled by a Path Selection Plug-In (PSP). SATPs and PSPs are sub-plug-ins within the NMP module.
EMC VNXe3200 is an affordable, flash-optimized, unified storage platform that delivers innovation and enterprise capabilities for file and block storage in a single, scalable, and easy-to-use solution. Ideal for mixed workloads in physical or virtual environments, VNXe3200 combines powerful and flexible hardware with advanced efficiency, management, and protection software to meet the demanding needs of today’s virtualized application environments.
Features and enhancements
VNXe3200 includes many features and enhancements designed and built on the success of the midrange EMC VNX family. These features and enhancements include:
Greater efficiency with a flash-optimized hybrid array
More capacity with multicore optimization with EMC Multicore Cache, Multicore RAID, and Multicore FAST Cache (MCx™)
Easier administration and deployment with VNXe Base Software components including Monitoring, Reporting, and Unified Snapshots
VMware ecosystem integration
Unified multiprotocol support for FC, iSCSI, NFS, and CIFS
VSPEX is built with the next generation of VNXe to deliver even greater efficiency, performance, and scalability than ever before.
Flash-optimized hybrid array
VNXe3200 is a flash-optimized hybrid array that provides automated tiering to deliver the best performance to your critical data, while intelligently moving less frequently accessed data to lower-cost disks.
VMware vSphere 5.5
EMC VNXe3200
Chapter 3: Solution Overview
21 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
In this hybrid approach, a small percentage of flash drives in the overall system can provide a high percentage of the overall IOPS. VNXe3200 takes full advantage of the low latency of flash to deliver cost-saving optimization and high performance scalability. EMC Fully Automated Storage Tiering Suite (FAST Cache and FAST VP) tiers both block and file data across heterogeneous drives. The suite boosts the most active data to the flash drives, ensuring that customers never have to make concessions for cost or performance.
Data is generally accessed most frequently at the time it is created; therefore, new data should first be stored on flash drives to provide the best performance. As the data ages and becomes less active over time, FAST VP can tier the data from high-performance drives to high-capacity drives automatically, based on customer-defined policies. FAST Cache dynamically absorbs unpredicted spikes in system workloads. FAST Cache can provide immediate performance enhancement by promoting suddenly active data from slower high-capacity drives to speedier flash drives. All VSPEX use cases benefit from the increased efficiency.
VSPEX Proven Infrastructures deliver private cloud, end-user computing, and virtualized application solutions. With VNXe3200, customers can realize an even greater return on their investment.
VNX Intel MCx code path optimization
The advent of flash technology has been a catalyst for changing the requirements of midrange storage systems. EMC redesigned the midrange storage platform to efficiently optimize multicore CPUs to provide the most efficient storage system at the lowest cost in the market.
MCx distributes all VNXe data services across all cores, and can dramatically improve the file and block performance for transactional applications such as databases or virtual machines over network-attached storage (NAS) and Fibre Channel (FC).
VNXe base software
The enhanced VNXe base software extends the EMC Unisphere® easy-to-use interface to include VNX monitoring and reporting for validating performance and anticipating capacity requirements. The suite also includes EMC Unisphere Central® for centrally managing thousands of VNX and VNXe systems.
Virtualization and ecosystem management
You can use the following applications for virtualization and ecosystem management:
VMware vSphere Storage APIs for Storage Awareness VMware vStorage APIs for Storage Awareness (VASA) is a VMware-defined API that displays storage information through vCenter. Integration between VASA technology and VNX makes storage management a seamless experience in a virtualized environment.
VMware vSphere Storage APIs for Array Integration VMware vSphere Storage APIs for Array Integration (VAAI) offloads VMware storage-related functions from the server to the storage system, enabling more efficient use of server and network resources for increased performance and consolidation.
Chapter 3: Solution Overview
22 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
EMC Storage Analytics for VNXe EMC Storage Analytics (ESA) for VNXe delivers a storage-only version of VMware vCenter Operations with a built-in VNXe connector that provides detailed analytics, relationships, and unique icons for EMC arrays and components.
EMC Virtual Storage Integrator EMC Virtual Storage Integrator (VSI) is a free plug-in for VMware vCenter that is available to all VMware users with EMC storage. VSPEX customers can use VSI to simplify management of virtualized storage. VMware administrators can gain visibility into their VNXe storage using the same familiar vCenter interface to which they are accustomed.
With VSI, IT administrators can do more work in less time. VSI offers unmatched access control that enables you to efficiently manage and delegate storage tasks with confidence. With VSI, you can perform daily management tasks with up to 90 percent fewer clicks and up to 10 times higher productivity.
Red Hat Enterprise Linux is a versatile platform for x86 and x86-64 that can be deployed on physical systems, as a guest on the major hypervisors, or in the cloud. It supports all leading hardware architectures with compatibility across releases. Red Hat Enterprise Linux 6.4 includes enhancements and new capabilities that provide rich functionality, especially the developer tools, virtualization features, security, scalability, file systems, and storage.
EMC Data Protection solutions—EMC Avamar® and EMC Data Domain®—deliver the protection confidence needed to accelerate deployment of a virtualized Oracle server.
Optimized for virtualized application environments, EMC Data Protection reduces backup times by 90 percent and increases recovery speeds by 30 times for worry-free protection. EMC Data Protection appliances add another layer of assurance with end-to-end verification and self-healing for ensured recovery.
For full technical guidance, refer to EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2 Design and Implementation Guide. This guide describes how to design, size, and implement EMC backup and recovery solutions for VSPEX Proven Infrastructure for a virtualized Oracle server.
Red Hat Enterprise Linux 6.4
EMC Data Protection solutions
Chapter 4: Choosing a VSPEX Proven Infrastructure
23 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Step 1: Evaluate the customer use case ................................................................... 24
Step 2: Design the application architecture ............................................................. 25
Step 3: Select the right VSPEX Proven Infrastructure ............................................... 27
Chapter 4: Choosing a VSPEX Proven Infrastructure
24 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Overview
This chapter describes how to design the VSPEX Proven Infrastructure with a virtualized Oracle server and how to select the right VSPEX Proven Infrastructure to meet your requirements. Table 3 outlines the main steps to complete when selecting a VSPEX Proven Infrastructure.
1 Use the VSPEX for virtualized Oracle OLTP qualification worksheet to evaluate the customer’s Oracle OLTP workload. Step 1: Evaluate the customer use case provides details.
2 Use the VSPEX Sizing Tool to determine the required infrastructure, Oracle server resources, and architecture. Step 2: Design the application architecture provides details.
Note: If the Sizing Tool is not available on the EMC Support website, manually size the application using the guidelines in Appendix B: High-level Oracle Database Server Sizing Logic and Methodology.
3 Select the right VSPEX Proven Infrastructure, based on the recommendations from Step 2. Step 3: Select the right VSPEX Proven Infrastructure provides details.
Step 1: Evaluate the customer use case
Before you select a VSPEX infrastructure solution, you must understand the customer’s real workload and dataset, based on the business requirements.
To help you better understand the customer’s business requirements for the VSPEX infrastructure design, EMC strongly recommends that you use the VSPEX for Virtualized Oracle OLTP qualification worksheet to evaluate the workload requirements for the VSPEX solution. For more information about the qualification worksheet for this solution, see Appendix A: VSPEX for Virtualized Oracle OLTP qualification worksheet.
In the VSPEX for Virtualized Oracle OLTP qualification worksheet, we asked some simple questions to help understand and describe the Oracle OLTP workload requirements and usage characteristics for the customer.
Table 4. VSPEX for virtualized Oracle OLTP qualification worksheet guidelines
Question Description
Do you have an existing Oracle database that you would like to size for in the environment?
Select Yes if the customer already has an Oracle database and understands the characteristics that are going to migrate to VSPEX Private Cloud in the VSPEX environment.
How many databases do you want to deploy?
Type the database number that the customer expects to deploy in the VSPEX environment.
Chapter 4: Choosing a VSPEX Proven Infrastructure
25 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Question Description
What is the size of user database (GB)? Type the database size that the customer expects to have in the VSPEX environment.
What is the annual growth rate (%)? Future growth is a key characteristic of the VSPEX solution. This value is the expected annual growth rate of the user database in three years. Type a number that is appropriate for the customer’s environment.
Do you intend to use FAST Cache? FAST Cache is particularly suited to applications that randomly access storage with high frequency. Fast Cache boosts Oracle application performance by ensuring that the most active data is served from high-performing flash drives and this data can reside on this the faster medium for as long as it is needed. A small number of flash drives implemented as FAST Cache provide a greater performance increase than a large number of hard disk drives.
Do you intend to use FAST VP? FAST VP significantly reduces total cost of ownership (TCO) and increases performance. A target workload that requires many Performance Tier drives can be serviced with a mix of tiers and a lower drive count. Using FAST VP instead of a homogeneous drive deployment has proven highly effective for lowering capital expenditures, reducing power and cooling costs, and increasing performance for many applications, including OLTP tests with Oracle Database.
What is the maximum number of IOPS (read/write)?
Understanding the maximum number of IOPS of Oracle databases can help to prevent potential storage performance issues. Work with the customer to estimate the IOPS at peak loads.
(Optional) What is the expected number of concurrent users for peak loads?
The maximum number of concurrent users is a key characteristic of the user database. If the customer can estimate the user number at peak loads in the environment, type that number.
Step 2: Design the application architecture
In this VSPEX Proven infrastructure solution, we defined how a representative customer workload is to be sized.
After you gather the customer’s information and populate the VSPEX for the virtualized Oracle qualification worksheet, you can use that information to populate
Chapter 4: Choosing a VSPEX Proven Infrastructure
26 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
the VSPEX sizing tool located on the EMC Business Value Portal, or you can use Appendix B: Sizing considerations to manually size the solution.
VSPEX Sizing Tool output: Requirements and recommendations
The VSPEX Sizing Tool enables you to put a database configuration from the customer’s answers into the qualification worksheet. Refer to the VSPEX Sizing Tool portal for more information about the Sizing Tool.
After you complete the inputs to the VSPEX Sizing Tool, the tool generates a series of recommendations, as listed in Table 5.
Table 5. VSPEX Sizing Tool output
Type Description Reference
vCPU The number of vCPUs to configure for each Oracle Server virtual machine
Oracle website
Memory The amount of memory recommended for each Oracle Server virtual machine
Storage layout for Oracle server databases
The suggested user database pool configuration on VNXe or VNX
Designing the storage layout
We also used a standard compute-size model for Oracle, which simplified and standardized the validation testing. The model enabled us to identify the configuration required to run a comparable TCP-C OLTP database workload with a 60:40 read/write ratio, yielding acceptable response time.
Table 6 shows how we mapped the Oracle sizing model to the virtual machine resources.
Table 6. Oracle sizing model for virtual machines
Oracle model Virtual machine resources
Small virtual machine for up to 200 users Compute requirements:
27 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Oracle model Virtual machine resources
Medium virtual machine for up to 1200 users
Compute requirements:
12 vCPU
38 GB memory
Storage requirements (OS and Oracle binaries):
100GB
50 IOPS
Storage requirements (Oracle datafiles):
1650 IOPS
Large virtual machine for more than 8000 users
Compute requirements:
32 vCPU
98 GB memory
Storage requirements (OS and Oracle binaries):
100GB
100 IOPS
Storage requirements (Oracle datafiles):
6600 IOPS
See the examples in Step 3: Select the right VSPEX Proven Infrastructure for more information.
If the sizing tool is not available on the EMC Support website, use the sizing instructions provided in Appendix B: High-level Oracle Database Server Sizing Logic and Methodology.
Step 3: Select the right VSPEX Proven Infrastructure
The VSPEX program provides many solutions designed to simplify the deployment of a consolidated virtual infrastructure using VMware vSphere and the EMC VNXe family of products and EMC Data Protection. After you confirm the application architecture, you can select the right VSPEX Proven infrastructure based on the calculated results.
Note: While this Design Guide is intended for Oracle Database OLTP requirements, the Oracle server may not be the only application deployed on the VSPEX Proven Infrastructure. You must carefully consider the requirements for each application that you plan to deploy. If you are uncertain about the best VSPEX Proven Infrastructure to deploy, consult an EMC representative before making the decision.
Considerations
Chapter 4: Choosing a VSPEX Proven Infrastructure
28 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Follow the steps in Table 7 when choosing a VSPEX Proven Infrastructure.
Table 7. Select the right VSPEX Proven Infrastructure
Step Action
1 Use the VSPEX Sizing Tool to determine the total number of required resources for virtual machines and any additional suggested storage layout requirements for the Oracle server.
2 Use the VSPEX Sizing Tool to design the resource requirements for other applications, based on business needs. The VSPEX Sizing Tool calculates the total number of required resources for virtual machines and recommended storage layout requirements for both the Oracle server and the other applications.
3 Discuss with your customers the maximum use of VSPEX Proven Infrastructure that meets their business requirements—this is the maximum use for both Oracle server and other applications. Put the maximum utilization percentage of the VSPEX Proven Infrastructure into the VSPEX Sizing Tool. The tool provides a minimum recommendation for the VSPEX Proven Infrastructure offering.
4 Select your network vendor and server vendor for the recommended VSPEX Proven Infrastructure offering. EMC VSPEX: Choose the Right Path to Your Cloud provides more information.
Overview
This section describes the following scenarios to help select the VSPEX Proven Infrastructure:
Multiple small-sized Oracle Database 12c servers, each with less than 200 concurrent users
A large-sized Oracle Database 12c server with up to 8,000 concurrent users
Example 1: Multiple small-sized Oracle Database 12c servers, each with less than 200 concurrent users
Use this scenario to deploy multiple small-sized Oracle Database 12c servers on a VSPEX Proven Infrastructure. The customer has multiple 500 GB databases. The expected R/W IOPS for each database is about 165/110.
After you type the answers from the qualification worksheet into the VSPEX Sizing Tool, the tool generates a series of recommendations for the resources needed from the VSPEX private cloud pool, as shown in Table 8.
29 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Question Answer
What is the annual growth rate (%)? 10
Do you intend to use FAST Cache? No
Do you intend to use FAST VP? No
What is the maximum number of IOPS (read/write)? 2,750
(Optional) What is the maximum number of concurrent users at peak loads?
N/A
After you type the answers from the qualification worksheet into the VSPEX Sizing Tool, the tool generates recommendations for the resources needed from the VSPEX private cloud pool, as shown in Table 9.
The VSPEX Sizing Tool also lists recommendations for the storage layout, as shown in Table 10.
Table 10. Recommendations for the storage layout example: Multiple small-sized Oracle 12c servers
Pool name RAID type Disk type Disk capacity No. of disks
VSPEX private cloud pool RAID 5 SAS disks 10,000 rpm 500 GB 5
Oracle database datafile pool
RAID 5 SAS disks 10,000 rpm 500 GB 30
Oracle database log file pool
RAID 1/0 SAS disks 10,000 rpm 500 GB 4
Example 2: A large-sized Oracle Database 12c server with up to 8,000 concurrent users
Use this scenario to deploy large-sized Oracle Database 12c servers on a VSPEX Proven Infrastructure. The customer has a 1 TB database. The expected number of concurrent users for the database is 8000 and the expected R/W IOPS is 3960/2640.
After you put the answers from the qualification worksheet into the VSPEX Sizing Tool, the tool generates a series of recommendations for the resources needed from the VSPEX private cloud pool, as shown in Table 11.
Chapter 4: Choosing a VSPEX Proven Infrastructure
30 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Table 11. Qualification worksheet example: Large-sized Oracle 12c server
Question Answer
Do you have an existing Oracle Server database that you would like to size for in the environment?
Yes
How many databases do you want to deploy? 1
What is the size of the user database (GB)? 1,000
What is the annual growth rate (%)? 10
Do you intend to use FAST Cache? No
Do you intend to use FAST VP? Yes
What is the maximum number of IOPS (read/write)? 6,600
(Optional) What is the maximum number of concurrent users at peak loads? N/A
After you type the answers from the qualification worksheet into the VSPEX Sizing Tool, the tool generates a series of recommendations for the resources needed from the VSPEX private cloud pool, as shown in Table 12.
Table 12. Required resources example: Large-sized Oracle 12c server
Oracle Server vCPUs Memory OS volume capacity OS volume
Resource requirement 32 98 GB 100 GB 100 IOPS
The VSPEX Sizing Tool also lists recommendations for the storage layout, as shown in Table 13.
Table 13. Recommendations for the storage layout example: Large-sized Oracle 12c server
Pool name RAID type Disk type Disk capacity No. of disks
VSPEX private cloud pool RAID 5 SAS disks 10,000 rpm 500 GB 5
Oracle database datafile pool RAID 5 SAS disks 10,000 rpm 500 GB 30
RAID 1 FAST VP SSD 200 GB 2
Oracle database log file pool RAID 10 SAS disks 10,000 rpm 500 GB 8
Chapter 5: Solution Design Considerations and Best Practices
31 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Chapter 5 Solution Design Considerations and Best Practices
Designing the network ............................................................................................. 32
Designing the storage layout ................................................................................... 33
Configuring FAST Cache for Oracle ........................................................................... 36
Configuring FAST VP for Oracle ................................................................................ 37
Designing the virtualization layer ............................................................................ 38
Designing the Oracle Database 12c implementation ................................................ 40
EMC Data Protection design considerations ............................................................. 43
Chapter 5: Solution Design Considerations and Best Practices
32 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Overview
This chapter describes the EMC VSPEX for Virtualized Database 12c OLTP solution design and describes the best practices for the network, storage, virtualization, and applications.
Designing the network
This section describes the network details for SAN and IP network configuration and an ESXi server network. In this VSPEX Proven Infrastructure for Virtualized Oracle Database 12c solution, EMC recommends that you consider the ESXi Server’s advanced settings and the network redundancy when designing the network.
The advanced networking features of the VNXe series provide protection against network connection failures at the array. Each virtual host also has multiple connections to storage Ethernet networks to guard against link failures. Spread the connections across multiple Ethernet switches to guard against component failure in the network.
EMC recommends that you use the following SAN best practices:
Use multiple host bus adapter (HBA) and FC switches for network redundancy.
Zone each FC port from the database servers to both storage processor ports for high availability.
Use path management and dynamic multipathing software, such as PowerPath, on the hosts to enable the failover process to alternate paths and to provide load balancing.
EMC recommends that you use the following IP network best practices:
Use multiple network cards and switches for network redundancy.
Use a 10 Gb Ethernet for network connection.
Use Virtual local area networks (VLANs) to logically group devices that are on different network segments or sub networks.
Enable and configure jumbo frames throughout the physical or virtual stack.
Note: Maximum Transfer Unit (MTU) sizes of greater than 1,500 bytes are referred to as jumbo frames. Jumbo frames require a Gigabit Ethernet across the entire network infrastructure, including servers, switches, and database servers.
Networking in the virtual world follows the same concepts as networking in the physical world, but some of these concepts are applied in the software instead of using physical cables and switches. Although many of the best practices that apply in the physical world continue to apply in the virtual world, there are additional considerations for traffic segmentation, availability, and throughput.
Overview
SAN best practices
IP network best practices
vSphere network best practices
Chapter 5: Solution Design Considerations and Best Practices
33 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
For example, in the VMware virtualization environment, use two physical NICs per vSwitch and uplink the physical NICs to separate physical switches.
When configuring the NIC teaming settings, it is a best practice to select No for the NIC teaming failback option. If there is some intermittent behavior in the network, this will prevent the NIC cards from flip-flopping.
When setting up VMware High Availability, also set the following ESX server timeouts and settings under the ESX Server advanced setting tab:
NFS.HeartbeatFrequency = 12
NFS.HeartbeatTimeout = 5
NFS.HeartbeatMaxFailures = 10
To access the NFS advanced options, follow these steps:
1. Log into the VMware vSphere Client.
2. Select the ESXi/ESX host.
3. From the Configuration tab, select Advanced Settings >NFS.
Refer to Best Practices for Running VMware vSphere on Network Attached Storage (NAS) for more NIC teaming best practices for VMware vSphere.
Refer to the VSPEX Proven Infrastructure Guide for other best practices in network design for the VSPEX Proven Infrastructure.
Oracle direct NFS or dNFS, which was introduced in Oracle 11g R2, allows the NFS operations to be moved to the database layer instead of the operating system kernel. Oracle strongly recommends configuring the Oracle database to use the Oracle dNFS Client ODM disk libraries. This is a one-time operation. After logging into the Linux system, type the following commands as an Oracle user to enable the dNFS Client ODM library:
Refer to the EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide for other best practices in network design for the VSPEX Proven Infrastructure.
Designing the storage layout
This section provides guidelines for planning storage for various business requirements in Oracle Database 12c environments.
Note: Your customers must select the deployment architecture that best fits their specific needs. EMC unified storage provides flexibility and manageability for a storage infrastructure that supports either file or block protocol.
Oracle settings for NFS-specific considerations
Overview
Chapter 5: Solution Design Considerations and Best Practices
34 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Figure 3 shows an example of the high-level architecture between the Oracle Database 12c components and the storage elements validated in the VSPEX Proven Infrastructure for virtual Oracle Database server on a VMware vSphere virtualization platform.
Figure 3. Example: Storage layout of Oracle 12c Database on VMware
Note: Figure 3 shows the use of Oracle Direct NFS (dNFS) over Internet protocol (IP) with Oracle. All of the Oracle Server volumes can also be stored in Virtual Machine Disk (VMDK) format in a VMware virtualized environment.
EMC recommends that you use different storage pools to store OS and Oracle Database data in addition to the VSPEX private cloud pool for virtual machines. Table 14 provides an example.
Table 14. Oracle Sever storage pools example
Storage pool name Purpose Recommendation
VSPEX private cloud pool
The private cloud pool where all the virtual machines reside.
Mixed pool including RAID 5 with SAS disks and RAID 1/0 with FAST VP SSD
Oracle data pool
(FAST Cache optional )
The Oracle data pool to serve the datafile LUNs and volumes for user databases
RAID 5 with SAS disks, or a mixed pool
Oracle redo pool The Oracle redo pool to serve the logfile LUNs and volumes for user databases
RAID 1/0 with SAS disks
Storage layout and design considerations
Chapter 5: Solution Design Considerations and Best Practices
35 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Consider the following best practices for storage and layout design in the VSPEX Proven Infrastructure for virtualized Oracle Database 12c solution.
Oracle Database data pool
Use SAS disks with RAID 5 (4+1) protection for the Oracle datafile and temp file systems. This combination of RAID protection and disk type provides high capacity use and good I/O performance at a low cost, while simultaneously ensuring data availability if a drive failure occurs.
Oracle Database redo log pool
In this solution, we configured the file system for redo logs on the different pools protected by RAID10 on SAS disks. For highly write-intensive workloads, consider a separate pool for redo file systems on physically separate disks.
Customization
EMC recommends that you work with the customer to estimate the capacity and IOPS requirements for the storage layout. Consider future growth when laying out the storage, and include projected growth as input to the VSPEX sizing tool.
Additional performance requirements for FAST Suite
The EMC FAST Suite—FAST VP and FAST Cache—provides two key technologies available on the VNX series that enable extreme performance in an automated fashion. See the VSPEX Proven Infrastructure website for more information on FAST Suite for VSPEX Proven Infrastructures.
FAST Cache can improve overall system performance for drive-related problems with Oracle applications. We recommend first using available flash drives for FAST Cache, which can globally benefit all data LUNs in the storage system. Then supplement performance as needed with additional flash drives in storage pool tiers using FAST VP.
Enabling FAST Cache is a transparent operation to Oracle Database 12c and no reconfiguration or downtime to the database is necessary. EMC recommends that you only use FAST Cache on the storage pool or LUNs that require it. FAST Cache is best for light random I/O where there is an uneven data distribution.
Users also can create blended storage pools composed of various disk types (SSD, SAS, and NL_SAS). The data migration in this highly consolidated virtualized environment produces the highest storage efficiency, from a performance and capacity perspective.
If you enable FAST Suite technology on the Oracle Database 12c, the response times, read/write throughput, and latencies will improve the Oracle Database 12c user experience. FAST Suite technology eases the burden on storage and database administrators by determining the most efficient storage layout for their customers.
This section describes the VNXe storage layouts in this VSPEX Proven Infrastructure for virtualized Oracle Database 12c based on the VSPEX private cloud. This example follows the best practice and design considerations, as previously discussed.
Chapter 5: Solution Design Considerations and Best Practices
36 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Figure 4 shows a storage layout dedicated to Oracle Database pools.
Figure 4. Storage layout example: Oracle Server for VNX series with FAST VP/Cache enabled
Note: This is only one example of a storage layout. To plan and design your own storage layouts for an Oracle database on an EMC VSPEX stack, follow the guidance for the VSPEX Sizing Tool and the best practices described in Designing the storage layout.
Configuring FAST Cache for Oracle
FAST Cache uses enterprise flash drives to add an extra layer of cache between dynamic random access memory (DRAM) cache and rotating disk drives, creating a faster medium for storing frequently accessed data. FAST Cache is an extendable, read/write cache that boosts application performance by ensuring that the most active data is served from high-performing flash drives, and can reside on the faster medium for as long as necessary.
FAST Cache tracks data activity at a granularity of 64 KB and promotes data that is frequently accessed into FAST Cache by copying it from the hard disk drives to the flash drives assigned to FAST Cache. Subsequent I/O access to that data is handled by the flash drives and is serviced at flash drive response times: this ensures very low latency for the data. As data ages and becomes less active, it is flushed from FAST Cache to be replaced by more active data.
A few flash drives implemented as FAST Cache provides a greater application performance increase than many short-stroked hard disk drives.
FAST Cache is particularly suited to applications that randomly access storage with high frequency, such as Oracle OLTP databases. Also, OLTP databases have inherent locality of reference with varied I/O patterns. Applications with these characteristics benefit most from deploying FAST Cache. The optimal use of FAST Cache occurs when the working dataset can fit within the FAST Cache.
Overview
Chapter 5: Solution Design Considerations and Best Practices
37 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
EMC recommends the following best practices:
Only enable FAST Cache on pool LUNs that require it
Size FAST Cache appropriately based on the application's active dataset
Disable FAST Cache on pool LUNs where Oracle online redo logs reside
Never enable FAST Cache on archive logs because these files are never overwritten and are rarely read back (unless you need to recover the database)
Spread FAST Cache SSDs across all available buses
Use the available FAST Cache SSDs first for FAST Cache and then supplement performance as needed with additional flash drives in the storage pool tiers
EMC recommends that you enable FAST Cache for the Oracle datafiles only. Oracle archive files and redo log files have a predictable workload composed mainly of sequential writes. The array's write cache and assigned hard disk drives can efficiently handle these archive files and redo log files. Enabling FAST Cache on these files is neither beneficial nor cost effective.
Enabling FAST Cache on a running system FAST Cache can improve overall system performance if the current bottleneck is drive-related. However, boosting the IOPS will result in an increase in CPU utilization on the VNX SPs. Systems should be sized so that the maximum sustained utilization is 70 percent.
Use Unisphere to check the SP CPU utilization and then proceed as follows:
SP CPU utilization less than 60 percent—Enable groups of LUNs or one pool at a time until they are equalized in the cache. Ensure that the SP CPU utilization is still acceptable before turning on FAST Cache for more LUNs/pools.
SP CPU utilization of 60 to 80 percent—Scale in carefully. Enable FAST Cache on one or two LUNs at a time, and verify that the SP CPU utilization does not exceed 80 percent.
SP CPU utilization greater than 80 percent—Do not activate FAST Cache.
Configuring FAST VP for Oracle
Like FAST Cache, FAST VP works best on datasets that exhibit a high degree of skew. FAST VP is very flexible and supports several tiered configurations, such as single tiered, multi-tiered, with or without a flash tier, and FAST Cache support. Adding a flash tier can locate “hot data” on flash storage in 256 MB slices.
For VNXe, EMC recommends that you first enable FAST VP on the Oracle datafile storage pool to make the best use of either of the FAST technologies.
Aggressive reduction of TCO and an increase in performance can be achieved with FAST VP. A common strategy is to use FAST VP to gain TCO benefits while using FAST Cache to boost overall system performance. This paper discusses considerations for an optimal deployment of these technologies.
FAST Cache best practices
Overview
Chapter 5: Solution Design Considerations and Best Practices
38 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
For further information on FAST VP algorithm and policies, see EMC FAST VP for Unified Storage Systems.
EMC recommends the following best practices:
Spread FAST VP SSDs across all available buses.
Enable FAST VP on a pool, even if the pool has only one tier, to provide ongoing load balancing of LUNs across available drives.
Maintain some unallocated capacity within the pool to help with relocation schedules when using FAST VP.
Schedule relocations for off-hours, so that the primary workload does not contend with the relocation activity
Designing the virtualization layer
Oracle Database 12c is fully supported when deployed in a virtual environment with VMware vSphere ESXi technology. The following sections describe the best practices and design considerations for Oracle Database 12c virtualization.
In this VSPEX Proven Infrastructure for Oracle database, EMC recommends that you consider implementing best practices for managing the following resources in your virtualization design:
Compute resources
Network resources
VMware features
Compute resources
EMC recommends implementing the following compute resource best practices:
Enable hyper-threading. Hyper-threading technology allows a single physical processor to execute multiple independent threads simultaneously. ESXi is designed to make use of hyper-threading by controlling the placement of logical processors on the same core and intelligently managing processor time to guarantee that load is spread evenly across all physical cores in the system.
Use Hardware-Assisted MMU Virtualization (Intel EPT and AMD RVI) to reduce memory consumption and to speed up workloads that cause guest operating systems to modify page tables too frequently.
Use Non-Uniform Memory Access (NUMA), a computer architecture in which memory located closer to a particular processor is accessed with less delay than memory located farther from that processor.
Allocate Virtual machine memory (vRAM) in a virtual machine to be less than the local memory accessed by NUMA node (processor).
Schedule the vCPU to use the fewest sockets required using the virtual machine parameter numa.vcpu.preferHT=TRUE.
FAST VP best practices
Overview
Virtualization best practices
Chapter 5: Solution Design Considerations and Best Practices
39 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Install VMware Tools that include utilities that enhance the performance of the virtual machine's guest operating system and improve the ability to manage the virtual machine.
Network resource
EMC recommends that you implement the following network resource best practices:
Use the most recent paravirtualized virtual network device from VMware: VMXNET Generation 3 (VMXNET3), which supports 10 GbE.
Use vLANs to separate vSphere infrastructure traffic from virtual machine traffic for security and isolation.
Enable and configure Jumbo frames throughout the virtual and physical stack for the vMotion and the IP storage networks.
Use an In-guest NFS mount from an Oracle dNFS client within the virtual machine rather than VMDK on an NFS datastore.
VMware features
EMC recommends that you implement the following VMware features:
vSphere HA—This feature uses multiple ESXi hosts, configured as a cluster, to provide rapid recovery from outages and provides cost-effective high availability (HA) systems for applications running in virtual machines. vSphere HA protects applications against:
Server failure by restarting the virtual machines on other ESXi servers within the cluster
Application failure by continuously monitoring a virtual machine and resetting it if a guest OS fails
VMware DRS—This feature automatically balances the workload between the hosts using the vMotion function when migrating virtual machines. When Oracle database workloads increase, DRS automatically moves a bottlenecked virtual machine to another host with more resources available, without downtime.
DRS Affinity rules—This feature controls the placement of virtual machines on hosts within a cluster. DRS provides two types of affinity rules:
A virtual machine host affinity rule, which specifies an affinity relationship between a group of virtual machines and a group of hosts
A virtual machine virtual machine affinity rule, which specifies whether particular virtual machines will run on the same host or remain on separate hosts
VMware templates
VMware templates are virtual machine master copies used to create and provision virtual machines. Use a template to deploy a virtual machine with a guest OS installed and applications and users configured. The guest OS is ready for use with minimal user intervention.
Chapter 5: Solution Design Considerations and Best Practices
40 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Templates minimize the deployment time and automate installation and configuration tasks required for each virtual machine. Virtual machine deployment is simplified by customizations stored in vCenter. A deployment wizard, automation tool, or script can use templates to automatically create or edit server settings before deploying the new virtual machine.
Monitoring the VSPEX Proven Infrastructure regularly
Monitor the performance of the VSPEX Proven infrastructure regularly. Performance monitoring happens not only at the virtual machine level, but also at the hypervisor level. With an ESXi hypervisor, you can use performance monitoring within the Oracle Database machine to ensure that the virtual machine or Oracle Database performs as expected. You can also use monitoring tools such as esxtop to observe host performance at the hypervisor level.
Designing the Oracle Database 12c implementation
One of the purposes of this solution is to detail how two storage networking technologies are used with Oracle: Oracle Automatic Storage Management (ASM) over Fibre Channel(FC) and Oracle Direct NFS(dNFS) over Internet Protocol(IP).
There are many design considerations for ASM and dNFS in the context of an online transaction processing (OLTP) workload from Oracle Database 12c. This section provides guidelines for the most common and important design considerations and best practices to follow.
Since the Oracle Database 10g Release 2, database storage management and provisioning has been simplified by using Automatic Storage Management(ASM). ASM provides file system and volume management tasks, such as creating or laying out database and diskspace management.
With ASM over FC, all database objects , including datafiles, online redo files, and control files, are stored on ASM disk groups and managed by Oracle ASM. ASM provides out-of-box enablement of redundancy and optimal performance. However, the following items should be considered to increase either performance or availability, or both:
Implement multiple access paths to the storage array using two or more HBAs or initiators
Deploy multi-pathing software over these multiple HBAs to provide I/O load-balancing and failover capabilities
Use ASM disk groups with similarly sized and performing disks.
The Oracle dNFS client is a standard feature with Oracle Database 12c and provides improved performance and resilience over the OS native NFS. dNFS also performs asynchronous IOPS, which allows processing to continue while the I/O request is submitted and processed.
With dNFS over IP, all database objects are accessible through an NFS mount. Datafiles, online redo log files and other database files are accessed using dNFS over
Overview
Best practices for ASM and database deployment
Best practices for dNFS configuration
Chapter 5: Solution Design Considerations and Best Practices
41 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
IP protocol. EMC recommends configuring the Oracle database to use the Oracle dNFS client ODM disk libraries. This is a one-time operation and after it is set the database uses the Oracle-optimized native Oracle dNFS client instead of the native NFS client.
Automatic Shared Memory Management (ASMM) is a standard method of dynamically managing memory in the Oracle database and has been available since Oracle Database 10g. EMC recommends that you implement ASMM to automate the management of the following shared memory structures:
DB_CACHE_SIZE
SHARED_POOL_SIZE
LARGE_POOL_SIZE
JAVA_POOL_SIZE
STREAMS_POOL_SIZE
Set the following initialization parameters to implement this feature:
SGA_TARGET set to a nonzero value
STATISTICS_LEVEL=TYPICAL (or ALL)
Note: Do not use Oracle Automatic Memory Management (AMM) because AMM is incompatible with HugePages. If you want to use Linux HugePages, ensure that both MEMORY_TARGET / and MEMORY_MAX_TARGET initialization parameters are not set.
ASMM versus AMM and LINUX x86-64 HugePages Support (Doc ID 1134002.1) on My Oracle Support provides more information about HugePages.
HugePages is crucial for faster Oracle database performance on Linux if you have a large RAM and SGA. If the combined database SGAs are large (more than 8 GB), you need to configure HugePages. The size of the SGA matters.
The advantages of enabling HugePages include:
Larger page size and fewer pages
Better overall memory performance
No swapping
Use Large Pages to Enable HugePages (Doc ID 1392497.1) on My Oracle Support provides more information about enabling and tuning HugePages.
Set DISK_ASYNCH_IO= true. Asynchronous I/O is now recommended on all the storage protocols. The default value in Oracle 12c is true .
Set FILESYSTEMIO_OPTIONS=SETALL. This setting enables both direct I/O and asynchronous I/O. With asynchronous I/O, processing continues while the I/O request is submitted and processed.
Direct NFS does not depend on the value of FILESYSTEMIO_OPTIONS. Direct NFS always uses asynchronous and direct I/O, as it does not depend on OS support.
Manage automatic shared memory
Enable the HugePages setting
Configure I/O operations for file system files
Chapter 5: Solution Design Considerations and Best Practices
42 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
However, you can always fall back to the OS NFS client if there is misconfiguration. As a precaution, set the filesystemio_options parameter to SETALL if the OS supports it.
Init.ora Parameter “FILESYSTEMIO_OPTIONS” Reference Note (Doc ID 120697.1) on My Oracle Support provides more information.
EMC recommends that you create the different storage pools for different Oracle data types, such as datafiles and online redo log files. You can also enable FAST Cache when appropriate, depending on the data type. The exact layout requirements will change for each deployment, based on each customer’s requirements.
Table 15 shows a database layout example in which FAST Cache is enabled on the datafile pool only.
Table 15. Database layout for a consolidated Oracle environment example
Redo pool Data pool FRA pool (optional)
Application
Data Type Redo Logs Data files FRA files
FAST Suite policies
FAST Cache No Yes No
FAST Policies No Auto Tier No
Note: Oracle recommends using FRA (Fast Recovery Area) to enable storing all backup files in one place. If the FRA is enabled, the sizing logic for Oracle archived log and backup files is different from the Oracle datafiles and redo log files. For detailed support, consult your EMC representative.
Oracle licensing considerations
In this VSPEX Proven Infrastructure for virtualized Oracle Server, EMC recommends that you consider the Oracle Server licensing models to achieve better cost savings.
The Oracle processor licensing option is based on the interaction between the software and the hardware. For Oracle Enterprise Edition (Oracle EE), the licensing is based on the number of physical cores that are available to the installed Oracle software. For Oracle SE, the licensing is based on the number of processor sockets that are available to the installed Oracle software. Oracle does not permit the soft partitioning of CPUs as a means to calculate or limit the number of software licenses required for a physical server. Oracle regards VMware vSphere technology as soft partitioning. In a vSphere environment, you must license all hosts where the Oracle executable files are either installed or running or both.
This means that the design and size of the vSphere ESXi cluster, together with the placement and movement of virtual machines that host the Oracle executable files, are essential to minimize Oracle licensing costs. Understanding Oracle Certification Support and Licensing for VMware Environments White Paper provides details.
Configure the database data type layout
Chapter 5: Solution Design Considerations and Best Practices
43 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
EMC Data Protection design considerations
All VSPEX solutions are sized and tested with EMC Data Protection products, including EMC Avamar and EMC Data Domain. If the solution includes EMC data protection components, refer to EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2 Design and Implementation Guide for detailed information on implementing these options into your VSPEX solution.
Chapter 5: Solution Design Considerations and Best Practices
44 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Chapter 6: Solution Verification Methodologies
45 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Verifying the solution ............................................................................................... 46
Chapter 6: Solution Verification Methodologies
46 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Overview
This chapter describes the verification methodologies required for the hardware, application, and data protection aspects of the solution. Hardware consists of the computer's physical resources such as processors, memory, and storage. Hardware also includes the physical network components such as NICs, cables, switches, routers, and hardware load balancers. You can avoid many performance and capacity issues by using the correct hardware for the VSPEX for virtualized Oracle Server solution. Conversely, a single misapplication of a hardware resource, such as insufficient memory on a server, can affect the performance of the Oracle server.
Verifying the solution
EMC recommends that you test the new VSPEX for Oracle Database 12c application overlay proven architecture before deploying it to a production environment. This test of the proven overlay architecture confirms that the design achieves the required performance and capacity targets, and it will also identify and optimize potential bottlenecks before they affect users in a live deployment. This section gives a summary description of the high-level steps that we performed when verifying this solution.
Before verifying the Oracle Database 12c performance in the VSPEX Proven Infrastructure, ensure that Oracle Database 12c is deployed in the VSPEX Proven Infrastructure based on the EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide.
Table 16 describes the high-level steps that you need to complete before you can implement the Oracle Database 12c environment in production.
Table 16. High-level steps for application verification
Step Description Step
1 Understand key metrics of the Oracle database environment to achieve the performance and capacity that your business requires.
Understand key metrics
2 Use the VSPEX Sizing Tool for Oracle Database to determine the architecture and resources required by the VSPEX Proven Infrastructure implementation
EMC VSPEX website
3 Design and build the Oracle database solution on a VSPEX Proven Infrastructure. Run the tests, analyze the results, and optimize the VSPEX architecture
VSPEX Implementation Guides
Understanding the goal of the Oracle server testing helps to decide which metrics to capture and what thresholds must be met for each metric when running the Oracle server validation tests. We considered the key metrics shown in Table 17 to validate the VSPEX for virtualized Oracle server solution.
47 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Table 17. Key metrics
Metric Threshold
Average CPU utilization (%) Less than 80%
Average wait time for User I/O Less than 20 milliseconds
Average wait time for Commit Less than 15 milliseconds
Use the VSPEX Sizing Tool
Use the VSPEX Sizing Tool to understand the basic metrics and thresholds to meet your customer’s business requirements.
For more information about using the VSPEX sizing tool, refer to VSPEX Sizing Tool for Oracle Database 12c available on the EMC VSPEX website.
Follow VSPEX Implementation Guides
After you have designed the VSPEX infrastructure, refer to the EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide for information on how to implement the solution.
In the VSPEX for Virtualized Oracle Database 12c solution, we ran the tests using an application such as TPC-C to validate the Oracle server performance. We recommend that you do the following:
Evaluate the workload and I/O pattern. If it is acceptable and the real workload is similar, you can use the test results as a reference. However, customers need to consider the potential risks.
Build a test environment first, and then copy and restore the production database to test the real workload and to verify the Oracle server performance if the real application workload types are different from what we validated in our test environment.
The EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide provides detailed configuration information.
Other documentation ............................................................................................... 51
Chapter 7: Reference Documentation
50 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
EMC documentation
The following documents, available from the EMC Online Support or EMC.com websites, provide additional information. If you do not have access to a document, contact your EMC representative.
EMC VNXe3200 Installation Guide
Proven Infrastructure Guide for EMC VSPEX Private Cloud with VMware vSphere 5.5 for up to 200 virtual machines
EMC VSPEX Proven Infrastructure for Virtualized Oracle
Data Protection for EMC VSPEX Proven Infrastructure White Paper
EMC VNXe Series Configuration Worksheet
Using a VNXe System with VMware NFS or VMware VMFS
52 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Appendix A: Qualification Worksheet
53 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Appendix A Qualification Worksheet
This appendix presents the following topics:
VSPEX for Virtualized Oracle OLTP qualification worksheet ..................................... 54
Gather the information from the customer’s Oracle databases example .................. 54
Printing the qualification worksheet ........................................................................ 57
Appendix A: Qualification Worksheet
54 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
VSPEX for Virtualized Oracle OLTP qualification worksheet
Before sizing the VSPEX solution, gather the information from the customer’s Oracle databases using the qualification worksheet shown in Table 18. This worksheet is appropriate for qualifying multiple databases.
Table 18. Qualification worksheet for virtualized Oracle OLTP databases
Question Answer
Do you have an existing Oracle server database that you would like to size for in the environment?
Yes or No
How many databases do you want to deploy?
What is the size of the user database (GB)?
What is the annual growth rate (%)?
Do you intend to use FAST Cache? Yes or No
Do you intend to use FAST VP? Yes or No
What is the maximum number of IOPS (read/write)?
(Optional) What is the maximum number of concurrent users at peak loads?
You can use the Oracle Automatic Workload Repository to obtain this information as described in the Oracle Database Performance Tuning Guide 12c Release 1 (12.1).
Gather the information from the customer’s Oracle databases example
The Automatic Workload Repository (AWR) provides more of the information required to populate the EMC Oracle qualification worksheet from each Oracle database. The AWR provides key statistics on database performance, load, and resources (both internal and external). You can access this data using standard Oracle supplied scripts. You can obtain the remaining information from the customer or by using the simple queries that are provided in this appendix.
Use the init.ora Parameters section of the AWR report to calculate the System Global Area (SGA) and Program Global Area (PGA) values as shown in Figure 5.
You can use the data and temporary file sizes to populate the DB Size (MB) column and calculate the total as follows:
SQL> select ltrim(to_char(sum(bytes)/(1024*1024))) as “Total size
(M)”
from (
select sum(bytes) as bytes from v$datafile
union
select bytes from v$tempfile);
Total size (M)
----------------------------------------
256000
1 row selected.
You can find the READ IOPS, WRITE IOPS, and Change Rate (MB/s) values in the IOStat by Function summary section of the AWR report. Figure 6 shows these values.
Find the number of concurrent users
Calculate database size
Find the datafile IOPS and change rate for the Redo logs
Appendix A: Qualification Worksheet
56 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Figure 6. IOStat by function summary from the AWR Report
The following Oracle wait events, as shown in Figure 7, provide key response time statistics for the Oracle database.
Use db file sequential read to populate the User I/O column. Oracle recommends that this value be under 20 ms.
Use log file sync to populate the Commit column. Oracle recommends that this value be under 15 ms.
Figure 7. Foreground Wait Event from the AWR report
How to Tell if the IO of the Database is Slow (ID 1275596.1) on My Oracle Support lists typical I/O response times.
You can find the value used to populate the TPS column of the worksheet from transactions in the Load Profile of the AWR report, as shown in Figure 8.
Calculate user I/O time and commit time
Calculate transactions in the Load Profile
Appendix A: Qualification Worksheet
57 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Figure 8. Transactions in a load profile from the AWR report
Printing the qualification worksheet
A stand-alone copy of the qualification worksheet is attached to this document in PDF format. To view and print the worksheet:
1. In Adobe Reader, open the Attachments panel as follows:
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
60 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Overview
The infrastructure supporting OLTP, including vCPU, memory, and the storage layout for Oracle server databases, must provide a robust, powerful, and flexible solution. Sizing the Oracle server depends on multiple factors, such as disk type, protection type, and cache. These resources must be included as part of the Oracle server sizing method.
Note: These manual sizing instructions may be used to provide an approximate single application sizing, if the VSPEX Sizing Tool is not available. The VSPEX Sizing Tool, with its multi application, multi instance capability, is recommended as the preferred sizing approach.
Resource considerations
To satisfy the performance requirements for Oracle Server databases, sufficient resources, including the compute and disk subsystem, must be ensured. This section defines the required resources for an Oracle server in a virtualized environment to provide predictable performance.
Sufficient disk utilization—Design the sizing tool to use the sufficient disk resources and to leave capacity for any possible peak disk activities.
Sufficient memory utilization—Design building blocks with sufficient system memory to support the designed workload with anticipated peak load activities.
Sufficient processor utilization—Design building blocks to have sufficient vCPU resources to support the designed workload and to anticipate peak load activities.
Sizing considerations
This section provides detailed sizing methods and recommendations for sizing each Oracle server instance:
Determine the virtual machine resources:
vCPU resources
Memory resources
Determine the OS capacity resources and IOPS
Determine the I/O workload and the disk type and number
Consider the advanced storage features
Select the VSPEX Proven Infrastructure
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
61 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
Table 19 shows an example of three different sized databases and the recommended user input.
Table 19. Example of user input for multiple Oracle databases
Database profile Maximum database size (GB)
Maximum database performance (R/W IOPS)
Maximum users for each database
Database 1 500 165/110 200
Database 2 500 990/660 1200
Database 3 1000 3960/2640 8000
Taking the user count for three different size databases in Table 20, and referencing Table 6, we can obtain the size of the virtual machine required.
Table 20. Required CPU and RAM for different size databases
Database profile Database model
Recommended CPU Recommended RAM (GB)
Small-sized database Small virtual machine for up to 200 users
2 8
Medium-sized database
Medium virtual machine for up to 1200 users
12 38
Large-sized database Large virtual machine for more than 1200 users
32 98
We recommend setting the capacity for each Oracle server to 100 GB and setting the OS IOPS to 50/100 IOPS for each small, medium, or large server OS volume. The EMC VSPEX for Virtualized Oracle Database 12c OLTP Implementation Guide provides more details.
Use the following design methods to determine the I/O workload and the disk type, number, and capacity in the VSPEX Proven infrastructure.
Calculate the correct number of drives for IOPS first, and then calculate the size of the databases on the disk layout. Consolidate the calculation results for both IOPS and the capacity.
Include the additional disk requirements such as annual growth rate.
Set the IOPS for different disk types based on the actual test result.
Determine virtual machine resources
OS capacity resources and IOPS
Determine the I/O workload and the disk type, number, and capacity
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
62 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Calculate the number of disks based on the following formula:
Required back-end IOPS=read IOPS +(write IOPS x RAID write penalty)
Disk number = required back-end IOPS / IOPS per disk
If you use EMC FAST suite technology, determine the active data set size to match the FAST Cache/VP size.
This section describes how to calculate the resources required in a VSPEX Proven Infrastructure for Oracle Database 12c OLTP. We used three working examples that show how the Oracle manual sizing method works. You can apply the methodology used in these examples to homogeneous provisioning pools with or without configuring FAST VP or FAST Cache.
The database entry medium-size Oracle database example, as shown in Table 19, has the following storage profile:
Database size of 500 GB
Database performance with 990 read and 660 write IOPS
Five percent annual growth gives a three-year capacity of 580 GB
The redo change rate is 2.5 MB/s
As described in Designing the storage layout, when calculating the storage requirement for a database, consider both I/O performance and capacity. First determine the RAID type of the pool and the drive-group size. In this solution, all datafiles reside on RAID 5 storage and the Oracle redo files reside on RAID 10.
Calculate the storage requirements for the database using the values collected in the Qualification Worksheet and use the recommendations in Table 21 and Table 22.
Table 21. RAID type, write penalty, and capacity utilization
RAID Capacity utilization
Multiple of Write penalty
Active drives
Parity drives
RAID 5 (4+1) 0.8 5 4 4 1
RAID 10(2+2) 0.5 4 2 4 N/A
Table 22. Random disk IOPS and bandwidth by drive type
Drive Type IOPS Bandwidth (R/W MB/s)
15 K SAS 180 30/25
10 K SAS 150 25/20
Near-line serial-attached SCSI (NL-SAS)
90 15/10
SAS Flash 3,500 90/75
Note: For the maximum supported IOPS for VNXe, consult an EMC representative.
Example 1: Homogenous pools without FAST suite
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
63 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
In this example, we calculated the drive load for the example medium-size database as follows:
Total I/O for RAID 5 = 990 + 4* 660 = 3630
The IOPS-based calculation has a higher request of spindles, including the higher speed of spindles and the larger quantity of spindles compared to a capacity-based calculation in a generic OLTP application. In the example in Table 23, a 500 GB OLTP database can have more than 3600 IOPS, which needs 25 of the 10 K rpm SAS or FC spindles to support the IOPS request. However, two 600 GB mirrored SAS/FC spindles can satisfy the capacity request completely. For this example, we recommend using 25 SAS/FC spindles as in Table 23.
Table 23. Storage pool calculation example without FAST Cache
Storage pool No. of drives Total capacity (GB)
Oracle data pool RAID 5
25 SAS drives
24 = 3,630/150
Round up to a multiple of 5 to allow for RAID 5 (4 +1) = 25 drives
600 GB x 25 x 0.8 = 120,000 GB
Oracle redo pool RAID 10
4 SAS drives
1 = (2.5 MB/s * 4) / 20 MB/s
Round up to a multiple of 4 to allow for RAID 10 (2+2) = 4 drives
600 GB x 4 x 0.5 = 1200 GB
Note: The data pool uses a 600 GB 10 K SAS drive with a random read/write workload. The redo pool also uses a 600 GB 10 K SAS drive with a sequential write workload. A conservative value of 20 MB/s per drive is used for sustained writes.
Consider the advanced storage features
If the FAST Suite (FAST Cache or FAST VP) feature is enabled, the sizing logic for VNXe is different from that for storage that does not use FAST Suite.
When calculating for performance with FAST Suite selected, the flash tier needs to serve the maximum number of I/Os; therefore, it has a higher calculation priority. We used a total back-end IOPS minus FAST Cache SSDs supported IOPS and then calculated the number of SAS/FC disks for this example.
Note: We recommend enabling the FAST Suite feature when each component of the storage array is running at reasonable levels, such as keeping the storage processor use at less than 70 percent.
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
64 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
The example in Table 25 sizes a workload for a data pool that is supplemented by FAST Cache. To configure FAST Cache, complete the following five steps.
Step 1: Determine the workload
The example in Table 25 uses the data from the database entry medium-size Oracle database as shown in Table 19.
Step 2: Size the FAST Cache
FAST Cache is best for small, random I/O where a small percentage of the data (active data set) produces the majority of the IO in the environment. EMC recommends sizing FAST Cache as follows:
Match the FAST Cache size to the size of active data set.
If the active dataset size is unknown, size FAST Cache to be 5 percent of your capacity.
Avoid enabling FAST Cache for Oracle log files for a small-block sequential workload
Note: Contact an EMC representative for help in determining the active data set size for customers.
In the example in Table 25, we implemented a two 200 GB flash drive FAST Cache as RAID 1. For the purposes of the calculation, we used a more conservative 50 percent cache hit rate, which means that there is a 50 percent chance that FAST Cache will handle the I/O.
This estimate is for cache that has been “warmed up” by being in place and running for a while. Initially, before the cache has a chance to “warm up,” the hit rate is lower.
Adjust the host IOPS served by mechanical drives by reducing IOPS by the FAST Cache hit rate percentage. Table 24 summarizes the calculations performed.
Table 24. FAST Cache hit rate and workload calculations
Database profile
Database Size(GB)
Host IOPS FAST Cache hit rate percent
Host IOPS after FAST Cache
Medium-sized Oracle
500 990+660=1650 50 495+330=825
Next, calculate the number of drives needed for the RAID levels to serve the required IOPS. To apply the performance calculation on a flash drive, use the random disk IOPS, as shown in Table 22, and the bandwidth by drive type to calculate the number of IOPS for the flash drive:
Flash as FAST cache = (495+330*2)/maximum supported IOPS for VNXe = ~2 aligned to 2 disks (RAID 1)
RAID 1 (1+1): (495+330*2) /3,500 = 2 drive total
Example 2: Sizing with FAST Cache
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
65 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
In this example, RAID 1 (1+1) is a default RAID type for FAST Cache, with two flash drives.
Step 3: Determine the required I/O load for non-flash drives
Apply the following formula to calculate the required IOPS served by non-flash drives:
Non-flash IOPS = total IOPS – FAST Cache IOPS
For example, calculate the drive load for database in the following way:
1650 – 825 = 825 IOPS
Table 25. Storage pool calculation example with FAST Cache
Storage pool Number of drives Total capacity (GB)
Oracle data pool RAID 5 (with FAST Cache enabled)
15 SAS drives
12 = (495+330*4)/150
Round up to a multiple of 5 to allow for RAID 5 (4 +1) = 15 drives
600 GB x 15 x 0.8 = 7,200
2 EFD drives
1= (495+330*2)/3,500
Round up to a multiple of 2 to allow for RAID 1 (1+1) = 2 drives
200 GB x 2 x 0.5 = 200
Oracle redo pool RAID 10
4 drives
1 = 2.5 MB/ 20 MB/s
Round up to a multiple of 4 to allow for RAID 10 (2+2) =4 drives
600 GB x 4 x 0.5 = 1200
Determine the capacity The number of drives in a storage system is determined by the needs of both performance and capacity. The method described in Table 25 calculates the minimum number of drives needed to meet performance requirements. The number of drives also needs to be resolved to meet the storage capacity requirement.
Unless a database is unusually large, the storage requirement for performance usually determines the number of data drivers. In this example, the storage solution using the least number of drives from the previous calculation meets the capacity requirements.
Plan for future growth by having enough storage capacity and performance available to satisfy the workload’s near-future requirements. In this example, we included a 5 percent annual growth rate.
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
66 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
This example uses the steps in Example 3: Sizing with FAST VP to create a virtual provisioning pool using the FAST VP feature to calculate the number of drives required to meet both the performance and capacity requirements.
To create tiered pools, enable FAST VP, and then follow these steps:
Step 1: Determine the workload.
The example in Table 25 uses the data from the database entry medium-size Oracle database as shown in Table 19.
Step 2: Determine the required I/O load of the top tier and drive estimate
Allocate the pool resources. The capacity required for each FAST VP tier depends on the locality of the active data. As a starting point, use a capacity per tier of 5 percent flash, 20 percent SAS, and 75 percent NL-SAS. This capacity is sufficient because less than 25 percent of the used capacity will be active and relocations from the lowest tier will occur infrequently.
Note: Locality depends on the statistical distribution between the most frequently accessed and infrequently accessed capacity. The most frequently accessed data is stored on the highest performing storage devices. Occasionally, less frequently accessed data is read or written. This I/O has a longer host response time than more frequently accessed data located on higher performing storage devices.
In this example, as shown in Table 26, we use a two-tiered FAST VP pool, which includes flash and SAS drives. This provides consistently good performance using a-tier pool comprised of flash and SAS. NL-SAS can be added later when capacity growth and aged data are required. We assumed a locality of 10 percent, which means that about 10 percent of the data is used most frequently. In this example, RAID 1 is a default RAID type for FAST VP.
Table 26. Example workload for a two-tier FAST VP pool capacity
Database profile
Database size (GB)
Percent locality Top tier capacity (GB)
Other capacity (GB)
Medium-sized Oracle database
500 10 50 450
Note: During the calculation, we considered the ideal maximum IOPS of flash disks and the real run-test values. For example, the ideal maximum IOPS of a flash disk can be 3,500, but when a flash disk such as FAST VP (upper tier) is used, the usable space for an application is limited. The most frequently accessed data can be larger than the usable space in flash (for example when using 2 x 200 GB FAST Cache SSDs to serve a 500 GB OLTP user database with 300 GB most frequently accessed data). To keep other disks, such as SAS/FC/SATA, storing the less frequently accessed data or used as the lower tier in FAST VP, the actual supported IOPS of the flash disk may be less than 3,000.
Example 3: Sizing with FAST VP
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
67 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection
Design Guide
+Step 3: Determine the required I/O load of the other tiers and drive estimate
Not all host IOPS require the highest tier. The lower tier has a performance margin that can also be productive. Using the workload captured in Table 19, subtract the FAST VP top tier to get an estimate of the IOPS load on the lower tier using the following calculation:
(990+ (660*4))*0.1= 363 IOPS
Always check that the lower tier can sustain the required IOPS load. In this case, a RAID 5 SAS group has about 363 IOPS, which reduces the host response times for data on the lower tier.
Step 4: Determine the capacity of FAST VP pool
Calculate the FAST VP pool storage capacity by adding the capacity of the usable drive space of all the data drives in the tiers. In this example, we used 600 GB capacity SAS drives and 200 GB flash drives. You can substitute the other capacity or speed drives in the calculation.
Note: Database redo logs have a predictable sequential write workload, and this type of activity does not benefit from relocating to flash. EMC recommends that you pin them to their existing tier.
Table 27. Storage pool calculation Example with FAST VP
Storage pool Number of drives Total capacity (GB)
Oracle data pool RAID 5 (with FAST VP enabled)
5 SAS drives
3 = (990+660*4)*0.1/150
Round up to a multiple of 5 to allow for RAID 5 (4 +1) = 5 drives
600 GB x 5 x 0.8 = 2400
2 EFD drives
1= (990+660*2)/3,500
Round up to a multiple of 2 to allow for RAID 1 (1+1) = 2 drives
200 GB x 2 x 0.5 = 200
Oracle redo pool RAID 10
4 drives
1 = 2.5 MB/ 20 MB/s
Round up to a multiple of 4 to allow for RAID 10 (2+2) =4 drives
600 GB x 4 x 0.5 = 1200
The storage solution with the fewest number of drives that meets the performance and capacity requirement is usually the best solution. However, a more reasonable solution matches the additional workload and expected annual data growth. In this example, flash drives with RAID 1 are a better solution that fits the requirements and uses fewer drives. Also, the lower tier performance, which includes NL_SAS(RAID 6), is compatible with extended future loads.
Appendix B: High-level Oracle Database Server Sizing Logic and Methodology
68 EMC VSPEX for Virtualized Oracle Database 12c OLTP Enabled by VNXe and EMC Data Protection Design Guide
Select the right VSPEX Proven Infrastructure
After sizing the application and determining the resources required and the recommended disk storage layouts, use the following steps to select the right VSPEX Proven Infrastructure based on the calculated results:
1. If the customer wants to deploy other applications in the same VSPEX Proven
Infrastructure, refer to the VSPEX Design Guides for those applications to determine the total resources required and the recommended storage layouts for the combined workload.
2. Aggregate the required virtual machine resources (number of disks, total IOPS, and so on) for all applications.
3. Discuss with your customer the virtualization platform they want to use to meet their business requirements.
4. Refer to the EMC VSPEX Proven Infrastructure Guide and calculate the number of disks required for the VSPEX private cloud pool.
5. Aggregate the total number of disks required, including the number of disks for the combined applications, for the VSPEX private cloud pool and for hot spares.
6. Calculate the total number of virtual machines based on the compute and storage resources of the combined applications workload. Refer to the EMC VSPEX Proven Infrastructure Guide for guidance.
Note: Ensure that the selected VSPEX Proven Infrastructure supports the total number of disks required for the combined applications and the private cloud. If it does not, you may need to upgrade to the next VSPEX Proven Infrastructure model. For detailed support, consult your EMC representative.