Top Banner
Infrastructure Specification BOP MFS Inov8 Limited 1/30/2014 Disclaimer: This document contains confidential information about Inov8 Limited, which is provided for the sole purpose of permitting the recipient to evaluate the document submitted herewith. In consideration of the receipt of this document, the recipient agrees to maintain such information in confidence and to not reproduce or
32
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification

BOP MFS

Inov8 Limited

1/30/2014

Disclaimer: This document contains confidential information about Inov8 Limited, which is provided for the sole purpose of permitting the recipient to evaluate the document submitted herewith. In consideration of the receipt of this document, the recipient agrees to maintain such information in confidence and to not reproduce or otherwise disclose this information to any person outside of the recipient's employees directly responsible for the evaluation of its contents. There is no obligation to maintain the confidentiality of any information which is known to the recipient prior to the receipt of such information from Inov8 Limited or becomes publicly known through no fault of the recipient, or is received without obligation of confidentiality from a third party owing no obligation of confidentiality to Inov8 Limited.

Table of ContentsTable of Contents....................................................................................................2

Page 2: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

SIGN-OFF SHEET...................................................................................................4

DOCUMENT INFORMATION..................................................................................5

DOCUMENT REVISION HISTORY.........................................................................5

DEFINITION OF TERMS, ACRONYMS AND ABBREVIATIONS ............................6

1 Introduction..........................................................................................................7

1.1 Overview........................................................................................................7

1.2 Purpose..........................................................................................................7

1.3 Scope.............................................................................................................7

2 Infrastructure Specification..................................................................................7

2.1 Hardware Requirements................................................................................7

2.2 Software Requirements.................................................................................8

2.3 Primary Site Network Architecture Branchless Banking.................................9

2.4 DR Site Network Architecture Branchless Banking......................................10

2.5 Interconnectivity between Partners.............................................................11

2.6 Bandwidth Requirements:............................................................................12

3 System Maintenance and Monitoring:................................................................12

4 Backup Policy:....................................................................................................12

5 Site Switchover:.................................................................................................12

6 DB Maintenance.................................................................................................13

6.1 Architecture:................................................................................................14

6.2 Periodic Maintenance Tasks.........................................................................14

6.2.1 Daily Tasks............................................................................................14

6.2.2 Verify instance status............................................................................14

6.2.3 Verify instance status............................................................................14

6.2.4 Verify Listener Status............................................................................15

6.2.5 Alert Log:...............................................................................................16

6.2.6 Configured metrics and alerts...............................................................17

6.3 RMAN backups.............................................................................................18

6.4 Check for Various Performance indicators:..................................................19

6.4.1 Weekly...................................................................................................19

6.4.2 Weekly Backup......................................................................................19

6.4.3 Re-compile invalid objects.....................................................................20

6.4.4 Tuning:..................................................................................................20

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 2

Page 3: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

6.4.5 Cleaning of alert logs.............................................................................20

6.4.6 Flash Recovery Area:.............................................................................21

6.4.7 Monthly..................................................................................................21

6.4.8 Recovery tests.......................................................................................21

6.4.9 Fragmentation.......................................................................................21

6.4.10 Row chaining and migration:.................................................................21

6.5 Oracle Streams Replication Maintenance:...................................................22

6.5.1 Start or stop a capture process:............................................................23

6.5.2 Enable or disable propagation:..............................................................24

6.5.3 Start or stop an apply process:..............................................................24

6.5.4 Display an overview of the replication components:.............................25

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 3

Page 4: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

SIGN-OFF SHEET

Inov8 Limited and Bank Of Punjab both acknowledge that they have reviewed this

Infrastructure Specification for BOP Mobile Financial Services (BOP MFS) ,

understand it, and that once it is signed by both parties’ Authorized Representatives, agree

to be bound by its requirements scope.

The signature below signifies that the parties involved have accepted this Infrastructure

Specification Document as final, and understand that further changes to the scope will

likely result in a delay in the final delivery date and could result in additional costs

Bank of Punjab

Name Role/Title Date Sign-Off

Inov8 Limited

Name Role/Title Date Sign-Off

Burhan Ahmad Project Manager

Madihah Islam Head of PPQA and Product Delivery

Maqsood Shahzad Head of Implementation & Technical Solutions

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 4

Page 5: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

DOCUMENT INFORMATION CATEGORY INFORMATIONCustomer Bank of PunjabProject BOP Mobile Financial ServicesDocument Infrastructure SpecificationDocument Version 1.0Identifier BOP-MFS-RS-Infrastructure SpecificationAuthor(s) Khawar Baig, Farhan SaleemApprover(s) Madihah Islam, Burhan Ahmad, Maqsood

ShahzadDocument Status Under Client ReviewIssue Date 1/30/2014Document LocationDistribution 1.

2.3.

DOCUMENT REVISION HISTORY

Author Date Version Description (change made, sec Ref, CR#)

Farhan Saleem 1/30/2014 0.1 Added details for Infrastructure Specification and monitoring/maintenance requirements

Khawar Baig 1/30/2014 1.0 Updated with details of DB maintenance and monitoring details

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 5

Page 6: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

DEFINITION OF TERMS, ACRONYMS AND ABBREVIATIONS

This section should provide the definition of all terms, acronyms and abbreviations

required to interpret these terms properly. This information may be provided by

reference to the project glossary.

Terms Description

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 6

Page 7: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

1 Introduction

1.1 OverviewThis document will serve as a subsection of Software Requirement Specification for BOP Mobile Financial Services implementation for Bank of Punjab. This document covers Infrastructure Specification, maintenance and monitoring requirements for BOP MFS implementation of software as a service.

1.2 PurposeThe purpose of this document is to provide a detailed description of hardware, software, network and db maintenance related requirements

1.3 ScopeThe scope of this document is limited to BOP MFS Implementation.

2 Infrastructure SpecificationThis section contains the list of required hardware software and network infrastructure for the proposed solution of BOP branchless banking.

2.1 Hardware RequirementsGiven below is the list of Infrastructure Hardware:

Category

Type Specifications QuantityPrimary + Failover

QuantityTotalH

ardware

Hardware LoadBalancer

IP load balancing, Including HTTP support

2 + 0 2

Hardwar

Network Cabinet Server Rack 2 + 1 3Hardware

Blade Center Chassis/Enclosure

Blade Center Chassis/Enclosure with Fully Redundant

1 + 1

2

Hardware

Blade Server Single Intel Xeon 2.3 GHz, 6C /1333MHz/15MB, 64 GBRAM, HD 600x2, 2 * 1 Gigabit NIC, RAID 1

2 + 1 3

Hardware

Blade Server Dual Intel Xeon 2.3 GHz, 6C /1333MHz/15MB, 64 GBRAM, HD 600x2, 2 * 1Gigabit NIC, RAID 1

6 + 3 9

Hardware

Blade Server Dual Intel Xeon 2.3 GHz, 6C /1333MHz/15MB, 32 GBRAM, HD 600x2, 2 * 1 Gigabit NIC, RAID 1

6 + 3 9

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 7

Page 8: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

Hardware

Storage Array 8 TB Usable with 8Gb FC 1 + 1 2

Hardware

Tape Library Tape library with capacity of 3 TB

1+ 0 1

2.2 Software Requirements

Item No. Category Component Description Unit of Quantity Note

1InfrastructureSoftware

Red HatEnterprise Linux 6

Operating System

Per Server21

14 servers at primary site and 7 servers for failover

2Infrastructure

SSL Certificate

Secure session Per Domain

1

3Infrastructure

Oracle Database11g R2Enterprise

RDMS Database Engine Per Core 9

4Infrastructure

Oracle 11g R2Cluster ware

DB Clustering andAutomatic Storage

Per Core 9

5InfrastructureSoftware

jboss-as-7.1.1

Application Platform

Per Server 12 Open Source

6InfrastructureSoftware

Jdk (v1.7) Java Environment

Per Server 12 Open Source

7InfrastructureSoftware

Kennel SMS Gateway Interface

Per Server 3 Open Source

8InfrastructureSoftware

Piranha Clustering Application

Per Server 8 8 servers at primary site

9 Infrastructure

mod_JKProxy & load balancing module between

Per Server 32 servers at primary site and 1 server at DR

10InfrastructureSoftware

Apache Web server Per Server 32 servers at primary site and 1 serverat DR

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 8

Page 9: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

2.3 Primary Site Network Architecture Branchless Banking

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 9

Page 10: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

2.4 DR Site Network Architecture Branchless Banking

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 10

Page 11: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

2.5 Interconnectivity between Partners

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 11

Page 12: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

2.6 Bandwidth Requirements: The bandwidth between each partner will be 5 MB.

3 System Maintenance and Monitoring: Applying the required updates for system stability Tracking and controlling system resources Maintaining the date and time on all servers Administering users and groups and their access privileges at OS level Administering processes at OS level Using scripts to automate tasks Using NMS for system monitoring and alerts

4 Backup Policy:All systems and application configurations file are backed up once on tape library and will be available to restore at the time of any system failure. And all other file that change on daily will be backed up on a scheduled basis.

5 Site Switchover:The mechanism for site switching is kept manual. The manual site switching mechanism is adopted to avoid the risk of any malpractice from the non-production site in case of keeping all the services alive all the time for automatic switchover.

At the time of site switching the peer IP will be moved on failover site without any intervention of partners.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 12

Page 13: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

6 DB MaintenanceThis section elaborates in detail the procedures/tasks for proactive monitoring, maintenance and troubleshooting of the Database system for ‘BOP’ Branchless Banking (BB) System. These tasks can be categorized into following periodic tasks.

Daily Tasks. Weekly Tasks. Monthly Tasks.

The database system design is based on oracle recommended maximum-availability architecture. The primary aim of the database maximum-availability architecture is to achieve superior data protection and availability by minimizing or eliminating planned and unplanned downtime at all technology stack layers including hardware or software components. Data protection and high availability are achieved regardless of the scope of a failure event - whether from hardware failures that cause data corruptions or from catastrophic acts of nature that impact a broad geographic area.

This architecture involves 2 node RAC database primary and 1 Node stand alone database disaster recovery (DR) sites. The primary site is our current production database. The secondary site is the real time replica of the primary site. In case of disaster to the primary site the secondary site will assume the role of production DB.

There are to two solutions available to synchronize secondary site with the primary production DB, i.e. standby database using Oracle data guard technology and replication using Oracle streams or Multi-master replication using Oracle Advanced Replication techniques.

We are using oracle RAC for load balancing and failover and Oracle Streams replication for data replication between primary and secondary sites.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 13

Page 14: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

6.1 Architecture:The primary database system is comprised of Oracle 11g R2 database 2 nodes RAC connected to SAN mounted by Oracle ASM for DB storage. The DR site comprise of only one database server connected to the SAN using Oracle ASM.

The sites are interconnected on dark fiber. The two databases are kept synchronized by Oracle Stream Real Time data Replication.

6.2 Periodic Maintenance Tasks

Following sections list all periodic maintenance tasks for daily, weekly and monthly maitnenace activities of the database.

6.2.1 Daily Tasks

6.2.1.1Verify instance status

A database instance is a set of memory structures that manage database files. A database is a set of physical files on disk created by the CREATE DATABASE statement. The instance manages its associated data and serves the users of the database.

A database instance includes background processes. Server processes, and the process memory allocated in these processes, also exist in the instance. The instance continues to function when server processes terminate.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 14

DR SANPrimary SAN

Node1

dblink “Primary_to_DR”

dblink “Dr_to_Primary”

Drdb1

Node2

Failover sitePrimary Site

Page 15: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

Therefore to connect database server from any network location the instance should be up and running. The status of instance should be verified to be up, to allow the application to connect to the database. The following steps should be performed to check the status of the database instance:

Connect to the any database node via SSH client (putty)

At the Linux bash shell enter

$ sqlplus

The sqlplus will be started, now enter the command

Sqlplus>select instance_name,status from gv$instance;

INSTANCE_NAME STATUS

---------------- ------------

INOV8 OPEN

The possible values for the command are open and close.

Possible outcome Required action1 OPEN If the command outcome value is OPEN then it’s ok. Then

Exit from sqlplus. No further action is required

2 Close If the command outcome value is CLOSE then, there is a need to start the instance.Enter the following command to start the instance.

Sqlplus> startup

If the last line of the command outcome is Database Open, then it’s ok. If command results in error then:

1. Refer to the oracle database error messages 11g release 2 documents to find the reason of the error.

2. If issue is not resolved then engage the inov8 team.

6.2.1.2Verify Listener StatusOnce it’s verified that oracle instance is up, you have to make sure the oracle listener is started and connected to the current database instance. Oracle Net Listener is a separate process that runs on the database server computer. It receives incoming client connection requests and manages the traffic of these requests to the database server. At Linux bash shell issue the command to verify the status of the listener:

$ Lsnrctl status

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 15

Page 16: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

The possible outcomes for this command are:

Possible outcome Required Action1 Instance "inov8", status READY,

has 1 handler(s) for this service...

Ok. The listener is connected to the database instance. No further action is required.

2 The listener supports no services

The listener is started but is not connected to the database instance.Enter the command $Lsnrctl reload

3 ORA-12541 TNS :no listener The listener is not started. start the listener with the command$listener start

6.2.1.3Alert Log:

The alert log file (also referred to as the ALERT.LOG) is a chronological log of messages and errors written out by an Oracle Database. Typical messages found in this file are: database startup, shutdown, log switches, space errors, etc. This file should constantly be monitored to detect unexpected messages and corruptions. Alert logs file is stored in the following location:

/u01/app/oracle/diag/rdbms/inov8/inov8/trace/alert_inov8.log

Or V$DIAG_INFO dictionary view can be queried to know the alert log and other diagnostic trace files paths.

select * from V$DIAG_INFO;

Open the file and move to the end. Check the file for any alerts or errors in the last 24 hours.

If there is any alert or error investigate further to take the necessary action.

Or Oracle Enterprise manager can also be used to view the latest alert log contents at https://localhost:1158/em.

The alert log is a chronological log of messages and errors, and includes the following items:

All internal errors (ORA-600), block corruption errors (ORA-1578), and deadlock errors (ORA-60) that occur.

Administrative operations, such as CREATE, ALTER, and DROP statements and STARTUP, SHUTDOWN, and ARCHIVELOG statements.

Messages and errors relating to the functions of shared server and dispatcher processes

Errors occurring during the automatic refresh of a materialized view.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 16

Page 17: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

The values of all initialization parameters that had no default values at the time the database and instance start

If there is any internal errors (ORA-600), block corruption errors (ORA-1578) then it should be communicated to the DBA or inov8 team.

6.2.1.4Configured metrics and alertsThe database is configured to generate the automated warnings and alerts for the following thresholds:

Archive Area Used (warning at 80 percent full) Broken Job Count and Failed Job Count (warning when goes above 0) Current Open Cursors Count (warning when goes above 1200) Dump Area Used (warning at 95 percent full) Session Limit Usage (warning at 90 percent, critical at 97 percent) Tablespace Space Used (warning at 85 percent full, critical at 97 percent full)

Check for any warnings and alerts generated in the last 24 hours by using the following query:

select object_type, object_name, reason, suggested_action, time_suggested, resolution, advisor_name, metric_value, message_type, message_group, message_level from dba_alert_history where creation_time > sysdate-1 and creation_time < sysdate --and resolution = 'cleared' order by creation_time desc

Use the following query to query all database performance metrics values in last 24 hours:

select metric_name,metric_unit,round(avg(value)) as avg_value,round(min(value)) as min_value,round(max(value)) from V$METRIC_HISTORYwhere begin_time > sysdate-1and begin_time < sysdate

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 17

Page 18: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

group by metric_name,metric_unit;

6.2.1.5RMAN backups

For the purposes of backup and recovery oracle provided tool Oracle Recovery Manager (RMAN) is used. Oracle Recovery Manager satisfies the most pressing demands of performant, manageable backup and recovery, for all Oracle data formats.

A complete high availability and disaster recovery strategy requires dependable data backup, restore, and recovery procedures. Oracle Recovery Manager (RMAN) provides a comprehensive foundation for efficiently backing up and recovering the Oracle database. It is designed to work intimately with the server, providing block-level corruption detection during backup and restore. RMAN optimizes performance and space consumption during backup with file multiplexing and backup set compression, and integrates with Oracle Secure Backup, as well as third party media management products, for tape backup.

RMAN takes care of all underlying database procedures before and after backup or restore, freeing dependency on OS and SQL*Plus scripts. It provides a common interface, via command line and Enterprise Manager, for backup tasks across different host operating systems and offers features not available through user-managed methods, such as parallelization of backup/restore data streams, backup files retention policy, and detailed history of all backups.

Production database is configured to take a cumulative incremental backup of the whole database daily at 10:00 PM. The RMAN backup set is stored at the

/u01/app/oracle/admin/flash_recovery_area/inov8/Backupsets

Check if the last night cumulative backup was successful. The backup set is then copied to the tape drive via BA client at 12:00 AM

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 18

Page 19: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

The above figure elaborates the backup strategy for this database system. A full level 0 database + archive log backup is taken on Sunday and a cumulative incremental backup of the whole database is performed daily in the week days.

6.2.1.6Check for Various Performance indicators:Check for these performance indicators in the enterprise manager console:

Storage CPU contention waiting times memory usage network load I/O stat

6.2.2Weekly

6.2.2.1 Weekly BackupThe RMAN is configured to take a full backup of the database and archive logs on every Sunday 10:00 PM .The RMAN backup set is stored at the

/u01/app/oracle/admin/flash_recovery_area/inov8/Backupsets

The backup set is then copied to the tape drive via Tivoli BA client at 12:00 AM

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 19

Page 20: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

The “manage current backups” page of the EM or list backup set command of RMAN is used to check the backup status.

6.2.2.2 Re-compile invalid objects Run the query

SELECT owner ,object_type ,object_name FROM dba_objects WHERE status != 'VALID' and

owner in (<Shema_Name>) order by owner, object_type ;

And look into the objects which have become invalid. Re-compile the objects if there is a need to recompile the objects. In case of invalid indexes, there is a need to rebuild the particular indexes.

To recompile all the invalid objects other than indexes use:

SELECT 'ALTER ' || OBJECT_TYPE || ' ' || OWNER || '.' || OBJECT_NAME || ' COMPILE;'

FROM dba_objects where status = 'INVALID' and owner in (<Schema_Name>);

And execute the resultant alter statements.

In order to rebuild the invalid index, execute the following command for the particular index:

ALTER INDEX <index_name> REBUILD [online];

6.2.2.3 Tuning: indexes and execution plans Top SQL

The top sql are the queries which are consuming a lot of resources.

The top SQL can be found in TOP SQL page of the oracle enterprise manger or the following query can be used to find the top SQL in terms of elapsed time.

SELECT * FROM(SELECT sql_fulltext, sql_id, child_number, disk_reads, executions, first_load_time, last_load_timeFROM v$sqlORDER BY elapsed_time DESC)WHERE ROWNUM < 10;Run the aforementioned query and analyze and try to tune the SQL.

6.2.2.4 Cleaning of alert logsArchive the alert log file every Monday with the naming convertion

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 20

Page 21: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

Alert_inov8_<from_date>to<to_date>.

6.2.2.5 Flash Recovery Area:Check the size of files in the flash recovery area.

If the flash recovery area files are approaching the max size allocated to the flash recovery area.

Move the archive logs and backup sets to some other location and free the area from RMAN by running crosscheck and delete commands from RMAN prompt.

Also check that:

The already backed up archive log are deleted from the flash recovery area.

The obsolete back upsets are deleted from the flash recovery area.

6.2.3 Monthly

6.2.3.1 Recovery testsA complete recovery test should be performed at the last backup of the whole database.

6.2.3.2 FragmentationCheck if there are any segments which are fragmented. If there are then try to reclaim the space from those segments by shrink space command.

Alter table <table_name> enable row movement;

Alter table <table_name> shrink space;

If the indexes have become unusable than rebuild the specific index.

6.2.3.3Row chaining and migration:Identify the tables with migrated/chains rows by issuing the following query:

set pages 9999;column c1 heading "Owner" format a9;column c2 heading "Table" format a12;column c3 heading "PCTFREE" format 99;column c4 heading "PCTUSED" format 99;column c5 heading "avg row" format 99,999;column c6 heading "Rows" format 999,999,999;column c7 heading "Chains" format 999,999,999;column c8 heading "Pct" format .99; set heading off;select 'Tables with migrated/chained rows and no RAW columns.' from dual;set heading on;select

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 21

Page 22: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

owner c1, table_name c2, pct_free c3, pct_used c4, avg_row_len c5, num_rows c6, chain_cnt c7, chain_cnt/num_rows c8from dba_tableswhereowner not in ('SYS','SYSTEM')andtable_name not in (select table_name from dba_tab_columns where data_type in ('RAW','LONG RAW') )andchain_cnt > 0order by chain_cnt desc;When the tables or indexes with migrated or chained rows have been identified then try reorganizing the objects.

Try to move the tables to another tablespace by alter table move command and rebuild the indexes suffering the same problem.

6.3 Oracle Data Guard Maintenance:

Oracle Data Guard ensures high availability, data protection, and disaster recovery for enterprise data. Data Guard provides a comprehensive set of services that create, maintain, manage, and monitor one or more standby databases to enable production Oracle databases to survive disasters and data corruptions. Data Guard maintains these standby databases as copies of the production database. Then, if the production database becomes unavailable because of a planned or an unplanned outage, Data Guard can switch any standby database to the production role, minimizing the downtime associated with the outage. Data Guard can be used with traditional backup, restoration, and cluster techniques to provide a high level of data protection and data availability.

A Data Guard configuration consists of one production database and one or more standby databases. The databases in a Data Guard configuration are connected by Oracle Net and may be dispersed geographically. There are no restrictions on where the databases are located, provided they can communicate with each other. For example, you can have a standby database on the same system as the production database, along with two standby databases on other systems at remote locations.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 22

Page 23: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

You can manage primary and standby databases using the SQL command-line interfaces or the Data Guard broker interfaces, including a command-line interface (DGMGRL) and a graphical user interface that is integrated in Oracle Enterprise Manager.

1.Primary Database

A Data Guard configuration contains one production database, also referred to as the primary database, that functions in the primary role. This is the database that is accessed by most of your applications.

The primary database can be either a single-instance Oracle database or an Oracle Real Application Clusters (Oracle RAC) database.

2.Standby Databases

A standby database is a transactionally consistent copy of the primary database. Using a backup copy of the primary database, you can create up to thirty standby databases and incorporate them in a Data Guard configuration. Once created, Data Guard automatically maintains each standby database by transmitting redo data from the primary database and then applying the redo to the standby database.

Similar to a primary database, a standby database can be either a single-instance Oracle database or an Oracle RAC database.

The types of standby databases are as follows:

Physical standby database

Provides a physically identical copy of the primary database, with on disk database structures that are identical to the primary database on a block-for-block basis. The database schema, including indexes, are the same. A physical standby database is kept synchronized with the primary database, through Redo Apply, which recovers the redo data received from the primary database and applies the redo to the physical standby database.

As of Oracle Database 11g release 1 (11.1), a physical standby database can receive and apply redo while it is open for read-only access. A physical standby database can therefore be used concurrently for data protection and reporting.

Logical standby database

Contains the same logical information as the production database, although the physical organization and structure of the data can be different. The

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 23

Page 24: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

logical standby database is kept synchronized with the primary database through SQL Apply, which transforms the data in the redo received from the primary database into SQL statements and then executes the SQL statements on the standby database.

A logical standby database can be used for other business purposes in addition to disaster recovery requirements. This allows users to access a logical standby database for queries and reporting purposes at any time. Also, using a logical standby database, you can upgrade Oracle Database software and patch sets with almost no downtime. Thus, a logical standby database can be used concurrently for data protection, reporting, and database upgrades.

Snapshot Standby Database

A snapshot standby database is a fully updatable standby database.

Like a physical or logical standby database, a snapshot standby database receives and archives redo data from a primary database. Unlike a physical or logical standby database, a snapshot standby database does not apply the redo data that it receives. The redo data received by a snapshot standby database is not applied until the snapshot standby is converted back into a physical standby database, after first discarding any local updates made to the snapshot standby database.

A snapshot standby database is best used in scenarios that require a temporary, updatable snapshot of a physical standby database. Note that because redo data received by a snapshot standby database is not applied until it is converted back into a physical standby, the time needed to recover from a primary database failure is directly proportional to the amount of redo data that needs to be applied.

Verify Data guard Configurations

1. Click on the Verify configuration link.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 24

Page 25: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

Upon successful completion the status will be as displayed on the screenshot below. We confirmed that the physical standby database is created successfully.

2. Data Guard Performance

Click on the Performance link and start a test application to generate a load on the primary database. What we are interested in are the redo generation rate, redo apply rate and the transport and apply lag.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 25

Page 26: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

After running for a while the application generating a load on the primary database we will have more meaningful data gathered as displayed on the screenshot below.

We can further drill down and look at details of the Transport Lag.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 26

Page 27: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

We can further drill down and look at details of the redo generation rate.

We can further drill down and look at details of the redo apply rate.

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 27

Page 28: BOP-MFS-Infrastructure Specification.docx

Infrastructure Specification Handbook

Copyrights 2014 © Inov8 Limited www.inov8.com.pk Page 28