Top Banner
@ViscosityNA www.viscosityna.com Bulletproof Your Data Guard Configuration 12.2 / 18c /19c RMOUG February 2020 [email protected] Charles Kim, CEO, Viscosity North America Oracle ACE Director Oracle Platinum Partner
112

Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Mar 22, 2023

Download

Documents

Khang Minh
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNAwww.viscosityna.com

Bulletproof Your Data Guard Configuration 12.2 / 18c /19c

RMOUG

February 2020 [email protected]

Charles Kim, CEO, Viscosity North America Oracle ACE Director

Oracle Platinum Partner

Page 2: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

• Oracle ACE Director

• Founder and CEO of Viscosity

• Over 27 years of Oracle Expertise: Mission Critical Databases,

RAC, Data Guard, ASM, RMAN, Shareplex/GoldenGate

• Specialize in “Complex Data Replication & Integration” with Shareplex & GG

• Oracle Management Cloud Certified

• Oracle Exadata Certified Implementation Specialist, (2014, 2016)

• Oracle Certified RAC Expert

• Oracle Linux Certified Expert

Charles Kim  @racdba

Page 3: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

We’ve written 23+ books on Data, Cloud, and Oracle…

Industry Expertso Viscosity founders hold 25+ years each in

the Oracle space

o Authors of 23+ books in the Oracle space

o 4 ACE Directors, only 36 in the United States o +2 ACE o And adding more :-)

o SharePlex Platinum Partner

o Direct connections to Oracle support and Product Managers support

o Expertise in Oracle 11g, 12c, 18c, 19c, RAC, ASM, Data Guard, Zero Downtime Upgrades, Performance Tuning, and much more

Page 4: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

http://viscosityna.com/resources/dba-resources/twelve-days-12-2/

Page 5: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Quarterly Tech Deep DivesQ1 2017

Page 6: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

What Happens Online in 60 Seconds? Why it is more important to have a DR Strategy

6

Page 7: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Oracle 19c (On-Premise Exadata) - Feb 13, 2019

https://blogs.oracle.com/oracle-database/oracle-database-19c-now-available-on-oracle-exadata

Page 8: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

We wrote the books

8

Page 9: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

4 Quarters of Tech Deep DivesReleased @OpenWorld 2017 2nd Edition @OpenWorld 2019

3rd Edition for IOUG Collaborate - April 2020

PDB Me ASAP! A Lazy DBA’s Guide to Mastering Multitenant Features of Oracle Database 12cR2

Page 10: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

The Cloud Journey Hybrid Cloud

Page 11: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Evolution of the DBA

11

Kind of DBA Timeline

CLI DBA Early 90’s DBAs

GUI DBA Late 90’s and Dot Com

Google DBA Dot Com and 2000’s

iDBA Dot Com, IOUG iDBA Master Curriculum

RAC DBAs (MAA DBAs) 2000+ after 9.2 (but major spike with 10.2) + Data Guard

DMA 2010+ Database Machine Administrator

vDBA / vRAC DBA 2010+ Evolving role of a DBA in the virtual world

Cloud DBA 2011+ Database Consolidation with Private Database CloudOracle Database 12c Launches June 2013

Public Cloud DBA 2015+ Oracle Public Cloud with Database Cloud Service, Database Backup Cloud Service, Storage Cloud Service, IaaS Cloud Service

PDBAs 2017+ Multi-Tenant with Oracle Database 12c Release 2GA – March 15, 2017

“It is not the strongest or the most intelligent who will survive but those who

can best manage [adapt to] change.”

Page 12: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Oracle Release DatesOracle Database 11g Release 1 Aug 2007

Oracle Database 11g Release 2 Sept 2009

Oracle Database 12c Release 1 June 2013

Oracle Database 12c Release 1 (Patchset) - 12.1.0.2 June 2014

Oracle Database 12c Release 2 March 2017

Oracle Database 12c Release 2 First Bundled Patch

May 2017

Oracle Database 12c Release 2 RU (July 18)

July 2017

Oracle 18c - “Cloud First” February 2018

Oracle 18c – Available on Exadata February 2018

Oracle 18c – Available on ODA March 2018

Oracle 18c – On-Premise July 2018

Oracle 19c - Exadata On-Premise Oracle 19c - General Availability - Terminal Release

Feb 13, 2019 April 2019

Page 13: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Oracle Database19c 3 PDBs - FREE TO USE

❑Starting in Oracle Database 20c ❑Containerized databases are a requirement ❑Starting in Oracle Database 19c, each container database can have up to 3 pluggable databases

❑Change how we perform database upgrades ❑Change how we do data center migrations ❑Change how we perform patches

Page 14: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Upgrade Plans Upgrade to Oracle Database 19c?

Factoids• 19c is the Terminal Release of Oracle 12.2• E-Business Suite is certified with Oracle 19c

• E-Business Suite customers on 12.1.0.2 have Premier support until 2020

• What version are you going to upgrade to?

So what is the delimma? 12.2

18c

19c

Page 15: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Pre-Upgrade/Migrate Considerations Always Consider Minimal or Zero Downtime Approach• Set Guaranteed Restore Points accordingly

• Ensure AWR retention is long enough to capture historical data for comparison

• Backup your database statistics, in case an older SQL plan needs restoring

• Definitely consider RAT for real workload testing, instead of synthetic workloads tests

• If upgrading older version of RAC to new version in place, special consideration need to be understood

• 9i RAC to 12c RAC• Upgrade to 12c GI stack first, Flex ASM or not• Legacy clients may not support new Net connections

Page 16: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

• Oracle Database 19c Release Update & Release Update Revision October 2019 Known Issues (Doc ID 2568305.1)

Last updated on JANUARY 17, 2020Follow N-1

19.6 is available

Page 17: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

19c Recommended Patches

Page 18: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Lifetime Support Commitments and Plans Planned Release and Support Timelines

2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027

11.2.0.4

12.1.0.2

EXTENDED

EXTENDED

12.2 EXTENDED

12.2.0.1

18c

19c EXTENDED*

20c

Premier

Waived Extended Support Fee

Paid Extended Support

*Oracle Database 19c and 22c is expected to the long-term support release. Always check MOS Note 742060.1 for the latest schedule

21c

22c

12.2.0.2Nov 30, 2020

12.2.0.3March 31, 2023

No Extended Support

No Extended Support

Page 19: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNAviscosityna.com

Oracle 19c RAC and SE2 De-Support

• SE does not provide all the HA capabilities in Oracle Database

• SE2 Socket Requirements are hard to meet (as hardware evolves)

• SE RAC has been diminishing demand

• MOS: 2504078.1 Desupport of Oracle Real Application Clusters (RAC) with Oracle Database Standard Edition 19c (Doc ID 2504078.1)

Page 20: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA Text “12C” to 444999 for the session slides!

Oracle 19c Autonomous DatabaseExtra 9 from Data Guard

• Continuous Adaptive Performance Tuning • Fully Automated Hardware Resource Elasticity

• SLA: 99.995 % Uptime leveraging – Exadata/RAC/ASM– Active Data Guard– RAC Rolling Upgrade– Transient Logical Standby– Online and Edition Based Re-definiition – Flashback transaction, table, or database

– KSplice on the server side

20

Page 21: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

99.995% Uptime

Availability % Downtime per year Downtime per month

99.95% ("three and a half nines") 4.38 hours 21.56 minutes

99.99% ("four nines") 52.56 minutes 4.38 minutes

99.995% ("four and a half nines") 26.28 minutes 2.16 minutes

99.999% ("five nines") 5.26 minutes 25.9 seconds

Page 22: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Read Mostly Physical Standby

October 7, 2019 22

Page 23: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

19c: Data Guard DML Re-Direction

• RUN DML on ADG

• Re-Direct DML statement back to the primary database • Data changes will happen on the primary database • Changed blocks will ship to ADG • ADG wil be in sync to maintain redundancy

Set  ADG_REDIRECT_DML initialization parameter = TRUE SQL> ALTER SESSION ENABLE ADG_REDIRECT_DML;

Page 24: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Additional 19c New Features

Hybrid Partitioned tables Enables partitions to reside in both Oracle Database segments and in external files and sources. — Enhances functionality of partitioning for Big Data SQL where large portions of a table can reside in external partitions.— Move non-active partitions to external files, such as Oracle Data Pump files, for a cheaper storage solution— Move non-active partitions to Big Data

Flashback physical standby when the primary database is flashed back

In previous releases, getting the secondary to the same point in time as the primary requires a manual procedure to flash back standby databases. A new parameter is introduced which enables the standby database to be flashed back automatically when Flashback Database is performed on the primary database.

Propagate restore points to the standby Restore points and GRP is stored in the control file and not propagated to the standby site

With Failover, the standby becomes the new primary and restore point information is LOST

Now, the restore points are preserved after a failover

MIRA + In-Memory ADG

It its one or the other before

Now you can have best of both

Page 25: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Additional 19c New Features

Observer in Observe Only Mode

Configure it to Observe only creating a 'test mode' to see when a failover or other interaction would have occurred during the normal production processing

- Can tune FSFO for finer granularity - Can see IF FSFO would have actually occurred without production impact - Improve failover validation

Page 26: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Additional 19c New Features

Dynamically Change FSFO target With 12.2, we can have multiple FSFO targets

With 19c, we can change the targets without FSFO being disabled

PDB Recovery Catalog PDBs are now supported as a target database

Can be used with Virtual Private Catalog (VPC)

Page 27: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Additional 19c New Features

Clear Flashback logs periodically to increase FRA size predictability

Starting in 19c, the management of space in the fast recovery area is simplified

Oracle Database monitors flashback logs in the fast recovery area and automatically deletes flashback logs that are beyond the retention period. When the retention target is reduced, flashback logs that are beyond the retention period are deleted immediately.

Flashback physical standby when the primary database is flashed back

In previous releases, getting the secondary to the same point in time as the primary requires a manual procedure to flash back standby databases. A new parameter is introduced which enables the standby database to be flashed back automatically when Flashback Database is performed on the primary database.

New Parameters for Tuning Automatic Outage Resolution

DATA_GUARD_MAX_IO_TIME sets the maximum number of seconds that can elapse before a process is considered hung while performing a regular I/O operation in an Oracle Data Guard environment. Regular I/O operations include read, write, and status operations.

DATA_GUARD_MAX_LONGIO_TIME sets the maximum number of seconds that can elapse before a process is considered hung while performing a long I/O operation in an Oracle Data Guard environment. Long I/O operations include open and close operations.

Page 28: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Oracle Database 18c New Features

Sharded RAC Logically partition data across instances in RAC

Per-PDB Switchover

Transportable Backups Rolling patches for OJVMIntegration with Active Directory Authorization to database through Active Directory user/group mappings to database schema users and roles

Per PDB Key storage Password-less schema creation No default passwords

In-Memory for external tables Automatic propagation of nologging data to standby

Alter Table Merge Partition Online Private Temporary Tables

Page 29: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNAviscosityna.com

18c: Scalable Sequencesselect hca_seq_scale_01.nextval from dual;

select sys_context('USERENV', 'INSTANCE'), sys_context('USERENV', 'SID')from dual;

Let’s dissect the sequence number:101808 0000000000000000000003

Page 30: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

18c: Scalable SequencesLet’s dissect the sequence number:101808 0000000000000000000003 = total 28 digits

6 digit scalable sequence offset numberNormal sequence number – 22 digits

Let’s dissect the scalable sequence offset:101 808

3 digit instance offset number: [ (instance id % 100) + 100 ] => 1013 digit session offset number: ( session id from session information % 1000 ) = 1808/1000 = remainder of 808

Page 31: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

18c Scalable SequenceSyntax: • create or alter sequence

– SCALE|NOSCALE ! NOSCALE is the default

– EXTEND | NOEXTEND ! NOEXTEND is the default – create sequence hca_seq_scale_noextend_01 scale maxvalue 999999999999;

– select hca_seq_scale_noextend_01.nextval from dual;

create sequence hca_seq_scale_extend_01 scale extend maxvalue 999999999999;select hca_seq_scale_extend_01.nextval from dual;

102104 0000000000016 digit offset12 digit sequence12 x 9’s in example

Page 32: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Oracle Database 18c New Features

Zero Impact Grid Infrastructure Patching Per-PDB Switchover

Sharded RAC Logically partition data across instances in RAC User Defined Sharding

Page 33: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Oracle Database 18c Additional Features

Shadow Lost Write Protection -DB, TS or Datafile

Transportable Backups

Backups from non-CDBs are usable after migration to CDB

- Backups on source DB are PREPLUGIN backups

RMAN duplicate PDB into existing CDB Backup to archive storage

- Enhances Oracle SBT Library to Archive Storage Cloud

Manual termination of run-away queries Manually kill a statement without breaking the session:

ALTER SYSTEM CANCEL SQL.

ALTER SYSTEM CANCEL SQL ‘SID, SERIAL, @INST_ID, SQL_ID’;

New parallel statement queue timeout and dequeue actions

Page 34: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Multi-Instance Redo Apply Supports Use of Block Change Tracking Files

for RMAN Backups

MIRA + BCT = ADG (Best of both worlds)

• RMAN block change tracking file can now be enabled on an Oracle Active Data Guard standby that is using multi-instance Redo Apply

• Fastest redo apply technology + incremental backup technology on the same Oracle Active Data Guard

Page 35: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

R2

Upgrading / Patching Considerationsfor Data Guard

Page 36: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

PSUs are Very Important for Data Guard (N-1) Note: 756671.1 - Oracle Recommended Patches - Oracle Database http://www.dbaexpert.com/blog/master-reference-for-psus/

FYI: 12.2 Release Date - Release Schedule of Current Database Releases (Doc ID 742060.1)

12.1.0.2

11.2.0.4

Page 37: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...
Page 38: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Data Guard Standby - First Patch Apply (11.2) Doc ID 1265700.1 - Data Guard Standby-First Patch Apply

What is allowed:• 11.2.0.4 JulPSU to 11.2.0.4

OctPSU• 12.1.0.2 JulPSU to 12.1.02

OctPSU

What is NOT allowed:• 11.2.0.3 JulPSU to 11.2.0.4• 11.2.0.4 JulPSU to 12.1.0.2

38

Starting in Oracle Database 11.2.0.1 and later• Apply candidate patch(see below) first on the standby

database (Binary Only)• Evaluate candidate patch• Switchover• Apply patch(es) on old primary database including DataPatch• Switchover (Switch Back)

Candidates Patches:• Database home interim patches• Exadata bundle patches (e.g. Monthly and quarterly database

patches for Exadata)• Database patch set updates (PSUs)

Read the README.html file

Page 39: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

R2

History

Page 40: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Evolution of Data Guard

40

Oracle Version Big Feature Added

Oracle 7.3 Hot Standby

Oracle 8i Read-Only Physical StandbyManaged Recovery

Oracle 9i Logical Standby, Switchover/FailoverAutomatic Gap ResolutionCascaded Redo

Oracle 10g Real-time applyStandby Redo Logs FSFOFlashback Database on DG

Oracle 11g Active Data GuardRedo CompressionSnapshot StandbyRolling Transient UpgradesRMAN Duplicate from Active Database

Oracle 12.1.0.1 Far SyncFast Sync (LOG_ARCHIVE_DEST_n attributes SYNC and NOAFFIRM)Data Guard Broker Switchover EnhancementsSYSDG RoleOnline Standby Database File RenameSignificantly Improved Rolling Upgrades (DBMS_ROLLING)

Page 41: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Evolution of Data Guard

41

Oracle Version Sequences in Oracle Active Data GuardOracle 12.1 Sequences on the primary database must be defined with:

• Cache • NoOrder

• Set Cache Size appropriately like we do in RAC• Goes back to the primary database when requesting next set

• Restriction• Cannot use NoCache or Order for sequences

Page 42: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Evolution of Data Guard

42

Oracle Version Some Feature AddedMore 12c Release 1 Continued

Alter Database Move Datafile (New in 12c) • Does not impact the standby database• 1* alter database move datafile '+DATA/VCE/DATAFILE/mydata.301.932660559' to '+FRA’ SQL> /

Database altered.

SQL> select file_name from dba_data_files where tablespace_name='MYDATA';

FILE_NAME ----------------------------------------------------+FRA/VCE/DATAFILE/mydata.279.932660601

• DML Operations on Temporary Tables on Oracle Active Data Guard Instances

Page 43: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Additional DG 12.1 New Features Overview

• Resumeable Switchover

• Can do DML operations on Temporary tables on ADG instances

• Do not need using current logfile for real-time apply• DEFAULT: alter database recover managed standby database using current logfile

disconnect;

• Can move location of an online data file … moves on the primary database do not affect the standby

• Can do Incremental Updates over TNS from the standby database

• Can use sequences on ADG• Cache • NoOrder

43

Page 44: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Oracle 12.1: Far Sync InstanceCommon Use Case

Instance

Page 45: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Far Sync (12.1) Zero Data Loss

• SYSDG (12.1) • STARTUP• SHUTDOWN• ALTER DATABASE• ALTER SESSION• ALTER SYSTEM• CREATE RESTORE POINT (including

GUARANTEED Restore Points)• CREATE SESSION• DROP RESTORE POINT (including

GUARANTEED Restore Points)• FLASHBACK DATABASE• SELECT ANY DICTIONARY (DBA_ Views)• SELECT

• X$ Tables• V$ and GV$ Views• APPQOSSYS.WLM_CLASSIFIER_PLAN

• DELETE on APPQOSSYS.WLM_CLASSIFIER_PLAN• EXECUTE SYS.DBMS_DRS

• dgmgrl

45

SYNC NOAFFIRMLimited Distance

ASYNCAny DistanceFar Sync

InstanceStandby

DBTransport compression

Over WAN

PrimaryDB

alter database create far sync instance controlfile as ‘/tmp/farsync.ctl’;

Page 46: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Far Sync Zero Data Loss

46

SYNC NOAFFIRMLimited Distance

ASYNCAny

Distance

Far SyncInstance

StandbyDB

Transport com

pressionO

ver WA

N

PrimaryDB

Primary DB Server: vna85 - Primary DB Name: JIN01FS Server: vna84 - Far Sync Server: JIN01fsPhysical Standby server: ika82 - Physical Standby Name: JIN01_DR

• create configuration JIN01_DGCONFIG as primary database is "JIN01" connect identifier is JIN01;

• add database "JIN01_DR" as connect identifier is JIN01_DR maintained as physical;

• enable configuration;

• add far_sync "JIN01fs" as connect identifier is JIN01fs;• enable far_sync "JIN01fs";

Configure DR to receive from Far Sync using RedoRoutes• EDIT FAR_SYNC "JIN01fs" SET PROPERTY

RedoRoutes='(JIN01:JIN01_DR ASYNC)';

Configure primary to only ship to Far Sync and modify to SYNC• EDIT DATABASE "JIN01" SET PROPERTY RedoRoutes ='(LOCAL:JIN01fs

SYNC)';

Page 47: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

R2

Page 48: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Evolution of Data Guard

48

Oracle Version Some Feature AddedOracle 12.2(Some of my favorites)

Block Comparison tool in DGMGRL

Multiple ObserversMultiple Automatic Failover TargetsFast-Start Failover in Max Protection ModeSupport for Transparent Destinations of Different Endianess • Cross Endian Data Guard (I.e AIX to ZDLRA)

In-Memory in Active Data Guard• Offload even more

Support for Diagnostic Pack (AWR) on Active DGSupport for Tuning Pack Features and SQL Plan Analyzer

OEM Support for Database Rolling UpgradesDG Broker Support for Active DG Rolling UpgradesDG Broker will take over session draining (switchover to DALLAS_DR wait;)

Page 49: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Evolution of Data Guard

49

Oracle Version Some Feature Added

Oracle 12.2More Favorites

Create standby databases with DBCA

Multiple Instance Redo Apply (MIRA)

Subset Standby Databases* Can do GRP at PDB Level now•Password Automatic Sync

Duplicate Database command to create farsync instance with RMAN

Data Guard Broker can do Scripting like SQL*Plus

Page 50: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Data Guarddbca

• Create standby databases with DBCA – Must use command line dbca interface– Can only create from Non Multi-tenant primary

database– Cannot do RAC– dbca -createDuplicateDB -gdbName global_database_name -primaryDBConnectionString

easy_connect_string_to_primary -sid database_system_identifier [-createAsStandby [-dbUniqueName

db_unique_name_for_standby]] [-customScripts scripts_list]

• -createAsStandby followed by dbUniqueName For Standby

• [-customScripts scripts_list] • List of scripts to be executed• Oracle will execute the scripts in the

order they are listed

– No Need for TNSNAMES.ORA entry– Can use easy connect string– "host[:port][/service_name][:server][/instance_name]"

Page 51: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Data Guarddbca

$ dbca -silent -createDuplicateDB \ -gdbName viscorcl \ -primaryDBConnectionString rac01:1521/viscorcl \ -sid viscorcl -createAsStandby \ -dbUniqueName sviscorc Enter SYS user password: Listener config step 33% complete Auxiliary instance creation 66% complete RMAN duplicate 100% complete Look at the log file "/u01/app/oracle/cfgtoollogs/dbca/sviscorc/viscorc.log" for further details.

Page 52: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Data GuardNoLogging & MIRA

NoLogging on Primary • Nologging blocks are recorded in the control

file on the physical standby database • On the Standby RMAN RECOVER DATABASE NONLOGGED BLOCK

• After switchover, the old primary database must be in a mounted state

Multiple Instance Redo Apply (MIRA) • Can run Redo Apply in some or all of the

standby instances • Redo Apply scales as wide as standby RAC

configuration • ALTER DATABASE RECOVER MANAGED

STANDBY DATABASE command now accepts a new INSTANCES [ ALL | integer] clause

• All the instances must be in the same mounted or open mode

• cannot specify which RAC instance(s) will perform the redo apply

Page 53: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Standby Nologging Standby Nologging tells the database not to log operations that qualify to be done without

logging. Standby Nologging tells the database to send the data blocks created by the

Nologging operation to each qualifying standby database in Data Guard configuration This typically results in those standbys NOT having invalid blocks. You can set standby no logging for load performance or data availability in the following

statements: ▪ ALTER DATABASE ▪ ALTER PLUGGABLE DATABASE ▪ CREATE DATABASE ▪ CREATE CONTROLFILE

53

18c Oracle

Database

Page 54: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Standby Nologging Database nologging extended for better use with Oracle Active Data Guard environment

(without significantly increasing the amount of redo generated). There are two new nologging modes:

Standby Nologging for Load Performance – Standbys receive non-logged data changes (minimum impact on loading speed at). Non-logged blocks automatically resolved by managed standby recovery. Standby Nologging for Data Availability - Standbys have data when primary load commits

(at the cost of throttling the speed of loading data at the primary), which means the standbys never have any non-logged blocks to worry about. Nologging can be used when loading data into your production databases without

compromising the integrity of Data Guard standby databases, pick your level of synchronization between primary & standby databases.

54

18c Oracle

Database

Page 55: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Data GuardSubset Standbys

• In 12.1, ENABLED_PDBS_ON_STANDBY initialization parameter only supported two values: all PDBs or none

• The ENABLED_PDBS_ON_STANDBY parameter is only applicable on the physical standby database – can accept a list of PDB names or a glob pattern such as “VNAPDB?” or “VNAPDB*a”,

“VNAPDB2”– Glob pattern rules are similar to regular expression rules in common UNIX shells – asterisk (*) and question mark (?) wildcard characters are supported.

• The question mark (?) represents a single unknown character; – the asterisk (*) represents matches to any number of unknown characters. – This parameter also accepts a minus sign (-) which can be used as the first character in a

PDB name to designate that the PDB should be excluded on the standby database

Page 56: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

Data GuardIn-Memory with Active Data Guard

In-Memory with Active Data Guard • IM option can be configured on the primary database, on

an ADG standby database, or on both the primary and the ADG standby databases

• The INMEMORY_ADG_ENABLED parameter needs to be enabled – By default, the INMEMORY_ADG_ENABLED parameter is

set to true. • For RAC configuration, this parameter must be set to the

same value across all the RAC instances

Page 57: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

@ViscosityNA

PDB Switchover Clause

• Reverses the roles between a refreshable clone PDB and a primary PDB. • The former Refreshable clone PDB becomes the primary PDB, which can

now be opened in read write mode. • The formerly primary PDB now is the refreshable clone and can only be

opened in READ ONLY mode. • This command must be executed from the primary PDB. • The dblink must point to the root CDB where the refreshable clone PDB

resides.

alter pluggable database refresh mode auto every 2 minutes from new_pdb@dblink switchover;

June 11, 2018 57

18c Oracle

Database

Page 58: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

58

My Top 12 Common Data Guard

Failures

That maycause

Data Guard failures & outages

Page 59: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Common Data Guard Issues Day-To-Day Problems That DBAs Face

Common Data Guard FailuresRemoval of archivelogs by accident before applied on the physical standby

For RAC, databases, add datafile to ASM without the + sign for +DATA diskgroup

Primary DB file system or ASM Disk Group space exhaustion

db_recovery_file_dest_size• Self inflicted outage

standby_file_management: Default: ManualDealing with UNNAMED datafiles

No PSUs applied

Primary Database Archive Log Destination 100% fullMore Common: Standby Archive Log Destination 100% full

Forget to re-start MRP after reboot or outage

SYS Password Change Broker changesAlter System changes

Other things to consider: Take GRPs before maintenance tasks

Other things to monitor:NOLOGGING transactions on the primary database

Page 60: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Bulletproof DG Configuration Where Does It Start

60

Monitoring • Monitoring Alert Logs on both Primary

and Standby Databases • Monitoring Archive Gaps or Deviances • Monitoring Transport Lag • Monitoring Apply Lag

Proactive • File System Monitoring • ASM Disk Group Monitoring • DB_RECOVERY_FILE_DEST + DB_RECOVERY_FILE_DEST_SIZE

• Latest PSU Patches (N – 1 Consideration) • OEM Full Integration and Lights-out Monitoring • Quarterly Switchover Testing

OEM 13c Cloud Control to monitor and manage a Data Guard environment:

» Data Guard status » Transport Lag » Apply Lag » Estimated Database Role Transition Time » Primary or standby is or is not accessible

Send Alerts!

Page 61: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Best Practice for Monitoring DGOEM 13c Cloud Control Monitoring and Maintenance

61

OEM 13c Cloud Control to monitor and manage a Data Guard environment:

» Data Guard status » Transport Lag » Apply Lag » Estimated Database Role Transition Time » Primary or standby is or is not accessible

Send Alerts!

Page 62: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

sysdg privileges (12.1)SQL> create user dgoper identified by oracle123 default tablespace users temporary tablespace temp; User created.

SQL> grant sysdg to dgoper; Grant succeeded.

62

JIN01_DR > dgmgrl dgoper/oracle123@JIN01 DGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production

Copyright (c) 2000, 2013, Oracle. All rights reserved.

Welcome to DGMGRL, type "help" for information. Connected as SYSDG.

• STARTUP• SHUTDOWN• ALTER DATABASE• ALTER SESSION• ALTER SYSTEM• CREATE RESTORE POINT (including GUARANTEED restore points)• CREATE SESSION• DROP RESTORE POINT (including GUARANTEED restore points)• FLASHBACK DATABASE

SELECT ANY DICTIONARYSELECT

• X$ tables (that is, the fixed tables)• V$ and GV$ views (that is, the dynamic performance views)• APPQOSSYS.WLM_CLASSIFIER_PLAN

DELETE• APPQOSSYS.WLM_CLASSIFIER_PLAN

EXECUTE• SYS.DBMS_DRS

SYSDG privilege enables you to connect to the database even if it is not open.

Page 63: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Best Practice Recommendations And General Guidelines

63

• Download DG Toolkit• DG Toolkit has all the best practices in-mind

Page 64: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Network Layer

SDU + BDP• Set SDU to 32k (default 8k) for SYNC

• No performance gain for ASYNC• Max Size = 64k (starting from

11.2.0.2)

• Calculate BDP- Set SEND_BUF_SIZE and

RECV_BUF_SIZE Oracle Net Parameters

- Set net.ipv4.tcp_wmem and net.ipv4.tcp_rmem kernel parameters

- Script: dg_bdp.ksh from DG Toolkit❖ dg_bdp.conf

Option #1: Set default SDU to be 32k in the SQLNET.ORADEFAULT_SDU_SIZE=32767

• Impacts all databases that share the Oracle Home

Option #2:Or in listener.ora and tnsnames.ora together

Page 65: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Bandwidth-Delay Product = Bandwidth * Latency * 3

cat dg_bdp.confWAN=OC3RTT_RESPONSE=44

$ ./dg_bdp.kshBANDWIDTH is: 155000000LATENCY is: .04400BDP = 2557500

TNSNAMES.ORA ENTRY

VISK_PRI = (DESCRIPTION = (SEND_BUF_SIZE=2557500) (RECV_BUF_SIZE=2557500) (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP) (HOST = rac561-vip) (PORT = 1521)) ) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = VISK) ) )

LISTENER.ORA ENTRY

LISTENER = (DESCRIPTION_LIST = (DESCRIPTION=(ADDRESS_LIST= (ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER))) (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP) (HOST = rac561-vip) (PORT = 1521) (SEND_BUF_SIZE=2557500) (RECV_BUF_SIZE=2557500))) ) SID_LIST_LISTENER = (SID_LIST = (SID_DESC = (ORACLE_HOME = /app/oracle/product/11.2.0/db) (SID_NAME = VISK1) ) )

Page 66: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

FRA Best Practices

FRA Best PracticesLOG_ARCHIVE_DEST_1 ’LOCATION=USE_DB_RECOVERY_FILE_DEST’

Always enable Flashback Database on primary and standby database

MAA Best Practice: DB_FLASHBACK_RETENTION_TARGET

60 for just flashback re-instantiation

MAA Best Practices: DB_FLASHBACK_RETENTION_TARGET

Minimum 6 hours for protection from user errors, logical corruptions, and malicious users

Can also be used for Snapshot Standby

Page 67: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Block Change Tracking

Block Change Tracking

Enable Block Change Tracking alter database enable block change tracking using file '+DATA';

You can enable BCT on the standby database now

Does incur a license of Oracle Active Data Guard

View the size of the BCT File select filename, status, bytes from v$block_change_tracking;

Page 68: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

“Disaster strikes without warning. > Data loss strikes without warning.”

Page 69: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

SRL

Do not multiplex SRLBest practices equation for the number for SRL is:• # of SRL = (# of ORL + 1 ) X Threads

Recommend SRL for even Maximum PerformanceExecute: ./dg_generate_standby_redo.ksh Generates:

- Execute SQL Script: cr_standby_redo_p.sql on VISK- Execute SQL Script: cr_standby_redo_s.sql on VISK_DR

Page 70: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

At Database Level – Best PracticeSQL> alter database force logging;

Some opt to enforce logging at tablespace level:- SQL> alter tablespace TS_NAME force logging;-MV tablespace- Temp tablespace for reports and stage data

Need to enable force logging monitoringExecute dg_check_force_logging.sql to check Force LoggingExecute dg_check_unrecoverable.sql to check Unrecoverable Activity

Logging

Page 71: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Date 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 ------- ---- ---- ---- ---- ---- ---- ---- --- --- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----

Feb 28 1 0 0 1 2 1 4 9 5 6 7 9 8 8 0 0 0 0 0 0 0 0 0 0

Feb 27 2 1 1 2 2 1 1 5 1 1 2 0 1 0 0 0 5 0 0 0 3 5 0 0

Feb 26 12 5 12 2 5 5 1 8 4 4 6 4 2 2 1 1 5 3 2 1 3 3 1 6

Feb 25 8 3 4 4 5 3 7 8 5 7 8 9 3 8 11 9 12 12 8 6 6 10 10 12

Feb 24 12 10 13 7 3 3 7 9 8 8 8 3 5 6 3 8 4 9 14 12 9 8 10 12

Feb 23 16 25 23 20 15 11 5 11 6 8 2 7 7 9 3 8 9 11 13 13 7 8 8 14

Feb 22 4 3 3 1 3 3 4 9 4 4 7 9 7 3 8 9 8 5 10 9 4 4 16 16

Feb 21 0 0 1 2 1 2 3 8 4 3 5 4 7 5 4 7 10 7 5 6 5 10 6 5

Feb 20 6 20 14 0 0 1 5 8 4 1 2 0 2 6 0 0 5 0 1 1 2 6 0 0

Feb 19 8 2 3 1 6 3 3 8 3 5 6 4 4 1 2 1 5 0 0 0 2 5 7 12

Feb 18 6 4 3 2 4 3 6 8 7 6 9 11 8 10 7 10 12 9 8 6 2 8 11 7

Know Your Archive Generation Rate

Page 72: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

SELECT TO_CHAR(TRUNC(FIRST_TIME),'Mon DD') "Date", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'00',1,0)),'9999') "00", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'01',1,0)),'9999') "01", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'02',1,0)),'9999') "02",

TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'03',1,0)),'9999') "03", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'04',1,0)),'9999') "04", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'05',1,0)),'9999') "05", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'06',1,0)),'9999') "06", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'07',1,0)),'9999') "07", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'08',1,0)),'9999') "08", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'09',1,0)),'9999') "09", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'10',1,0)),'9999') "10", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'11',1,0)),'9999') "11",

TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'12',1,0)),'9999') "12", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'13',1,0)),'9999') "13", TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'14',1,0)),'9999') "14", ... TO_CHAR(SUM(DECODE(TO_CHAR(FIRST_TIME,'HH24'),'23',1,0)),'9999') "23" FROM V$LOG_HISTORY GROUP BY TRUNC(FIRST_TIME) ORDER BY TRUNC(FIRST_TIME) DESC;

Know Your Archive Generation Rate

Page 73: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

RMAN Image Copy Backups * Duplicate Database for Standby Database

Page 74: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Daily Incremental Backups1. recover copy of database 2. backup incremental level 1 for

recover of copy3. Backup archivelogs

DB Server

DATA: Datafiles, Redo Log, Control File, BCT

RMAN Image Copy Backup StrategyDatabase Architecture

RECO: Redo Log, Control File, Archive Log, Flashback Logs, spfile, Datapump dumps,

Backups

ORAPRD

Updated copy of the database that is in sync

with PROD DB

Similar to EMC’s Sync and Split

DOCPRD

DWPRD ODSPRD

1st Sat/Sun of the month - RMAN Image Copy (Once Per month or

Quarter or Once Ever)

* backup as copy incremental level 0

/dNFS/bkup: L0 /dNFS/bkups: L1 /dNFS/bkups – Updated L0

/dNFS/10GigE Network

Replication

Storage that support snapshots

Backup files to Direct NFS1. Database Image Copy2. Incremental backups3. Archive Log backups4. Nightly No Rows Full database

exports5. Autobackup of controlfiles6. SPFILE backup

Daily Snapshots

Page 75: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

7584

RMAN To Disk Backups

rman2disk.ksh from http://dbaexpert.com/rman2disk/

Cron Job Examples: ## RMAN backups ## 00 23 * * 0 /app/oracle/general/sh/rman2disk.ksh -d DATABASE -l 0 -n 2 -c catalog > /tmp/rman_full_DATABASE.log 2>&1 00 23 * * 1-6 /app/oracle/general/sh/rman2disk.ksh -d DATABASE -l 1 -n 2 -c catalog > /tmp/rman_incr_DATABASE.log 2>&1 01 12,18 * * 1-6 /app/oracle/general/sh/rman2disk.ksh -d DATABASE -a y > /tmp/rman_arch_DATABASE.log 2>&1

Usage Information for rman2disk.ksh:Regular Usage with RMAN Backupsets: 1. Perform a full level 0 backup: rman2disk.ksh -d DATABASE -l 0 -c catalog > /tmp/rman2disk_DATABASE.L0.log 2>&12. Perform an incremental backup: rman2disk.ksh -d DATABASE -l 1 -c catalog > /tmp/rman2disk_DATABASE.L1.log 2>&1

Compressed Backups with RMAN Backupsets1. Perform a full level 0 compressed backup: rman2disk.ksh -d DATABASE -l 0 -c catalog -z y > /tmp/rman2disk_DATABASE.L0Z.log 2>&12. Perform a compressed incremental backup: /rman2disk.ksh -d DATABASE -l 1 -c catalog -z y > /tmp/rman2disk_DATABASE.L1Z.log 2>&1

Advanced Usage with Database Copies: 1. Perform a baseline level 0 backup: rman2disk.ksh -d DATABASE -l baseline > /tmp/rman2disk_DATABASE.baseline.log 2>&12. Perform an incremental backup: rman2disk.ksh -d DATABASE -l 1 -r merge > /tmp/rman2disk_DATABASE.L1.log 2>&1

Archivelog only backups: rman2disk.ksh -d DATABASE-a y > /tmp/rman_arch_emrepos.log 2>&1

Notes: 1. -r option can only be used with Copy Backups2. -r must find a baseline tag in the $SH directory; otherwise, the script will perform a full level 0 copy

Page 76: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

7685

RMAN To Disk Backups #2

rman2disk.ksh from http://dbaexpert.com/rman2disk/

Pertinent Information for each database being backed up:

1. Symbolic link for $ORACLE_BASE/admin/$ORACLE_SID/bkups must be created to a backup file system directory

2. $ORACLE_BASE/admin/$ORACLE_SID/log directory must be created3. For RMAN catalog backups, the .rman.pw file must exist in the $SH directory

RMAN To Disk Backup scripts:-rwxr-xr-x. 1 oracle oinstall 1018 Nov 6 10:49 rman2disk.arch.sql -rwxr-xr-x. 1 oracle oinstall 9978 Nov 26 09:12 rman2disk.ksh -rwxr-xr-x. 1 oracle oinstall 1963 Nov 26 09:43 rman2disk.sql -rwxr-xr-x. 1 oracle oinstall 1232 Jan 21 2015 rman2disk.sql.baseline -rwxr-xr-x. 1 oracle oinstall 1275 Jan 21 2015 rman2disk.sql.cold -rwxr-xr-x. 1 oracle oinstall 963 Jan 21 2015 rman2disk.sql.cold.noarchivelog -rwxr-xr-x. 1 oracle oinstall 1544 Nov 26 09:42 rman2disk.sql.compression -rwxr-xr-x. 1 oracle oinstall 1370 Jan 21 2015 rman2disk.sql.for_recover -rwxr-xr-x. 1 oracle oinstall 1506 Jan 21 2015 rman2disk.sql.nocompression -rwxr-xr-x. 1 oracle oinstall 227 Jan 21 2015 rman2disk.tag.history

Page 77: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Build Physical Standby Menu

Page 78: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Duplicate Database (From Active Database)

Execute ./dg_duplicate_database.ksh manually or from DG Toolkit

./dg_duplicate_database.ksh |tee dg_duplicate_database_out.ksh chmod 755 dg_duplicate_database_out.ksh ./dg_duplicate_database_out.ksh

echo ""; echo "# ------------------------------------------------------------- #” echo "# -- Scripts are generated in the /tmp/dba directory #"; echo "# ------------------------------------------------------------- #” ls -l /tmp/dba echo "";

Here’s what it generated: # ------------------------------------------------------------- # # -- Scripts are generated in the /tmp/dba directory # # ------------------------------------------------------------- # total 12 -rwxr-xr-x. 1 oracle oinstall 1594 Apr 12 03:41 01_dg_oratab_listener_ps.ksh -rwxr-x---. 1 oracle oinstall 881 Apr 12 03:41 02_dg_tnsnames_all.ksh -rw-r--r--. 1 oracle oinstall 3334 Apr 12 03:41 10_dg_duplicate_database_ps.ksh

Page 79: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Duplicate Database From Active Database #1

. /home/oracle/dg/dg.conf # --------------------------------------------------------------------------------------- # -- scp password file to the standby database server: # ika82 from primary server: vna85 # --------------------------------------------------------------------------------------- scp vna85:/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwJIN01 /u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwJIN01_DR

export TMPDIR=/tmp/dba [ ! -d "$TMPDIR" ] && mkdir -p /tmp/dba

# --------------------------------------------------------------------------------------- # -- Create the following initialization file for the JIN01_DR instance: # -- /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initJIN01_DR.ora # --------------------------------------------------------------------------------------- cat <<EOF > /u01/app/oracle/product/12.1.0/dbhome_1/dbs/initJIN01_DR.ora db_name=JIN01 db_unique_name=JIN01_DR cluster_database=false EOF

# --------------------------------------------------------------------------------------- # -- Execute the following RMAN script on the standby database server # -- # -- First startup nomount the database with either SQL*PLUS or RMAN> # --------------------------------------------------------------------------------------- echo "startup nomount; alter system register;" |sqlplus -s / as sysdba

Page 80: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

rman <<EOF connect target sys/oracle123@JIN01; connect auxiliary sys/oracle123@JIN01_DR; run { allocate channel prmy1 type disk; allocate channel prmy2 type disk; allocate channel prmy3 type disk; allocate channel prmy4 type disk; allocate auxiliary channel stby type disk; duplicate target database for standby from active database dorecover spfile parameter_value_convert 'JIN01','JIN01_DR' set 'db_unique_name'='JIN01_DR' set control_files='+FRA' set log_archive_max_processes='5' set fal_client='JIN01_DR' set fal_server='JIN01' set standby_file_management='AUTO' set db_create_file_dest='+DATA' set db_create_online_log_dest_1='+DATA' set log_archive_config='dg_config=(JIN01,JIN01_DR)' set log_archive_dest_1='service=JIN01 LGWR ASYNC valid_for=(ONLINE_LOGFILES,PRIMARY_ROLE) db_unique_name=JIN01' set cluster_database='FALSE' set parallel_execution_message_size='8192' set db_lost_write_protect='TYPICAL' set db_block_checking='TRUE' set db_block_checksum='FULL' nofilenamecheck ;

Duplicate Database From Active Database #2

Page 81: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

sql channel prmy1 "alter system set log_archive_config=''dg_config=(JIN01,JIN01_DR)''"; sql channel prmy1 "alter system set log_archive_dest_2= ''service=JIN01_DR LGWR ASYNC valid_for=(online_logfiles,primary_role) db_unique_name=JIN01_DR''"; sql channel prmy1 "alter system set log_archive_max_processes=5"; sql channel prmy1 "alter system set fal_client=JIN01"; sql channel prmy1 "alter system set fal_server=JIN01_DR"; sql channel prmy1 "alter system set standby_file_management=auto"; sql channel prmy1 "alter system set log_archive_dest_state_2=enable"; sql channel prmy1 "alter system set parallel_execution_message_size=8192 scope=spfile sid=''*''"; sql channel prmy1 "alter system archive log current";

sql channel stby "alter database recover managed standby database using current logfile disconnect"; } EOF

Duplicate Database From Active Database #3

Page 82: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Monitor Data Guard With Scripts

Page 83: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Monitoring

• Alert Log Monitoring• Monitoring MRP• Monitoring Archive Lag Gaps• Monitoring Transport and Apply Lag• Monitoring Apply Rate• How Far Are We Really Behind

Page 84: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

84

Database Alert Log Monitoring

Oracle Database Alert Log Monitoring

2. wrapper_alertlog.ksh

1. prep_alertlog.ksh

oraping.ksh

blockmon.ksh

long_running_sql.ksh

alertlog.pl1. Leverages the Tiny.pm perl module

• This is located in the $SH/lib/Config directory (please note that Config is a Capital C)

• Must copy the contents of the lib subdirectory to every node

2. Leverages the alertlog.conf filecat alertlog.confORACLE_BASE=/app/oracle SH=/app/oracle/general/sh SQL=/app/oracle/general/sql

Note: • alertlog.conf file is leveraged by Tiny.pm perl module and

cannot have any variables• All parameters must be hard-coded

3. Add ORA- error messages that you want the alertlog.pl alerting to ignore to the $SH/ignore.dat file

vmstat.ksh

alertlog.pl

prep_alertlog.ksh:

From the $SH directory, invoke the prep_alertlog.ksh script for each database1. Change ORACLE_SID to database you want to setup alert log monitoring2. Execute prep_alertlog.ksh (checks v$diag_info to determine trace directory)3. Create the OFA directory structure: $ORACLE_BASE/admin/$ORACLE_SID/bdump4. Creates symbolic link for alert_$ORACLE_SID.log

export TRACE_DIR=$( sqlplus -s / as sysdba <<EOF set head off ver off lines 2000 trims on pages 0 feed off select value from v\$diag_info where name='Diag Trace' ; exit; EOF )

export ALERT_LOG=$TRACE_DIR/alert_$ORACLE_SID.log echo "Trace Dir: $TRACE_DIR" echo "-> Alert Log: $ALERT_LOG"

export BDUMP=$ORACLE_BASE/admin/$ORACLE_SID/bdump echo "-->BDUMP is: $BDUMP" if [ ! -d $BDUMP ]; then echo "$BDUMP directory does not exist! Creating directory: $BDUMP" mkdir $BDUMP fi

cd $BDUMP

if [ -h "alert_$ORACLE_SID.log" ]; then echo "--->Alert Log for Oracle SID Exists... removing symbolic link for $alert_$ORACLE_SID.log" rm alert_$ORACLE_SID.log fi ln -s $ALERT_LOG alert_$ORACLE_SID.log

[ ! -d "$SH/log" ] && ( echo "----> $SH/log directory does not exist.. making dir $SH/log"; mkdir -p $SH/log ) [ ! -f "$SH/ignore.dat" ] && ( echo "-----> $SH/ignore.dat file does not exist... touching $SH/ignore.dat"; touch $SH/ignore.dat ) [ ! -f "$SH/errLogAlert.dat" ] && ( echo "------> $SH/errLogAlert.dat file does not exist... touching $SH/errLogAlert.dat"; touch $SH/errLogAlert.dat )

wrapper_alertlog.ksh#!/bin/ksh

---------------------------------------------------------- . $HOME/.ORACLE_BASE . $ORACLE_BASE/general/sh/functions.ksh export PATH=/usr/local/bin:$PATH cd $SH

./alertlog.pl emrepos

Page 85: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Monitoring

• Alert Log Monitoring - Looks at entries in /etc/oratab - Can customize ignore list - Performs diff and compares since last

execution - Sends alerts based on Alert Notifications

- Contact me for a copy

Page 86: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

LogRotatefunction rotate {export LOGFILE=$1export CONFIG_FILE=$2

export PATH=$PATH:/usr/sbinexport CONF_DIR=$SH/logrotate[ ! -d "$CONF_DIR" ] && ( echo $CONF_DIR does not exist .. issuing mkdir; mkdir -p $CONF_DIR )

export CONF=$CONF_DIR/$CONFIG_FILE

cat <<!! >$CONF$LOGFILE {rotate 4dateextcopytruncatecompresssize 100M}!!

echo Executing: logrotate -s $CONF_DIR/log_rotate_status.$CONFIG_FILE -f $CONFlogrotate -s $CONF_DIR/log_rotate_status.$CONFIG_FILE -f $CONF}

Page 87: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Monitoring

• Monitoring MRP

dg_check_mrp.sql set time on set lines 132 set pagesize 9999 col client_pid format a12 SELECT PID, PROCESS, STATUS, CLIENT_PROCESS, CLIENT_PID, THREAD#, SEQUENCE#, BLOCK#, BLOCKS, DELAY_MINS FROM V$MANAGED_STANDBY;

Page 88: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Monitoring

• Monitoring Transport and Apply Lag

dg_lag.sql col name for a13 col value for a20 col unit for a30 set lines 122 SELECT NAME, VALUE, UNIT, TIME_COMPUTED FROM V$DATAGUARD_STATS WHERE NAME IN ('transport lag', 'apply lag');

• Need to be on the latest • PSU

Or • RU / RUR

Page 89: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Monitoring

• Monitoring Apply Rate

dg_apply_rate.sql col type for a15 set lines 122 set pages 33 col item for a20 col units for a15 select to_char(start_time, 'DD-MON-RR HH24:MI:SS') start_time, item, sofar, units from v$recovery_progress where (item='Active Apply Rate' or item='Average Apply Rate' or item='Redo Applied')

Page 90: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

EBS

Page 91: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

www.viscosityna.com

Poor Man’s EBS DR

RSYNCrsync -avz -e ssh --stats --progress /appsvr/CM applmgr@dr_app_server:/apps702 > $LOGFILE 2>&1

function rsync_appstmp { APPLCSF=/apps/tmp/PROD/APPLCSF rsync -avz -e ssh --del --stats --progress ${APPLCSF}/log applmgr@dr_app_server:/apps702 >

$LOGFILE 2>&1 rsync -avz -e ssh --del --stats --progress ${APPLCSF}/out applmgr@dr_app_server:/apps702 >

$LOGFILE 2>&1 }

Data Guard (Switchover)(P) ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY WITH SESSION SHUTDOWN;

(S)ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY WITH SESSION SHUTDOWN; (S) ALTER DATABASE OPEN;

(P)ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;

91

Page 92: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

www.viscosityna.com

Poor Man’s EBS DR

Data Guard (Primary)SQL> SHUTDOWN IMMEDIATE;SQL> STARTUP MOUNT;SQL> ALTER SYSTEM FLUSH REDO TO

'<target_db_name>';Repeat for Each Thread

Data Guard (Standby - Resolve Gaps if Possible)SQL> ALTER DATABASE REGISTER PHYSICAL

LOGFILE 'filespec1';

92

Complete Loss Primary Database (Start Here)• SQL> ALTER DATABASE RECOVER MANAGED STANDBY

DATABASE CANCEL;Database altered.

Finish applying all received redo data.• SQL> ALTER DATABASE RECOVER MANAGED STANDBY

DATABASE FINISH;Database altered.

Verify that the target standby database is ready to become a Primary• SQL> SELECT SWITCHOVER_STATUS FROM V$DATABASE;• SWITCHOVER_STATUS

----------------- TO PRIMARY

Switch the physical standby database to the primary role.SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY WITH SESSION SHUTDOWN;SQL> ALTER DATABASE OPEN;

Page 93: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Broker

Page 94: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Broker Configuration SPFILE Setup

# -- Execute the following on the Primary Database: VISKalter system set dg_broker_config_file1='+DATA/VISK/broker1.dat' scope=both sid='*';alter system set dg_broker_config_file2='+FRA/VISK/broker2.dat' scope=both sid='*';alter system set dg_broker_start=true scope=both sid='*';

# -- Execute the following on the Standby Database: VISK_DRalter system set dg_broker_config_file1='+DATA2/VISK_DR/broker1.dat' scope=both sid='*';alter system set dg_broker_config_file2='+FRA/VISK_DR/broker2.dat' scope=both sid='*';

alter system set dg_broker_start=true scope=both sid='*';

Page 95: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Broker Configuration Listener Setup

Broker Configuration – Listener Setup 2 # -- Primary Database Server # -- LISTENER_VISK = . .. SID_LIST_LISTENER_VISK = (SID_LIST = (SID_DESC = (SDU=32767) (GLOBAL_DBNAME = VISK_DGMGRL.DOMAIN) (ORACLE_HOME = /u01/app/oracle/product/11.2.0/db) (SID_NAME = VISK1)

Page 96: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Broker Configuration Create Configuration

create configuration VISK_DGCONFIG as primary database is "VISK" connect identifier is VISK_PRI;add database "VISK_DR" as connect identifier is VISK_STDBY maintained as physical;enable configuration;

Broker Configuration – Create FSFOedit database 'VISK' set property 'LogXptMode'='SYNC';edit database 'VISK_DR' set property 'LogXptMode'='SYNC';edit database 'VISK' set property FastStartFailoverTarget='VISK_DR';edit database 'VISK_DR' set property FastStartFailoverTarget='VISK';edit configuration set protection mode as maxavailability;enable fast_start failover;show fast_start failover;

Page 97: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Broker Configuration Create Configuration

DGMGRL> create configuration JIN01_DGCONFIG as primary database is "JIN01" connect identifier is JIN01; Configuration "jin01_dgconfig" created with primary database "JIN01" DGMGRL> add database "JIN01_DR" as connect identifier is JIN01_DR maintained as physical; Database "JIN01_DR" added DGMGRL> enable configuration; Enabled.

DGMGRL> show configuration Configuration - jin01_dgconfig

Protection Mode: MaxPerformance Members: JIN01 - Primary database Warning: ORA-16789: standby redo logs configured incorrectly

JIN01_DR - Physical standby database Warning: ORA-16809: multiple warnings detected for the database

Fast-Start Failover: DISABLED

Configuration Status: WARNING (status updated 45 seconds ago)

Page 98: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Observer

Start Observer

# -- # -- You can use the following script to start the observer #!/usr/bin/ksh dgmgrl <<___EOF >/tmp/observer_`hostname`.log connect sys/oracle123@VISK_PRI start observer ___EOF

Page 99: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Archive Log Retention

Archive Log Retention

Primary Database Standby Database

configure archivelog deletion policy to shipped to all standby;

------------------------ OR -----------------------------

configure archivelog deletion policy to applied on all standby;

configure archivelog deletion policy to applied on standby;

Page 100: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

DG Toolkit

Page 101: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...
Page 102: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

102

Page 103: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

103

Page 104: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

At the heart of it is the dg.conf configuration file

Page 105: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

DG Toolkit Simplified Simplified Configuration File for non-RAC (12 Parameters To Instantiate Data Guard)

# --------------------------------------------------------------------------- # PRIMARY_HOST=vnadb01 PRIMARY_DB=VNA PRIMARY_PORT=1521 # --

STANDBY_HOST=vnadr01 STANDBY_DB=VNADR STANDBY_PORT=1523 # --------------------------------------------------------------------------- # # - File System information (for duplicate database, rman restore, dbca) # --------------------------------------------------------------------------- # PRIMARY_FS=/oradata01 PRIMARY_ARCH=/oraarch STANDBY_FS=/oradata01 STANDBY_ARCH=/oraarch # --------------------------------------------------------------------------- # PRIMARY_ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1 STANDBY_ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1

105

Page 106: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Useful DG Toolkit Scripts

Page 107: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Misc DG Toolkit Simple Built-Ins

On the Standby Database:• dg_start.sql Start MRP (with SRL) disconnect alter database recover managed standby database using current logfile disconnect;

• dg_stop.sql Stop MRP alter database recover managed standby database cancel;

On the Primary Database• dg_client_start.sql Enabled archive dest #2 alter system set log_archive_dest_state_2='enable' sid='*’;

• dg_client_stop.sql Disable archive dest #2 alter system set log_archive_dest_state_2='defer' sid='*';

Page 108: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

SQL Script Description

dg_redo.sql Check Redo Log information

dg_sredo.sql Check Standby Redo Log information

dg_lag.sql Check transport and apply lag for the Data Guard environment

dg_check_mrp.sql Check MRP process and status

dg_client_start.sql Enabled archive dest #2

dg_client_stop.sql Disable archive dest #2

dg_register_archivelog_from_primary.sql Register archivelogs from primary

rman2disk.sql RMAN template to backup the database to FRA disk group

dg_database_info.sql PL/SQL to view DG related database information

dg_prim_diag.sql Troubleshooting script from Metalink ID: 241374.1

Page 109: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Shell Script Description

./dg Launches the DG Toolkit

dg_bdp.ksh Calculate Bandwidth Delay Product – leverages dg_bdp.conf

dg_adr_menu.ksh Submenu to setup Automatic Diagnostic Repository

dg_physical_standby_menu.ksh Submenu for checking/monitoring on the physical standby database

dg_prepare_standby_menu.ksh Primary Submenu for creating the physical standby database

dg_preliminary_check_menu.ksh Submenu for validating the environment prior to creating the physical standby database

dg_rman2disk_menu.ksh Submenu for launching RMAN backups and restore activities

rman2disk.ksh korn shell script to backup the database to the FRA or file system disk group

dg_broker_menu.ksh Submenu for Broker tasks

dg_logical_menu.ksh For old timers who are still using Logical Standby – Submenu for Logical Standby database maintenance and monitoring

Page 110: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

oracle.com/gbtour

New Free Tier Always Free

Oracle Cloud Infrastructure

Services you can use for unlimited time

30-Day Free TrialFree credits you can use for more services

+

Page 111: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...

Facebook.com/ViscosityNA

Linkedin.com/company/Viscosity-North-America

@ViscosityNA

Viscosity North America

Facebook.com/ViscosityNA

@Viscosity_NA

Follow Us Online!

111

Page 112: Bulletproof Your Data Guard Configuration 12.2 / 18c /19c ...