Oracle Technology Night #36 · 5/19/2020 · SAP Sybase ASE IBM Informix SAP Sybase ASE Netezza TERADATA IBM Db2 HPE NonStop (Guardian) Oracle Database MySQL Database Server Microsoft
Post on 22-Aug-2020
10 Views
Preview:
Transcript
May, 2020
GoldenGateOracle Technology Night #36
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
Safe harbor statement
Copyright © 2020, Oracle and/or its affiliates 2
Copyright © 2020, Oracle and/or its affiliates 3
•• Certification�Matrix••
• /•• DR
GoldenGate
••
DBA GoldenGate
Data Guard GoldenGate (Object)DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates4
• DB• REDO
( 1)�
Data�GuardActiveData�Guard
Oracle�GoldenGate
OS,Version DB( 1)
( 2)
( 2)� Data�Guard� Data�Guard� (Doc�ID�2201589.1)
••
GoldenGate -DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates5
Oracle Stream Analytics
KafkaETL&ML
Spark
DBMS
Cloud
Big Data
NoSQL
Streams
MySQL Database Server
Oracle Database
IBM Db2
Microsoft SQL Server
Cassandra
Java Message Service
HPE NonStop(Guardian)
SAP Sybase ASE
IBM Informix
SAP Sybase ASE
Netezza
TERADATA
IBM Db2
HPE NonStop(Guardian)
OracleDatabase
MySQLDatabaseServer
MicrosoftSQL Server
IBM Informix Greenplum
SAP HANA
ElasticsearchMongoDB
JDBC
JDBC
Cloudera Hortonworks
Cassandra Oracle NoSQLJava MessageService
Oracle CloudObject Storage
MapR
Google BigQuery
AmazonS3
AmazonQurora
AmazonKinesis
AmazonRedShift
OracleDatabaseCloud
OracleAutonomousDatabase
Confluent
OracleAutonomousDatabase
Kafka
Kafka
Kafka
HBase
HDFS
Snowflake MicrosoftAzure Ecosystem
HIVE
Flink
Copyright © 2020, Oracle and/or its affiliates 6
GoldenGate -DBA GoldenGate
AWS
Azure
Private�Cloud
Oracle Cloud Infrastructure
Copyright © 2020, Oracle and/or its affiliates 7
GoldenGate -DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates 8
GoldenGate -DBA GoldenGate
GoldenGate
Copyright © 2020, Oracle and/or its affiliates 9
GoldenGate -DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates 10
GoldenGateDBA GoldenGate
( )
Trail File Trail File
SourceTarget
TargetSource
Copyright © 2020, Oracle and/or its affiliates 11
GoldenGate for�Big�Data� ?DBA GoldenGate
GoldenGate for�Big�Data
GG for Big Data
Handler
Flat File
Streams
No SQL
Big Data
DBMS
Cloud
Copyright © 2020, Oracle and/or its affiliates12
•• Certification�Matrix••
• /•• DR
GoldenGate
••
DBA GoldenGate
GoldenGate -DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates13
Oracle Stream Analytics
KafkaETL&ML
Spark
DBMS
Cloud
Big Data
NoSQL
Streams
MySQL Database Server
Oracle Database
IBM Db2
Microsoft SQL Server
Cassandra
Java Message Service
HPE NonStop(Guardian)
SAP Sybase ASE
IBM Informix
SAP Sybase ASE
Netezza
TERADATA
IBM Db2
HPE NonStop(Guardian)
OracleDatabase
MySQLDatabaseServer
MicrosoftSQL Server
IBM Informix Greenplum
SAP HANA
ElasticsearchMongoDB
JDBC
JDBC
Cloudera Hortonworks
Cassandra Oracle NoSQLJava MessageService
Oracle CloudObject Storage
MapR
Google BigQuery
AmazonS3
AmazonQurora
AmazonKinesis
AmazonRedShift
OracleDatabaseCloud
OracleAutonomousDatabase
Confluent
OracleAutonomousDatabase
Kafka
Kafka
Kafka
HBase
HDFS
Snowflake MicrosoftAzure Ecosystem
HIVE
FlinkSource Target
Copyright © 2020, Oracle and/or its affiliates 14
- Certification�MatrixDBA GoldenGate
Source / Target (Certification Matrix)
https://www.oracle.com/middleware/technologies/fusion-certification.html
(Source/Target ) DB GoldenGate Release
GoldenGate 19.1�
Oracle�Database�R11.2.0.4+Oracle�Database�19c
ADW/ATP( )
Copyright © 2020, Oracle and/or its affiliates 15
- Certification�MatrixDBA GoldenGate
Oracle 10g ?
GoldenGate R11.2�Oracle�Database�R10.2.0.4+�
https://www.oracle.com/technetwork/middleware/data-integration/goldengate1121-cert-matrix-1522269.xls
Copyright © 2020, Oracle and/or its affiliates 16
-DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates 17
( )DBA GoldenGate
Extract�( )https://docs.oracle.com/cd/F22974_01/index.html
Copyright © 2020, Oracle and/or its affiliates 18
DBA GoldenGate
Check Script�------ System�Info:
DateTime:-------------------03-09-2020�17:10:18
BANNER--------------------------------------------------------------------------------Oracle�Database�11g�Enterprise�Edition�Release�11.2.0.4.0�- 64bit�ProductionPL/SQL�Release�11.2.0.4.0�- ProductionCORE 11.2.0.4.0 ProductionTNS�for�Linux:�Version�11.2.0.4.0�- ProductionNLSRTL�Version�11.2.0.4.0�- Production
NAME������LogMode������SupLog:��PK��UI��For�FK��All�Created--------- ------------ -------- --- --- --- --- --- -------------------SL11������ARCHIVELOG���NO�������NO��NO��YES�NO��NO��02-23-2017�14:51:00
------ Tables�with�Deferred�constraints.Deferred�constraints�may�cause�TRANDATA�to�chose�an�incorrect�Key
------ Any�table�with�a�deferred�constraint�will�need�more�research�to�avoid�any�issues
------ Tables�with�Deferred�PK�constraints�should�be�added�using�KEYCOLS�in�the�trandata�statement.
------ Tables�Defined�with�Rowsize�>�2M�in�all�Schemas
------ Tables�With�No�Primary�Key�or�Unique�Index�and�Column�lenght�>�1M
------ Integrated�Replicat�does�not�support�Transient�Primary�Key�Updates:
------ on�primary�key�constraints�that�have�a�non-unique�index�under�them.
------ Tables�With�CLOB,�BLOB,�LONG,�NCLOB�or�LONG�RAW�Columns�in�ALL�Schemas------ Tables�With�Columns�of�UNSUPPORTED�Datatypes�in�ALL�Schemas
------ All�tables�that�have�compression�enabled�(need�IE�for�support):
------ Cluster,�or�Object�Tables�- ALL�UNSUPPORTED�- in�ALL�Schemas
------ IOT�(Fully�support�for�Oracle�10GR2�(with�or�without�overflows)�using�GGS�10.4�and�higher)�- in�All�Schemas:
------ Tables�with�Domain�or�Context�Indexes
Object
MOS
Copyright © 2020, Oracle and/or its affiliates 19
DBA GoldenGate
/� A)/� Offload�/�DR /
B)
C)( )�
/�( )
D)PoC /�
E)
A)
B)
C)
D)
Copyright © 2020, Oracle and/or its affiliates20
– ( )DBA GoldenGate
GoldenGateOracle-Oracle
( )
( )
DB
Copyright © 2020, Oracle and/or its affiliates 21
– ( )DBA GoldenGate
Oracle�Database� (Integrated)GoldenGate 18.1
Classic
Integrated
Classic
Integrated
(REDO) Target�DB Client SQL
(REDO) DB(XStream/Log�Miner)�
DBInbound�Server�
(Trail�File)
https://docs.oracle.com/cd/F22974_01/oracle-db/choosing-capture-and-apply-modes.html#GUID-46EEC8E7-F1A3-41F6-ABEE-706B48DC338F
4�
Copyright © 2020, Oracle and/or its affiliates 22
– (Integrated)�DBA GoldenGate
(Integrated)
Reader PreparePreparePrepare
Builder Capture
Integrated
LogMining Server
LogMining Server�
OracleStreams�Pool�
• Reader:�REDO
• Preparer:�REDO
• Builder:�Prepare� SCN
• Capture:��Logical�Change�Records(LCRs)GoldenGate• LogMining Server LCR
•§ Trail
Copyright © 2020, Oracle and/or its affiliates 23
- (Integrated)�DBA GoldenGate
(Integrated)
Reciver ApplierApplierApplier
Prepare Coordinator
Integrated
InBound ServerLightweight�Streaming�API
InBound Server�
OracleStreams�Pool�
• Receiver:�LCR
• Preparer:�
• Coordinator:�Applier
• Applier:��
• Trail�• Traillogical�change�records�(LCRs)�
• Lightweight�Streaming�APILCR Oracle�DB
Copyright © 2020, Oracle and/or its affiliates24
••
-DBA GoldenGate
?!
?!
?!
1
1
Copyright © 2020, Oracle and/or its affiliates 25
- Classic�DBA GoldenGate
- Replicat◦ Replicat
◦ DataPumpDB
- Capture◦ DB
◦ Capture
Copyright © 2020, Oracle and/or its affiliates 26
– (Replicat)DBA GoldenGate
https://docs.oracle.com/cd/F22974_01/oracle-db/choosing-capture-and-apply-modes.html#GUID-05FC3EA1-5A9E-4587-9E4E-3532AB4B6FC7
Copyright © 2020, Oracle and/or its affiliates 27
–DBA GoldenGate
GoldenGate 19.1�on�Microservice�Architecture�
Copyright © 2020, Oracle and/or its affiliates 28
-DBA GoldenGate
• DB GoldenGate•• DB
�REDO• AWR REDO
•REDO•
( / /•
POC
DB�to�DB�Source N/W
(replicat) (Target)
??
REDO ?
(× ?)
Copyright © 2020, Oracle and/or its affiliates 29
- (GoldenGate )DBA GoldenGate
A) REDO (logging / )B) Capture/Pump� CPU/C) Trail
A) GoldenGate
A) ReplicatB) Trail
Copyright © 2020, Oracle and/or its affiliates 30
- (GoldenGate )DBA GoldenGate
A) REDO (logging / )- Logging�
◦ Nologging GoldenGate Logging�◦◦ Nologging
-◦ REDO◦ (PK/UK/Unique�Index)◦ 10%◦
※
Copyright © 2020, Oracle and/or its affiliates 31
- (GoldenGate )DBA GoldenGate
B) Capture/Pump� CPU/-
◦ Classic�/�Integrated� Capture REDO30-50MB( )�◦ Data�Pump� Capture
- CPU◦ Classic�Capture� 1�CPU◦ Integrated�Capture� Oracle +α◦ Data�Pump CPU Capture
-◦ Capture
•• Capture OS• Capture
◦ Integrated�Capture� Oracle streams_pool_size◦ Data�Pump Capture
※
Copyright © 2020, Oracle and/or its affiliates 32
•• Certification�Matrix••
• /•• DR
GoldenGate
••
DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates 33
-DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates 34
-DBA GoldenGate
update … where key=‘P001’
update … where key=‘P001’
※
delete
(insert / update)
Copyright © 2020, Oracle and/or its affiliates35
•• DR (H/W,OS )•• ( )
Switch Over
- DRDBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates 36
- ( )DBA GoldenGate
Application Application Application
1)� 2)� 3)�
Copyright © 2020, Oracle and/or its affiliates37
• … ?
- ( )DBA GoldenGate
ApplicationApplication
Copyright © 2020, Oracle and/or its affiliates 38
- GoldenGate (1) w/RACDBA GoldenGate
Real�Application�Clusters� GoldenGate• Node1� GoldenGate• GoldenGate F/O�• GoldenGate S/O�
Trail FileTrail�File� GoldenGateI/O
---DBFS�/�ACFS
NFS
GoldenGateOracle�Clusterware
Node1:
Node2:
RAC GI�(Oracle� )�
Copyright © 2020, Oracle and/or its affiliates 39
- GoldenGate (1) w/Data GuardDBA GoldenGate
Data�Guard� GoldenGate• Primary�DB GoldenGate• GoldenGate F/O�• GoldenGate S/O�
Data Guard
GoldenGate
transaction
Trail�File� GoldenGateI/O
---DBFS�/�ACFS
NFS
Data�Guard� REDOCaptureData�Guard� GoldenGate
Primary DB(GG: Source DB)
Standby DB Trail File
GG : Target DB
Copyright © 2020, Oracle and/or its affiliates40
•• Certification�Matrix••
• /•• DR
GoldenGate
••
DBA GoldenGate
Copyright © 2020, Oracle and/or its affiliates 41
-DBA GoldenGate
No.
1 Remote�Extract Oracle 11.2.0.4+
2 Cross�Endian�Extract Big�endian�<->�little�endian� ( ) Oracle 11.2.0.4+
3 Microservice�Architecture REST Web I/F
4 instantiation�scn Oracle
5 Parallel�Replicat Oracle
6 GG�on�OCI�Marketplace� Cloud GoldenGate (VM) GGCS->DIPC->GG�on�MP
Oracle�Database�19c� GoldenGate
Copyright © 2020, Oracle and/or its affiliates 42
GoldenGate 19.1�DBA GoldenGate
Remote�Cross-Endian�Integrated�Extract�
Any�Targe
t
https://www.oracle.com/technetwork/middleware/ias/downloads/fusion-certification-100350.html
All�Certifications:
OGG�19.1OGG�19.1 Linux OGG DB�on�Solaris�
Oracle�Database 11.2.0.4
OracleSolaris
HP-UX
IBMAIX
Red�HatEnterprise�Linux
Copyright © 2020, Oracle and/or its affiliates 43
- Cross�Endian�Remote�CaptureDBA GoldenGate
Solaris (Oracle R12.1 non-CDB)Oracle Linux (Cross Endian Remote Capture) Oracle Linux (Oracle R19.3 : PDB) Oracle
Solaris
Copyright © 2020, Oracle and/or its affiliates 44
- <Microservice�Architecture>DBA GoldenGate
Classic�Architecture
Source Target
TCP/IPTrail�File Trail�File
ManagerProcess
ManagerProcess
Classic orIntegrated
Copyright © 2020, Oracle and/or its affiliates 45
-DBA GoldenGate
Classic�Architecture� I/F�<GGSCI>
#ggsci <return>
GGSCI> info allProgram Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNING EXTRACT RUNNING EINTA 00:00:09 00:00:00 EXTRACT RUNNING PUMP 00:00:00 01:21:21
Copyright © 2020, Oracle and/or its affiliates 46
-DBA GoldenGate
Microservice�Architecture
Source Target
TCP/IPTrail�File Trail�File
Distribution Receiver
ServiceManager
ServiceManager
AdministrationServer
AdministrationServer
Classic orIntegrated
Copyright © 2020, Oracle and/or its affiliates 47
- -DBA GoldenGate
Service�Manager� Microservice�Architecture�
Copyright © 2020, Oracle and/or its affiliates 48
- -DBA GoldenGate
Administration�Server�
Copyright © 2020, Oracle and/or its affiliates 49
(Oracle�Cloud�Infrastructure)DBA GoldenGate
GoldenGate on�OCI�Marcketplace
Copyright © 2020, Oracle and/or its affiliates50
GoldenGate on�OCI�Macketplace
- /�GoldenGate on�OCI�MacketplaceDBA GoldenGate
(On-P)• Remote capture / Remote replicat
Copyright © 2020, Oracle and/or its affiliates 51
- /�GoldenGate on�OCI�MacketplaceDBA GoldenGate
GoldenGate on�OCI�Marketplace
Target�DBSource�DB
./oracle11
./oracle12
./oracle18
./oracle19
./dirdatTrail�File�
Remote�Integrated�Capture Remote�ReplicatOracle�11g
(R11.2.0.4) Oracle�19c
GoldenGate 19.1
Copyright © 2020, Oracle and/or its affiliates 52
-DBA GoldenGate
https://www.oracle.com/middleware/technologies/goldengate.html
•Oracle�GoldenGate 19.1�Data�Sheet�(PDF)
•Oracle�GoldenGate Studio�12c�Data�Sheet�(PDF)•Oracle�Management�Pack�for�Oracle�GoldenGate Data�Sheet�(PDF)
•Oracle�GoldenGate for�Big�Data�Data�Sheet�(PDF)
•Ensuring�Data�Consistency�with�Oracle�GoldenGate Veridata (PDF)
Copyright © 2020, Oracle and/or its affiliates53
GoldenGate Management�PackGoldenGate Studio
-DBA GoldenGate
GUI (Big Data / non-Oracle )
GUI ( / )
• GoldenGate�Monitor
• GoldenGate�Director• GoldenGate Plug-In�for�EMCC
Copyright © 2020, Oracle and/or its affiliates 54
DBA GoldenGate
GoldenGate for�Big�Data
GG for Big Data
Handler
Flat File
Streams
No SQL
Big Data
DBMS
Cloud
- ( )
Copyright © 2020, Oracle and/or its affiliates 55
-DBA GoldenGate
GoldenGate VeridataDB S/W
Copyright © 2020, Oracle and/or its affiliates 56
DBA GoldenGate
• MAA�white�paper�Oracle�GoldenGate Performance�Best�Practiceshttp://www.oracle.com/technetwork/database/availability/maa-gg-performance-1969630.pdf
• MOS�Doc�ID�1557031.1� Recommended�patches�for�Integrated�Capture/Replicat• MOS�Doc�ID�1448324.1� OGG�Integrated�Healthcheck Script• MOS�Doc�ID�1488668.1��GoldenGate Performance�Data�Gathering• MOS�Doc�ID�1596493.1� GoldenGate Integration�with�AWR• MOS�Doc�ID�1485620.1� Best�Practices�Downstream�Capture•
http://www.slideshare.net/oracle4engineer/oracle-goldengate-r122
Copyright © 2020, Oracle and/or its affiliates 57
Replicat :
REPLICAT REP01USERID ogg_user, PASSWORD ogg_userASSUMETARGETDEFSDISCARDFILE ./dirrpt/REP01.DSC, PURGEMAP sales_app.demo*, TARGET sales_app.*;
Extract: (DataPump)
EXTRACT DP01PASSTHRURMTHOST blue, MGRPORT 7809RMTTRAIL ./dirdat/rtTABLE sales_app.demo*;
Extract: (Capture)
EXTRACT CAP01USERID ogg_user, PASSWORD ogg_userEXTTRAIL ./dirdat/ltTABLE sales_app.demo*;
-DBA GoldenGate
58 Confidential – © 2019 Oracle Internal/Restricted/Highly Restricted
Online�Manual• Using�Oracle�GoldenGate on�Oracle�Cloud�Marketplacehttps://docs.oracle.com/en/middleware/goldengate/core/19.1/oggmp/
• Oracle�GoldenGate 19.1https://docs.oracle.com/en/middleware/goldengate/core/19.1/index.html- Oracle�GoldenGate Users�(Database)�- Granting�the�Appropriate�User�Privileges
https://docs.oracle.com/en/middleware/goldengate/core/19.1/oracle-db/establishing-oracle-goldengate-credentials.html#GUID-79122058-27B0-4FB6-B3DC-B7D1B67EB053
- Replicating�Data�to�the�Autonomous�Databasehttps://docs.oracle.com/en/middleware/goldengate/core/19.1/oracle-db/replicating-data-oracle-autonomous-data-warehouse-cloud.html#GUID-E878AFC9-40F4-4F5E-8A72-E1A50ECECC82
- Loading�Data�from�File�to�Replicat in�Microservices Architecturehttps://docs.oracle.com/en/middleware/goldengate/core/19.1/admin/loading-data-file-replicat-ma-19.1.html
- Installing�Support�for�Oracle�Sequenceshttps://docs.oracle.com/en/middleware/goldengate/core/19.1/oracle-db/additional-oracle-goldengate-configuration-considerations.html#GUID-2FBECF4E-1D8D-42A7-B4B1-A1FDE56039D8
- How�to�Use�the�Admin�Clienthttps://docs.oracle.com/en/middleware/goldengate/core/19.1/using/how-use-admin-client.html#GUID-84B33389-0594-4449-BF1A-A496FB1EDB29
- GoldenGate RDS-Oracle Cloud Infrastructure DBA GoldenGate
59 Confidential – © 2019 Oracle Internal/Restricted/Highly Restricted
Online�Manual• Oracle�GoldenGate 19.1� Classic�Extract�
- Configuring�Capture�in�Classic�Mode�https://docs.oracle.com/en/middleware/goldengate/core/19.1/oracle-db/configuring-capture-classic-mode.html
- Configuring�DDL�Supporthttps://docs.oracle.com/en/middleware/goldengate/core/19.1/oracle-db/configuring-ddl-support.html#GUID-A2997164-CBDF-4E9B-B2B0-2CE6889BC2DF
- Installing�Trigger-Based�DDL�Capturehttps://docs.oracle.com/en/middleware/goldengate/core/19.1/oracle-db/installing-trigger-based-ddl-capture-1.html#GUID-70B1B422-2369-40BA-B845-7AE2F7D08A3E
MOS• Oracle�GoldenGate -- Oracle�RDBMS�Server�Recommended�Patches�(Doc�ID�1557031.1)https://mosemp.us.oracle.com/epmos/faces/DocumentDisplay?id=1557031.1
Amazon�RDS�Online�Manual• Oracle�Database�Engine�Release�Noteshttps://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.Oracle.PatchComposition.html
- GoldenGate RDS-Oracle Cloud Infrastructure DBA GoldenGate
60 Confidential – © 2019 Oracle Internal/Restricted/Highly Restricted
- Autonomous Database (ADB HOL) DBA GoldenGate
ADB�HOLhttps://community.oracle.com/docs/DOC-1034727
ADB�HOL�#8-3�:GoldenGate
https://community.oracle.com/docs/DOC-1035834https://community.oracle.com/docs/DOC-1035835
Copyright © 2020, Oracle and/or its affiliates61
Thank you
Our mission is to help peoplesee data in new ways, discover insights,unlock endless possibilities.
top related