Replication to SAP HANA Database with Oracle GoldenGate · PDF fileReplica on to SAP HANA Database with Oracle GoldenGate 12c Partner with Bristlecone to op ize your supply chain investment
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Replica on to SAP HANA Database with Oracle GoldenGate 12c
Partner with Bristlecone to op ize your supply chain investment and implement quick -to-value solu ons. To set up a discussion with our team, please send your inquiry to [email protected]
Replication to SAP HANA Database with Oracle GoldenGate 12c
5 | P a g e
Oracle Logon userid, password User ID and password for the source or target database. When implementing Integrated Extract or Replicate, this user must be granted admin privileges with the DBMS_GOLDENGATE_AUTH procedure on the both source and target databases.
Additional OGG Considerations
Password Security
Passwords used by GoldenGate for Database access encrypted using AES
Process Management
GoldenGate can be configured to auto start / restart processes upon any failure
including network failure
Monitoring
All GoldenGate deployments can be monitored by Oracle Enterprise Manager (EM) with
optional Monitor Agent which are not covered in this document
Overview
This document will address how to replicate data from Oracle to SAP HANA Database using Oracle
GoldenGate Application Adapter.
The Oracle GoldenGate Application Adapters extend the capabilities of Oracle GoldenGate to provide
mechanisms for real-time integration with SAP HANA Database through a custom Java Plug-in for
GoldenGate Application Adapter. The custom Java plug-in should be compiled/customized by user as per
specific needs of customer. Plug-in code provides flexibility to users to write Java code on each
operation event based on specific integration/replication needs.
SAP HANA supports JDBC interface to perform DML operations on the objects created in HANA
database. The custom Java adapter is leveraging JDBC interface to push the data into HANA database.
For the purpose of illustration in this whitepaper, we have used Oracle RDBMS as source. GoldenGate
replication engine generates Trail Files on fly as soon as changes made to objects in Oracle databases,
Application Adapter read these Trial Files and write data to HANA target objects using custom Java Plug-
in. Scenario in this document covers only real-time replication use case, Batch processing for existing
data in Oracle database would need a different approach.
Oracle GoldenGate Application Adapters can be used to replicate change data to targets other than a
relational database: for example, ETL, JMS messaging, or custom Java APIs. There are a variety of
options for integration with Oracle GoldenGate using these Adapters:
Replication to SAP HANA Database with Oracle GoldenGate 12c
6 | P a g e
Once Extract and Replicate are running, changes are replicated perpetually.
Notes on Command Syntax: Commands throughout the document make specific references to
directories, file names, checkpoint group names, begin times, etc. Unless otherwise noted, these items
do not have to correspond exactly in your environment; they are used to illustrated concrete examples.
For exact syntax, consult the Oracle GoldenGate Reference Guide.
Solution
As mentioned earlier Oracle GoldenGate (OGG) includes an optional set of adapters that adds functionality to
apply transactional data to targets other than relational databases. The implementation details of OGG Application
Adapter is as follows:
OGG Application Adapter
OGG Application Adapter (aka OGG Java Adapter) provides the ability to execute code written in Java from the
OGG Extract program. OGG for Java consists of two components:
A dynamically linked or shared library, implemented in C/C++, integrating as a user exit (UE) with the OGG
Extract process through a C API.
A set of Java libraries (jars), which comprise the OGG Java API. This Java framework communicates with
the user exit through the Java Native Interface (JNI).
Through the OGG Java API, transactional data captured by OGG can be delivered to targets such as JDBC, Java
Message Service (JMS), disk files, or a custom application’s Java API.
Replication to SAP HANA Database with Oracle GoldenGate 12c
7 | P a g e
Core Product Specific
The OGG Application Adapter is core product specific; meaning the version 11.2 adapter is designed to function
only with the OGG 11.2 core product. This is due to changes in the OGG Trail format and other core product
improvements which prevents the UE (User Exits) from being linked with other versions of the Extract program.
Custom Adapter (custom application’s Java API)
Custom Adapter (aka Custom event handlers, Custom java handlers) can be written in Java to process transaction,
operation and metadata changes captured by Oracle GoldenGate on the source system. These custom Java
handlers can apply these changes to a third-party Java API exposed by the target system. In our case, SAP HANA
has exposed JDBC interface, which we will use to apply OGG changes to SAP HANA database.
The custom adapter extends the Abstract class provided by OGG Application Adapter, which has all the event
handlers. For example, Begin Transaction, Commit Transaction, Insert Operation, Update Operation and Delete
Operation etc. These events are captured inside the custom code and appropriate operation is performed on
objects in HANA database using JDBC interface.
The OGG Application Adapter will read source trail files and call the appropriate events in custom adapter. These
event handlers then format the DML data into SAP HANA specific events. In this custom adapter we have been
accumulating the whole transaction in memory and on the event of Commit Transaction, we are flushing out the
complete transaction into SAP HANA database through JDBC interface.
The Custom Adapter was compiled using MAVEN tool. Please see detailed information in following section.
Architecture
Referring to the architecture diagram below, custom handler integration with SAP HANA Database is developed
using Oracle GoldenGate's Java API and SAP HANA JDBC API. The custom adapter is deployed as an integral part of
the Oracle GoldenGate Pump process (OGG Application Adapter). The Pump and the custom adapter are
configured through Pump parameter file and custom adapter's properties file. We will discuss the adapter
properties for integrating with SAP HANA in the implementation section below.
Replication to SAP HANA Database with Oracle GoldenGate 12c
8 | P a g e
The Pump process executes the custom adapter in its address space. The Pump reads the trail file created by the
Oracle GoldenGate Extract process (Source System) and passes the transactions to the custom adapter. Based on
the configuration, the custom adapter will write the transactions to SAP HANA Database. Please refer to the
Oracle GoldenGate – Developing Custom Java Adapter Handbook for details about the architecture and developing
a custom adapter. Please see section where to go for More Information.
Implementation Process
The process for implementing OGG delivery to SAP HANA includes:
1. Download and install the OGG Java Adapter package
2. Modify the existing Primary Change Data Capture (Extract) to get before images for updates
3. Generate definitions for the source data
4. Setup an Extract Data Pump to move data from the database server to the target implementation server
5. Configure the SAP HANA Database with target tables.
Replication to SAP HANA Database with Oracle GoldenGate 12c
9 | P a g e
6. Configure the target integration server OGG Manager
7. Configure the OGG Java Adapter properties
8. Create the OGG Java Extract Data Pump (Java Data Pump)
Custom Adapter code
The reference code for Custom Adapter can be downloaded from Bristlecone’s E-Delivery site following the link.
Custom Adapter compilation
The compilation of custom adapter was done through MAVEN tool. The Oracle GoldenGate Application Adapter
build comes with ‘out of the box’ examples for HDFS, Hive, HBase & Flume. The examples are available in following
path.
./$INSTALLATION_DIR/AdapterExamples/bigdata/hdfs
./$INSTALLATION_DIR/AdapterExamples/bigdata/hive
./$INSTALLATION_DIR/AdapterExamples/bigdata/hbase
This path also contains PDF document for how to compile the custom adapter for Big Data platforms. We have
used these files and documents as reference to build our Custom Adapter for HANA. We referred section 2.2
Building with Maven. We also changed pom.xml file to remove the dependencies of Big Data platforms.
Replication from Oracle to SAP HANA Database
Setup of OGG on Oracle DB (Source System)
Source Setup
See 12c Installation Guide
1. Install OGG Software
2. Set Library paths, Oracle Environmental Variables
3. Create subdirectories for OGG
GGSCI> create subdirs
4. Enable minimal supplemental logging in the database
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
5. Increase UNDO to allowed maximum (recommend 24 hours)
6. Turn on FORCE LOGGING in the tablespaces where tables are created with