Active/Active Database Solutions with Log Based Replication in xDB 6.0

© Copyright EnterpriseDB Corporation, 2015. All rights reserved. 1

Active / Active Database Solutions with Log Based Replication in xDB 6.0


•  xDB Overview −  Features −  Use Cases −  Components of a Multi Master Replication System

•  Supporting Active / Active Database Solutions with xDB 6.0 −  Faster and more efficient replication with log-based Multi Master replication for

Postgres Plus and PostgreSQL −  Easier to configure publication tables in bulk with pattern matching selection rules −  Ensuring High Availability with integration of the 'Control Schema’ −  Improved performance in conflict detection rules

Agenda


xDB is a highly flexible database replication tool to address multiple problem sets

What is xDB?

Multi Master: Consolidated view from any node and multiple conflict detection / resolution rules •  Write Availability & Scalability •  Warm Standby •  Localized Data Access

Single Master: Heterogeneous (Oracle, SQL Server, PG, PPAS) & filtered data sources •  Offload Reporting or BI Queries from the

OLTP system •  Migrate or test systems at low cost


Features •  Distributed Publication / Subscription

Architecture •  Snapshot and continuous synchronization

modes •  Replicate one or more tables •  Define and apply row filters •  Flexible replication scheduler •  Supports cascading replication •  Replication History Viewer •  Graphical Console and CLI

Use Cases •  Write Availability •  Write Scalability •  Warm Standby Servers •  Localized Data Access •  Offloading Reporting and Business

Intelligence Queries •  Testing Systems in Parallel •  Migrating Data

xDB Feature Highlights


Why xDB over other replication tools?

•  Rich graphical management console −  complete control over your replication system −  a history viewer −  a replication scheduler

•  Command line operations suitable for scripting.

•  Multi-Master is an ideal solution for geographically dispersed data centers or co-located redundant data sets. It provides a critical component of a 24x7 near zero downtime maintenance operation.

•  Single Master Heterogeneous Sources provide integration and a path to reduce other license and maintenance costs


•  Publication Server & Configuration Files: −  Java application; configures publication database and

master nodes for replication; performs replication. •  Controller database (Can be any Master Node):

−  Primary provider of the replication system information to the publication server.

−  Must be first one running and accessible to publication server; can be set to any node.

•  Master Definition Node −  First node added to the MMR system −  Table definitions and optionally initial data to publish −  Role can be reassigned

•  Master Node −  Replicated & updatable data stores

•  More Info in Documentation Section 2.3.1.x – Physical Components of xDB system

Architecture and Physical Components of xDB MMR xDB is a set of programs + datastores containing configuration information and metadata that form a replication system.


•  xDB Overview −  Features −  Use Cases −  Components of a Multi Master Replication System

•  Supporting Active / Active Database Solutions with xDB 6.0 −  Faster and more efficient replication with log-based Multi Master replication for

Postgres Plus and PostgreSQL −  Easier to configure publication tables in bulk with pattern matching selection

rules −  Ensuring High Availability with integration of the 'Control Schema’ −  Improved performance in conflict detection rules

Agenda


•  Changes to rows in the source tables are extracted from the Write-Ahead Log segments (WAL files).

•  xDB connects via walsender interface. Changes are streamed on continuous basis. Eliminates need to poll.

•  The extracted changes are converted to an in-memory data structure and applied to the target tables by means of SQL statements executed using JDBC.

•  See Documentation section 2.2.10 for information on the log-based method.

Faster and more efficient replication with log-based MMR for PPAS & PG 9.4 Capture data changes without impacting the online transaction processing rate

High-level architecture of how xDB communicates with database server to fetch logical changes.


•  Changes to rows in the source tables are extracted from the Write-Ahead Log segments (WAL files).

•  xDB connects via walsender interface. Changes are streamed on continuous basis. Eliminates need to poll.

•  The extracted changes are converted to an in-memory data structure and applied to the target tables by means of SQL statements executed using JDBC.

•  See Documentation section 2.2.10 for information on the log-based method.

Faster and more efficient replication with log-based MMR for PPAS & PG 9.4 Capture data changes without impacting the online transaction processing rate


•  When you have a lot of tables to choose from when building your publication, the wildcard selection dialog is a useful tool

•  Here we see a few examples −  % as a wildcard character to identify all

tables of the _gb designation −  [character lists] to identify more patterns

with specific characters

•  Detailed Help is available in the client •  See Documentation section 7.1 for

information on the wildcard selector UI.

Easier to Configure Publication Tables in Bulk Use sophisticated pattern matching rules to identify tables for publication


•  In an xDB MMR systems, if any master goes offline, remaining nodes continue to synchronize amongst themselves.

•  When an offline master is brought back, pending transactions are synchronized.

•  Each node serves as a backup of other nodes* •  As part of the Control Schema integration project,

normal replication configuration (control tables, procedures/functions, sequences…) and history tables are also replicated amongst master nodes.

•  Thus, there is resiliency built into the system such that you could lose any database node and replication operations will continue

•  See Documentation 6.11 for more information

*Keep care when using Table Filters, as some nodes may not have all data replicated onto it.

Ensuring High Availability

Control Schema

Control Schema

Control Schema


•  There is however special significance of the Controller Database −  Publication server accesses it for configuration

information −  So any changes made to system config using GUI

or CLI is done on Controller DB first

•  For planned maintenance −  You can identify a different node to be your

Controller using the GUI Console or CLI

•  For unplanned maintenance −  Edit the xDB Configuration file to contain

connection information of another master node −  Restart publication server. −  You can now use the CLI or GUI tool to connect to,

monitor and manage the replication system.

•  See Documentation 6.11 for more information

Ensuring High Availability


•  xDB 5.x conflict detection query is based on IN clause for doing the comparison of PK attribute(s).

•  i.e. SELECT s.* FROM _edb_replicator_pub.rrst_public_orders WHERE order_id IN (1,5,10,15,20,21);

•  Performance starts to degrade as table grows or if PK type is not numeric.

•  We replaced the existing query with a ‘common table expression’ optimize the query execution time. •  i.e. WITH temp_table AS (SELECT…

Improved Performance in Conflict Detection Over 100x improvement with large tables and non-numeric PK

Internal Testing: Create table and seed 10 million rows of text data

create table x (t text not null); insert into x select md5(n::text) from (select generate_series(1, 10000000)) w(n);

Show timing with old and new algorithm\timing ontest=# select xdb60_style(); --Time: 3,629.105 ms (3 seconds)

test=# select xdb51_style(); --Time: 2,198,287.369 ms (36 minutes!)


•  Active / Active Databases with fast and efficient log based replication and high performance conflict detection algorithms

•  Pattern matching selection rules make it easier to configure publication tables in bulk

•  Simple High Availability is now possible with integration of the 'Control Schema’

Summary – xDB 6.0 Enables New Replication Solutions!


Active/Active Database Solutions with Log Based Replication in xDB 6.0

Software