PostgreSQL High Availability via SLONY and PG POOL II

[email protected]

PG East, 2010by Kevin Kempter

SLONY Replication and PG POOL II

Except where otherwise noted, this work is licensed underhttp://creativecommons.org/licenses/by/3.0/

[email protected]


Session Topics

● SLONY Replication● SLONY Walk Through● PG POOL II

[email protected]


SLONY Replication

● SLONY is a “Master to Multiple Slaves” Replication System

● Topics We'll Cover:– Installation and General Info

– Creating and activating a replication set

– Methods for Failover & Switchover

● Note: Thie is ONE way to setup SLONY, not necessarily the only way

[email protected]


SLONY Installation

● Download

● Uncompress

● Configure

– ./configure [options]

– ./configure –with-pgconfigdir=<dir> \ --with-pgbindir=<dir> \ --with-pgsharedir=<dir>● Make

● Make Install (as root)

[email protected]


SLONY The SLONIK Preamble

● Each server to be included in the cluster must be prepared

● Slony will create a schema on each node (i.e. if your replication cluster is named customer then slony will create a schema on each node called _customer.)

● Allow slony to create this cluster schema itself (i.e. keep your hands off)

● The same preamble must be included in each slonik script to manage the cluster :

cluster name = customer;node 1 admin conninfo = 'dbname=custdb host=yoda user=slony';node 2 admin conninfo = 'dbname=custdb host=r2d2 user=slony';node 3 admin conninfo = 'dbname=custdb host=c3po user=slony';

[email protected]


SLONY General Info

● The database must exist on all nodes

● The PL/pgSQL language must be installed on all nodes

● Connection Strings and passwords (use trust or a .pgpass file)

[email protected]


SLONY Create Variable File

● The use of a variables file can help you manage the setup process

● Example (slony_setup.env):export CLUSTERNAME=sample_rep

export MASTERDBNAME=slony_test

export MASTERHOST=localhost

export MASTERPORT=5444

export SLAVEDBNAME=slony_test

export SLAVEHOST=localhost

export SLAVEPORT=5445

export REPUSER=postgres

[email protected]


SLONY Create / Prepare Slave Node

● Create the database on the slave node

● Replicate users on the slave node

● Create the database structures (DDL) on the slave node

● Install PL/pgSQL on the slave node

[email protected]


SLONY Initialize the replication cluster

● Initialize the replication cluster

● Create a replication set

● Add tables to the replication set

● Store the slave node

● Store paths

[email protected]


SLONY Start the slon daemons

● $ slon [options] clustername [connection string]Options:

-h print usage message and exit

-v print version and exit

-d <debuglevel> verbosity of logging (1..4)

-s <milliseconds> SYNC check interval (default 10000)

-t <milliseconds> SYNC interval timeout (default 60000)

-o <milliseconds> desired subscriber SYNC processing time

-g <num> maximum SYNC group size (default 6)

[email protected]


SLONY Start the slon daemons – More Options

More Options:

-c <num> how often to vacuum in cleanup cycles

-p <filename> slon pid file

-f <filename> slon configuration file

-a <directory> directory to store SYNC archive files (SLONY Log Shipping)

-x <command> program to run after writing archive file

-q <num> Terminate when this node reaches # of SYNCs

-r <num> # of syncs for -q option

-l <interval> this node should lag providers by this interval

[email protected]


SLONY Start replication

● Subscribe set

SUBSCRIBE SET (

ID = 1,

PROVIDER = 1,

RECEIVER = 3,

FORWARD = YES

);

[email protected]


SLONY Switchover

lock set (id = 1, origin = 1);

wait for event (origin = 1, confirmed = 2);

move set (id = 1, old origin = 1, new origin = 2);

wait for event (origin = 1, confirmed = 2);

[email protected]


SLONY Failover

failover (id = 1, backup node = 2);

drop node (id = 1, event node = 2);

[email protected]


SLONY DDL Execution

● ALTER TABLES, CREATE INDEXES, etc– EXECUTE

● CREATE TABLE(s) and add to replication– Merge Set

[email protected]


SLONY EXECUTE

#!/bin/sh

. ../etc/slony.env

slonik <<_EOF_

cluster name = $CLUSTERNAME ;

node 1 admin conninfo = 'dbname=$MASTERDBNAME host=$MASTERHOST port=$MASTERPORT user=$REPUSER';

node 2 admin conninfo = 'dbname=$SLAVEDBNAME host=$SLAVEHOST port=$SLAVEPORT user=$REPUSER';

EXECUTE SCRIPT ( SET ID = 1, FILENAME = '/usr/local/pgsql/slony_init/test.sql', EVENT NODE = 1);

_EOF_

[email protected]


SLONY Merge Set

#!/bin/sh

. ../etc/slony.env

slonik <<_EOF_

cluster name = $CLUSTERNAME ;

node 1 admin conninfo = 'dbname=$MASTERDBNAME host=$MASTERHOST port=$MASTERPORT user=$REPUSER';

node 2 admin conninfo = 'dbname=$SLAVEDBNAME host=$SLAVEHOST port=$SLAVEPORT user=$REPUSER';

merge set (id=1, add id=2, origin=1);

_EOF_

[email protected]


SLONY Example / Walk through

● SLONY Setup● IP alias setup / control● switchover

[email protected]


SLONY

Summary

[email protected]


PG POOL II

[email protected]


PG POOL IIOverview

● Connection Pooling– Connections beyond max_connections are

queued instead of rejected

● Replication ● Load Balancing● Parallel Query

[email protected]


PG POOL IIInstallation Prerequisites

● PostgreSQL header files● libpq● make

[email protected]


PG POOL IIInstallation

● Install PostgreSQL on all nodes

● Install pgpool on node 1– ./configure --prefix=<install dir> \

--with-pgsql=<path to top level pg install dir>

– e.g.:

./configure --prefix=/usr/local/pgsql/pgpool \

--with-pgsql=/usr/local/pgsql/pg841

– $ make

– $ sudo make install

– Chown -R postgres:postgres <install dir>

[email protected]


PG POOL IIConfiguration

● <install dir>/etc/pgpool.conf (copied from pgpool.conf.sample)

# connections

listen_addresses = 'localhost'

# Port number for pgpool

port = 9999

[email protected]


PG POOL IIConfiguration (cont)

# number of pre-forked child process

num_init_children = 32

# Number of connection pools allowed for a child process

max_pool = 4

# If idle for this many seconds, child exits. 0 means no timeout.

child_life_time = 300

# If idle for this many seconds, connection to PostgreSQL closes.

# 0 means no timeout.

connection_life_time = 0

[email protected]



# If child_max_connections connections were received, child exits.

# 0 means no exit.

child_max_connections = 0

# If client_idle_limit is n (n > 0), the client is forced to be

# disconnected whenever after n seconds idle (even inside an explicit

# transactions!)

# 0 means no disconnect.

client_idle_limit = 0

[email protected]



# Maximum time in seconds to complete client authentication.

# 0 means no timeout.

authentication_timeout = 60

# Logging directory

logdir = '/tmp'

# pid file name

#pid_file_name = '/var/run/pgpool/pgpool.pid'

pid_file_name = '/usr/local/pgsql/pgpool/etc/pgpool.pid'

[email protected]



# Replication mode

replication_mode = false

# Load balancing mode, i.e., all SELECTs are load balanced.

# This is ignored if replication_mode is false.

load_balance_mode = false

# if there's a data mismatch between master and secondary

# start degeneration to stop replication mode

replication_stop_on_mismatch = false

[email protected]



# If true, replicate SELECT statement when load balancing is disabled.

# If false, it is only sent to the master node.

replicate_select = false

# Semicolon separated list of queries to be issued at the end of a session

reset_query_list = 'ABORT; RESET ALL; SET SESSION AUTHORIZATION DEFAULT'

# for 8.3 or newer PostgreSQL versions DISCARD ALL can be used as

# follows. However beware that DISCARD ALL holds exclusive lock on

# pg_listener so it will be a serious performance problem if there are

# lots of concurrent sessions.

# reset_query_list = 'ABORT; DISCARD ALL'

[email protected]



# If true print timestamp on each log line.

print_timestamp = true

# If true, operate in master/slave mode.

master_slave_mode = false

# If true, cache connection pool.

connection_cache = true

[email protected]



# Health check timeout. 0 means no timeout.

health_check_timeout = 20

# Health check period. 0 means no health check.

health_check_period = 0

# Health check user

health_check_user = 'nobody'

[email protected]



# Execute command by failover.

# special values: %d = node id

# %h = host name

# %p = port number

# %D = database cluster path

# %m = new master node id

# %M = old master node id

# %% = '%' character

#

failover_command = ''

[email protected]



# Execute command by failback.

# special values: %d = node id

# %h = host name

# %p = port number

# %D = database cluster path

# %m = new master node id

# %M = old master node id

# %% = '%' character

#

failback_command = ''

[email protected]



# If true, automatically locks a table with INSERT statements to keep

# SERIAL data consistency. If the data does not have SERIAL data

# type, no lock will be issued. An /*INSERT LOCK*/ comment has the

# same effect. A /NO INSERT LOCK*/ comment disables the effect.

insert_lock = true

# If true, ignore leading white spaces of each query while pgpool judges

# whether the query is a SELECT so that it can be load balanced. This

# is useful for certain APIs such as DBI/DBD which is known to adding an

# extra leading white space.

ignore_leading_white_space = true

[email protected]



# If true, print all statements to the log. Like the log_statement option # to PostgreSQL, this allows for observing queries without engaging in full # debugging. # log_statement = false log_statement = true

# If true, incoming connections will be printed to the log.

log_connections = false

# If true, hostname will be shown in ps status. Also shown in

# connection log if log_connections = true.

# Be warned that this feature will add overhead to look up hostname.

log_hostname = false

[email protected]



# if non 0, run in parallel query mode

parallel_mode = false

# if non 0, use query cache

enable_query_cache = false

#set pgpool2 hostname

pgpool2_hostname = '192.168.242.137'

[email protected]



# system DB info

#system_db_hostname = 'localhost'

#system_db_port = 5432

#system_db_dbname = 'pgpool'

#system_db_schema = 'pgpool_catalog'

#system_db_user = 'pgpool'

#system_db_password = ''

[email protected]



# backend_hostname, backend_port, backend_weight

# here are examples

backend_hostname0 = '192.168.242.138'

backend_port0 = 5432

backend_weight0 = 1

backend_data_directory0 = '/usr/local/pgsql/pg841/data'

#backend_hostname1 = 'host2'

#backend_port1 = 5433

#backend_weight1 = 1

#backend_data_directory1 = '/data1'

[email protected]



# - HBA -

# If true, use pool_hba.conf for client authentication. In pgpool-II

# 1.1, the default value is false. The default value will be true in

# 1.2.

enable_pool_hba = false

[email protected]



# - online recovery -

# NOTE: these values are used to re-attach (after failure) a node or to attach a new node# when pgpool replication is used# online recovery user

recovery_user = 'nobody'

recovery_password = ''

recovery_1st_stage_command = ''

recovery_2nd_stage_command = ''

# maximum time in seconds to wait for the recovering node's postmaster

# start-up. 0 means no wait.

recovery_timeout = 90

[email protected]


Start PG POOL

● $ cd <install dir>/bin

● $ ./pgpool -n > log 2>&1 &

[email protected]


Controlling PG POOL

Usage:

pgpool [ -c] [ -f CONFIG_FILE ] [ -F PCP_CONFIG_FILE ] [ -a HBA_CONFIG_FILE ]

[ -n ] [ -d ]

pgpool [ -f CONFIG_FILE ] [ -F PCP_CONFIG_FILE ] [ -a HBA_CONFIG_FILE ]

[ -m SHUTDOWN-MODE ] stop

pgpool [ -f CONFIG_FILE ] [ -F PCP_CONFIG_FILE ] [ -a HBA_CONFIG_FILE ] reload

[email protected]


Controlling PG POOL (cont)

Common options:

-a HBA_CONFIG_FILE Sets the path to the pool_hba.conf configuration file

(default: /usr/local/pgsql/pgpool/etc/pool_hba.conf)

-f CONFIG_FILE Sets the path to the pgpool.conf configuration file

(default: /usr/local/pgsql/pgpool/etc/pgpool.conf)

-F PCP_CONFIG_FILE Sets the path to the pcp.conf configuration file

(default: /usr/local/pgsql/pgpool/etc/pcp.conf)

-h Prints this help

[email protected]


Controlling PG POOL (cont)

Start options:

-c Clears query cache (enable_query_cache must be on)

-n Don't run in daemon mode, does not detach control tty

-d Debug mode

Stop options:

-m SHUTDOWN-MODE Can be "smart", "fast", or "immediate"

Shutdown modes are:

smart quit after all clients have disconnected

fast quit directly, with proper shutdown

immediate quit without complete shutdown; will lead to recovery on restart

[email protected]


Configure for pcp commands

● Use pg_md5 to create passwd hash$ /usr/bin/pg_md5 postgres

e8a48653851e28c69d0506508fb27fc5

● pcp.conf

postgres:e8a48653851e28c69d0506508fb27fc5

[email protected]


pcp Commands

pcp_node_count

pcp_node_count _timeout_ _host_ _port_ _userid_ _passwd_

Displays the number of total nodes defined in pgpool.conf. It does not distinguish nodes status, ie attached/detached. ALL nodes are counted.

pcp_proc_count

pcp_proc_count _timeout_ _host_ _port_ _userid_ _passwd_

Displays the list of pgpool-II child process IDs. If there is more than one process, IDs will be delimitted by a white space.

[email protected]


pcp Commands

pcp_node_info

pcp_node_info _timeout_ _host_ _port_ _userid_ _passwd_ _nodeid_

Displays the information on the given node ID. The output example is as follows:

The result is in the following order:

Hostname port number status load balance weight

Status is represented by a digit from [0 to 3].

0 - This state is only used during the initialization. PCP will never display it.

1 - Node is up. No connections yet.

2 - Node is up. Connections are pooled.

3 - Node is down.

[email protected]


pcp Commands

pcp_proc_info

pcp_proc_info _timeout_ _host_ _port_ _userid_ _passwd_ _processid_

Displays the information on the given pgpool-II child process ID.

pcp_systemdb_info

pcp_systemdb_info _timeout_ _host_ _port_ _userid_ _passwd_

Displays the System DB information.

[email protected]


pcp Commands

pcp_detach_node

pcp_detach_node _timeout_ _host_ _port_ _userid_ _passwd_ _nodeid_

Detaches the given node from pgpool-II.

pcp_attach_node

pcp_attach_node _timeout_ _host_ _port_ _userid_ _passwd_ _nodeid_

Attaches the given node to pgpool-II.

[email protected]


pcp Commands

pcp_stop_pgpool

pcp_stop_pgpool _timeout_ _host_ _port_ _userid_ _passwd_ _mode_

Terminate pgpool-II process with the given shutdown mode. The availabe modes are as follows:

s - smart mode

f - fast mode

i - immediate mode

[email protected]


End

PostgreSQL High Availability via SLONY and PG POOL II

Documents

kevin kempterslony example

kevin kempterslony merge

replication cluster

node id

setup slony

event node

replication set store

replication merge