pureScale 014 revision 2 - DUGIdugi.molaro.be/wp-content/uploads/2009/10/pureScale_DB2_RUG_site… · – Limited to AIX and IBM ... – single command addition or removal of cluster’s

© 2009 [email protected] 0

DB2 pureScale Technology preview

CRISTIAN MOLARO Belgium 2009


DISCLAIMER

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. THE AUTHOR SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION

The content of this presentation is based on information provided by IBM to the general public or trough the IBM Information Champion program ! there is no guaranty on the correctness of the contents or the comments expressed during this presentation My opinions are mine


Agenda

PURESCALE AT A GLANCE – GOALS – ARCHITECTURE

HOW IT WORKS – TECHNICAL FUNDAMENTS – CAPACITY – AVAILABILITY

CONCLUSIONS


pureScale at a glance What is it?


DB2 pureScale goals

Today’s business: – Down-time is not acceptable – Dynamic capacity / capacity on demand / license costs – Application changes are expensive

Continuous availability – Deliver uninterrupted access to data – Online recover, WLB, stealth maintenance

High + flexible capacity – Start with today’s capacity requirements – Add capacity as business needs grow – More granular licensing costs management

Application transparency – Avoid the risk and cost of application changes – Scale without application impact


DB2 pureScale at a glance

DB2 pureScale is a new optional DB2 feature Data Sharing Cluster configuration – multiple database servers + multiple DB2 sharing data and

logs

“Leverage” of DB2 for z/OS data sharing technology Some benefits are: – allows dynamic and easy capacity changes to cope with

business requirements – provides very convenient continuous availability features – application transparency – DBA friendly package solution


More about pureScale

DB2 pureScale would be deployed as follows: – v 9.7 ! continue with normal fixpack stream but does not include

pureScale – v 9.8 ! only for pureScale – v 10 ! pureScale integrated as additional feature of DB2 ESE

Pricing – DB2 licenses do not apply to powerHA pureScale servers – Get advantage of easy scalability: pay extra capacity charges only

when needed, i.e. end of the year processing – Of course: contact your friendly IBM representative for details…


pureScale is a new technology…

At the date of this presentation, pureScale is not GA Initial release December 2009 – Limited to AIX and IBM power hardware – Some features not supported (XML, MDC…) – Focus on OLTP and ERP – Tools integration, PE: 1st or 2nd quarter 2010 – Linux may be supported in the future

Expect to see integration with HADR


pureScale architecture

Single entry point + WLB DB2 cluster services DB2 v9.8 InfiniBand + RDMA IBM Power6 + AIX powerHA pureScale Shared Data + GPFS

Data

Log Log Log

DB2 DB2 DB2

CS CS CS

CS CS

CF CF


Highlights

A lot of efforts were done in order to isolate the DBA from the implementation technical details

– powerHA pureScale policies are predefined, no DBA intervention required

– single command addition or removal of cluster’s members

Easy migration: does not require data movement – 2 paths: GPFS and NOT GPFS

Not exactly the same as z/OS data sharing – Sysplex timer functionalities are software implemented – Cannot run a member in different DB2 versions – pureScale is NOT a replacement of z/OS data sharing!


Availability & capacity

Architected for no single point of failure – Automatic workload balancing – Duplexed global lock and memory manager – Tivoli System Automation automatically handles all

component failures ! DB2 pureScale stays up even with multiple node failures

– Shared disk failure handled using disk replication technology

Capacity – DB2 pureScale has been designed to grow with

business requirements – Flexible licensing designed for minimizing costs of

peak times – Only pay for additional capacity when you use it even

if for only a single day

Data

Log Log

DB2 DB2

CS CS

CS

CF


Other technologies?

DB2 HADR? – Not a replacement of HADR – Not integration (yet) ! DR based on disk technology, for example

SRDF

DB2 DPF? – pureScale focused on OLTP and ERP workload where CPU is

normally the bottleneck ! provides easily scalable CPU processing power

– DPF is a shared nothing cluster configuration ! could be better for Data Warehouse workloads

ORACLE RAC? – pureScale is probably a better solution than RAC


Inter-node communication pureScale: central locking and

memory manager Other database cluster may

require CPU intensive inter-node communication


IBM benchmark: – 95% scalability ! 32 members – 81% scalability ! 112 members

Scalability: example


How it works Technology fundations


pureScale technical overview Client workload balancing

– Single database view – Can create member affinity – Connection and txn level WLB and ACR

DB2 engine runs on >1 hosts – Independent bufferpools, logs, memory

Integrated cluster services (CS) – Part of the solution – Failure detection, recovery automation,

cluster file system High speed interconnection

– InfiniBand and RDMA

PowerHA pureScale (CF) – Global locking and buffer management – Support synchronous duplexing for high

availability

Data sharing architecture – Data and logs accessible by every

member – GPFS technology

Data

Log Log Log

DB2 DB2 DB2

CS CS CS

CS CS

CF CF


GPFS: General Parallel File System Provides file system services to parallel and serial

applications running on multiple nodes Allows parallel applications simultaneous access to the same

files, or different files, from any node that has the GPFS file system Each node that has a GPFS file system mounted must be

able to communicate with all storage devices that are part of this file system Requisite for DB2 pureScale Installation does configure but doesn’t install GPFS 2 migration paths based on current GPFS use or not More information:

– GPFS V3.1 Advanced Administration Guide, SC23-5182-00 – GPFS V3.1 Administration and Programming Reference, SA23-2221-00 – GPFS V3.1 Concepts, Planning, and Installation Guide, GA76-0413-00


CS: DB2 Cluster Services Integrated DB2 component Single install as part of DB2 installation Upgrades and maintenance through DB2

fixpacks DB2 Cluster Services:

– Reliable Scalable Cluster Technology – Tivoli Systems Automation for Multi-Platforms – IBM General Parallel File System – DB2 CS tightly integrates these IBM products into

DB2 pureScale – DB2 instance creation creates RSCT and GPFS

domains across hosts – Single command used to add hosts to the instance: db2iupdt –add -m newhost.toto.be db2inst1

– Install includes DB2, PowerHA pureScale and DB2 Cluster Services

Cluster Manager (RSCT) Cluster Automation (Tivoli SA MP)

GPFS

CS


Members A member is a DB2 engine address space

– i.e. Db2sysc process and its threads

Members share data – All members access the same shared

database – Logs are individual per DB2 but accessible to

all the members

Each member has own structures: – Bufferpools – Memory regions – Log files

Members are logical: – Can implement 1 per host ! recommended – Can implement >1 per host ! not

recommended

All members are at the same software level Add / remove members using db2iupdt

Member 1

db2sysc process

heaps

db2 agents & other threads

buggerpools

Member 0

db2sysc process

heaps

db2 agents & other threads

buggerpools


InfiniBand High speed and low latency network transport protocol used

to connect system nodes in HPC and Large Enterprise Servers Fabric architecture supporting up to

– 40Gb/s host connectivity and – 120Gb/s switch to switch links – measured delays of 1!s end to end

Provides direct support of Remote Direct Memory Access Problem: Network CPU limits CPU bound applications’ throughput

Solution: InfiniBand adapters + RDMA provides zero-copy functionality

Memory Chipset CPU

Network adapter

Receiving host

Memory-copy flow


InfiniBand and RDMA RDMA:

– A direct memory access from the memory of one computer into that of another without involving either one's operating system

– Bypasses the kernel interactions and IP stack on the server allowing the access to take place without interrupting the processing of the CPU

Less than 10 µS between any two elements without involving the CPU in the network data transfer

Memory Chipset CPU

Network adapter

Receiving host

Memory Chipset CPU

Network adapter Sending host

Zero-copy flow


CF GBP GLM SCA

powerHA pureScale Software implementation of System z

Parallel Sysplex & Coupling Facility technology Assists in global bufferpool coherency

management and global locking Services provided include:

– Group Bufferpool (GBP) – Global Lock Management (GLM) – Shared Communication Area (SCA)

Synchronous duplexing of structures to primary and secondary CF

– Done by members – Optional but highly recommended – Set up automatically by default

CF GBP GLM SCA

Primary Secondary


The role of the GBP Acts as fast disk cache

– Dirty pages stored in GBP ! later written to disk – Provides fast retrieval of such pages when

needed ! hot pages available without disk I/O – RDMA and dedicated thread allows read page

operations in 10s of µS

Includes a page registry – Keeps track of what pages are buffered in each

member and at what memory address – Used for fast “silent” invalidation of such pages

when they are written to the GBP ! no CPU cycles on impacted members

FORCE-AT-COMMIT (FAC) protocol – DB2 forces writes of updates pages to the GBP

at COMMIT, or before – GBP synchronously invalidates any copies of

such pages on other members – New references to the page on other members

will retrieve a new copy from the GBP – In progress references to the page can continue

CF

GBP GLM SCA

DB2

bufferpool

DB2

bufferpool Update page

Write page

Page registry

”Silent” invalidation

Read page


DB2

bufferpool

The role of the GLM Grants locks to members upon request

– If not already held by another member, or held in a compatible mode

Maintains global lock state – Which member has what lock, in what

mode – Interest list of pending lock requests for

each lock

Grants pending lock requests when available

– Using asynchronous notification

When a member owns a lock it may grant further locally Lock avoidance

– DB2 avoids lock request when log sequence number in page header indicates no update on the page could be committed

CF

GBP GLM SCA

DB2

bufferpool Update

page

Write page

Page registry

”Silent” invalidation

Read page

Lock Mgr Lock Mgr

Global lock state


Client connectivity and Workload Balancing

Runtime load information used to balance load across members – Load information of all members kept on each member – Information sent to the client regularly – Support for transaction level routing for selected SQL

Failover: load of failed member evenly distributed to other members Fallback: once the failed member is back ! inverse process

Optional affinity to host: set via client configuration


Adding capacity Initial installation

– DBA verifies pre-requisites: AIX, hosts on the network, access to shared disks enabled, etc

– DBA copies the pureScale install image to the Install Initiating Host – DB2 installs the code on the specified hosts using a response file – DB2 creates the instance, members and CFs as directed via GUI

– DB2 adds all members, CFs, hosts, HCA cards, etc. to the domain resources – DB2 creates the cluster file system and sets up each member’s access to it

Add a member – DBA verifies pre-requisite for new host – DBA adds the member

db2iupdt –add –m <MemHostName> InstName

– DB2 does all tasks to add the member to the cluster: • Copies the image and response file to new member

• Runs install • Adds new member to the resources for the instance • Sets up access to the cluster file system for new member

Also possible: – Drop member – Add / drop CF

+


Stealth System Maintenance Goal:

– Allow DBAs to apply system maintenance without negotiating an outage windows

Procedure: – Quiesce the member – Remove and maintain – Re-integrate – Repeat until done

Minimize the Impact of Planned Outages Keep the system up

– During OS fixes – HW updates – Administration

!

!


Member / CF Failure Restart “light”

– Software failure ! Member restarted on Home host

– Hardware failure ! Member restarted on guest host

Online recovery: – Automated failure detection and recovery :

about seconds – Redistribute workload to surviving nodes

immediately – Only data IN FLIGHT in failing member

remains locked during automated recovery Primary PowerHA pureScale Failure

– Secondary CF assumes primary role with no or minimal impact on application RT

!

!


Single failures summary

Failure Other members remain online?

Automatic & transparent?

" "

" "

" "

!

!

!


Multiple failures summary

Failure Other members remain online?

Automatic & transparent?

" "

" "

" "

!

!

!

!

!!

! !


Monitoring

Support of PE expected for 1st / 2nd quarter 2010 Commands:

> db2start 12/13/2009 09:52:59 0 0 SQL1063N DB2START processing was successful. 12/13/2009 09:53:00 1 0 SQL1063N DB2START processing was successful. 12/13/2009 09:53:01 2 0 SQL1063N DB2START processing was successful. 12/13/2008 09:53:01 3 0 SQL1063N DB2START processing was successful. SQL1063N DB2START processing was successful.

> db2instance -list ID TYPE STATE HOME_HOST CURRENT_HOST ALERT 0 MEMBER STARTED host0 host0 NO 1 MEMBER STARTED host1 host1 NO 2 MEMBER STARTED host2 host2 NO 3 MEMBER STARTED host3 host3 NO 4 CF PRIMARY host4 host4 NO 5 CF PEER host5 host5 NO


Conclusions Final thoughts and


Agenda

PURESCALE AT A GLANCE – GOALS – ARCHITECTURE

HOW IT WORKS – TECHNICAL FUNDAMENTS – CAPACITY – AVAILABILITY

CONCLUSIONS & SUMMARY


Conclusions & summary

DB2 for z/OS Data Sharing leveraged into DB2 AIX – State of the art high availability option – Unique scalability features

Designed to be a better option than ORACLE RAC DBA friendly Application friendly Further hardware and software support in the future


THANKS! [email protected]

pureScale 014 revision 2 - DUGIdugi.molaro.be/wp-content/uploads/2009/10/pureScale_DB2_RUG_site… · – Limited to AIX and IBM ... – single command addition or removal of cluster’s

Documents