DB2_WebSphere_BestPractice

1DB2 WebSphere Integration Best Practice

Peter ShumIBM Toronto Laboratory

07 November 2007 5:30 p.m. 6:30 p.m.Platform: Linux, Unix, Windows

Session: F13

In many DB2 shops, more and more solutions are based on WebSphere Application Server and yet there is a limited understanding on how WebSphere and DB2 interact with each other beyond just the SQL. In this session, we'll walk through the architecture of WebSphere, the flow of a query, and then identify the best practice for monitoring, configuration, and application when interfacing with DB2.

2IBM Software Group

2003 IBM Corporation2

Agenda

Architecture Overview Performance Monitoring Best Practice Configuration Best Practice Application Best Practice Miscellaneous Whats Coming

3IBM Software Group


Disclaimer

The information in this presentation concerns new products that IBM has announced. The specification of some of the features described in this presentation may change before the General Availability date of these products. Any discussion of non-IBM products is based upon information which has been publicly available and is subject to change

REFERENCES IN THIS PUBLICATION TO IBM PRODUCTS, PROGRAMS, OR SERVICES DOES NOT IMPLY THAT IBM INTENDS TO MAKE THESE AVAILABLE IN ALL COUNTRIES IN WHICH IBM OPERATES

IBM MAY HAVE PATENTS OR PENDING PATENT APPLICATIONS COVERING SUBJECT MATTER IN THIS DOCUMENT. THE FURNISHING OF THIS DOCUMENT DOES NOT IMPLY GIVING LICENSE TO THESE PATENTS.

TRADEMARKS AND REGISTERED TRADEMARKS ARE THE PROPERTY OF THEIR RESPECTIVE COMPANIES

4IBM Software Group


Architecture Overview

Acknowledgment: Diagrams in this section were taken from the DB2 UDB/WebSphere Application Server V6 Performance Tuning and Operations Workshop, by ITSO

5IBM Software Group


J2EE Architecture

Note the 4 components: client, web container, EJB container, and the datasource (i.e. database).The J2EE platform specification describes the run-time environment for a J2EE application. This environment includes application components, containers, and resource manager drivers. The elements of this environment communicate with a specified set of standard services.J2SE Java 2 Platform, Standard Edition provides a complete environment for applications development on desktops and servers and for deployment in embedded environments. It also serves as the foundation for the Java 2 Platform, Enterprise Edition (J2EE) and Java Web Services.RMI-IIOP Remote Method Invocation Internet Inter-Orb Protocol delivers Common Object Request Broker Architecture (CORBA) distributed computing capabilities to the Java 2 platform.JDBC Java Data Base Connectivity is an API (included in both J2SE and J2EE releases) that provides cross-DBMS connectivity to a wide range of SQL databases and access to other tabular data sources, such as spreadsheets or flat filesJMS Java Message Service provides a standard Java-based interface to the message services of a Message-Oriented-Middleware (MOM) of some other provider.JNDI Java Naming and Directory Interface is part of the Java platform, providing applications based on Java technology with a unified interface to multiple naming and directory servicesJAF Java Agent Framework builds upon the Java Beans model by supplying a

b f f iliti d i d t k t d l t d t

6IBM Software Group


WebSphere Appl. Server Architecture (Standalone)

Web container

Application Server

Messaging engines

Node

Adm

inap

plic

atio

n

Application Database

W ebcontainer Inbound chain(

EJB container

JCA services

Config repository (XML files)

Name Server (JNDI)

Security server

appl

icat

ion

(EA

R)

Msg Queue

Msg Queue

manages

managed by external provider

(W ebSphere MQ)Web Services engine

Adm

in s

ervi

ce

JMS, MQ

W eb Service Provider or Gateway

SOAP/HTTP

W eb browser

client

Java client

Client container

HTTP server

W ebSphere plug-in

HTTP(s)

Scripting client

Admin UI

RMI/IIOP

SOAP or RMI/IIOP

HTTP(s)

UD

DI r

egis

try

In WebSphere, you still have the 4 major components of client, web container, EJB container, and databasource. However, many of the services are shared by both the web container and EJB container. WAS components for standalone are:Node - Logical grouping of server processes managed by WebSphere that share a common configuration and operational control -- not necesarily a machineApplication server (instance) - One JVM processWeb container and EJB container - Can be separated but is not recommended due to negative performance impactJNDI and Security services Administrative service - One in each JVM. Manipulates application server configurationAdministrative Console - Web administration interface. Connected through an admin application to the Admin serviceConfiguration repository - No database, but XML file. Contains configuration informationWeb server and Web server plug-in - Accepts requests from clients. If rule describes, forwards request to application server plugin-cfg.xmlEmbedded HTTP transport - Located in Web container. Receives requests from plug-in. HTTP/HTTPS communication with plug-in. Forwards requests to servletsEmbedded Messaging (JMS) Server - Point-to-point and publish/subscribe

i i N t i W bS h E

7IBM Software Group


WebSphere Appl. Server Architecture (w/ Dist. Server)

EJB container

Web container

Application Server

Node Agent

Node

Application Database

UD

DI r

egis

try

Adm

in s

ervi

ce

Config repository

(file)

appl

icat

ion

(EA

R)

Webcontainer Inbound chain(s)

SOAP (Web

Services) engine

Cell

Master repository

(file)

Deployment ManagerAdmin

applicationName Server (NDI)

Admin Service

Admin Service

Session Database

Web browser

client HTTP server

WebSphere plug-in

HTTP(s)

Scripting client

Admin UI

Java client

Client container RMI/IIOP

SOAP or RMI/IIOP

HTTP(s)

WAS components for cluster are:Administrative cell - Several nodes in a single administration domain.Server cluster - Identical application(s) on several nodes. Purpose: workload management, failover, etc. Application servers in a cluster are called "members" and part of the same administrative cell - Can span distributed platform nodes --but NOT distributed and z/OSDeployment Manager - Central point of administration. Manages the entire cell's configuration repository. Required for both horizontal and vertical scalingNode Agent - Communicates directly with Deployment Manager. Used for configuration synchronization and administration tasks. Located in application server instanceNode group - A node group is a new concept introduced with WebSphere Application Server V6. A node group is a grouping of nodes within a cell that have similar capabilities. A node group validates that the node is capable of performing certain functions before allowing those functions. A DefaultNodeGroup is automatically created based on the deployment manager platform. This node group contains the deployment manager and any new nodes with the same platform type.Node can be part of multiple node groups (except in the case of z/OS sysplexgroups)Plus the WAS components for standalone.Web administration interface - In Network Deployment - connected to Deployment ManagerConfiguration repository - No database but XML file Contains configuration

8IBM Software Group


Typical Query Flow

DB2application database

Browser Client

output page

input page

HTML HTML 1

2

3

plug-in 4

13 14

5

Web Container

Servlet

JSP

EJB Container

JNDI

Data Sources

EJB

WebSphere Application Server

7 8a

8b 9 10a 12

6

10b 11

Embedded HTTP Server

EnterpriseJava Bean

Web Server

Even though the full architecture of a J2EE system could be quite complicated, we could still simplify the basic and typical query flow to make it easier to understand how it works for database requests.

1. A Web client requests a URL in the browser (input page).2. The request is routed to the Web server over the Internet.3. The Web server immediately passes the request to the WebSphere plug-in.

All requests go to the WebSphere plug-in first.4. WebSphere plug-in examines the URL, verifies the list of hostname aliases

from which it will accept traffic based on the virtual host information, and chooses a server to handle the request.

5. A stream is created. A stream is a connection to the Web container. It is possible to maintain a connection (stream) over a number of requests. The Web container receives the request and based on the URL, dispatches it to the proper servlet.

6. If the servlet class is not loaded, the dynamic class loader loads the servlet. Servlet init(), then doGet() or doPost().

7. JNDI is now used for lookup of either datasources or EJBs required by the servlet. Depending upon whether a datasource is specified or an EJB is requested,the JNDI will direct the servlet:

8. a) To the corresponding database, and get a connection from its connection pool in the case of a datasourceb) To the corresponding EJB container, which then instantiates the EJB, when an EJB is requested

9IBM Software Group


Multiple Caching Tiers and Caches Browser cache Web Container

Static file cache Dynamic cache (includes Servlet/JSP result caching) Edge Side Includes (ESI) Web Service Call results

EJB Container Command cache Java Object cache JNDI cache EJB caching

Application Server-to-Database Layer Connection pool Prepared statement cache

Database Buffer Pool Catalog Cache Package Cache

Browser cache -- good for only one user -- minimal reuseStatic file cache --HTML, GIF, JPEG, CSS, JavaScript for example. These are cached via several means -- browsers, web servers, proxy servers, WAS plugin, or WAS externally as remote cache adapters Dynamic cache -- saving some runtime result (either ar the presentation or business logic tier). Technology developed for massive highly dynamic distributed sitesServlet/JSP result caching -- page fragment or full page responses from Servletor JSP execution cached as HTMLEdge Side Includes (ESI) -- a simple markup language for dynamic page assembly at the edge, included in WAS V5 pluginWeb Service Call results -- SOAP results from web services call are cached thereby saving expensive marshalling and XML parsingCommand cache -- cached results from execution of some complex or intensive command -- resulting object/collection cachedJava Object cache -- specific objects/collections directly cachedJNDI cache -- all or part of WAS JNDI namespace can be cached on startup. Allows customer code for caching of JNDI lookups to be removed to simplify code, or to improve perfromance if you did not follow that practiceEJB caching -- the EJB container has several options available for caching stateful session bean homes and entity bean instances between transactions. CMP beans are cached and managed by the WAS Persistence Manager -- in WAS V6 stateful session beans and entity bean persistence manager cache now replicated

10

IBM Software Group


Follow the Query Flow into DB2

MAXAPPLSMAXAGENTSMAX_COORDGENTSMAX_POOLAGENTS

Listener

SA

appl heapsort heapstmt heapstat heap

agent stackquery heap

CA

db2tcpcmdb2ipccm

Obtaintoken

from DBM

MAX_CONNECTIONS

LA

Dispatcher

Agent Pool

ApplheapszAppgroup_mem_szGroupheap_ratioAppl_ctl_heap_szSortheapStmtheapStat_heap_szAgent_stack_szQuery_heap_szdbheap

Aguire memory +token

Client

catalog cache

package cachecatalogcache_szpkcachesz

DMS

RDS

Table/Index Manager

locklistMAXLOCKS

MINCOMMITlogbufsz

DPS Acquirelogging

BPM Sync/Async I/O ManagerSort

over flow

Temp Index Data Log

Operation System Services (OSS)

Buffer Pool

Prefetcherqueue

Sort

sortheapsheapthres_shrsheapthres

CHNGPGS_THRESHNUM_IOSERVERSNUM_IOCLEANERSSOFTMAX

DPS Acquirelock

MAXCAGENTS

LEGENDLogical Agent (Application)Coordinator AgentSubagentRelational Data ServicesData Management ServicesData Protection ServicesBuffer Pool Manager

LACASARDSDMSDPSBPM

locklist

logbuffer

THIS FOIL SHOWS CONNECTION CONCENTRATOR ENABLED. (It is usually disable when supporting appl servers.)Every transaction or query begins with a connect to the database, followed by one or more SQL statements, and ending with a commit and disconnect from the database.

The following series of interactions occur in DB2 as described during the execution of a typical transaction with connection concentrator enabled:When the application issues a connect to a database, the appropriate DB2 listener task (db2tcpcm, or db2ipccm) creates a logical agent which is a representation of a client connection to the database, if the creation does not exceed the limit specified by the max_connections database manager configuration parameter.

If the max_connections limit would be exceeded by the creation of this logical agent, then an error code SQL1226N is returned to the application.

When the first SQL call is issued by the application, the following occurs:The dispatcher (db2disp) associates a coordinator agent (if one is available) with the logical agent created and its application group applications always belong to a particular application group, which is the logical representation of a set of shared resources (shared that is between applications within that application group). The coordinator agent process will then serve requests on behalf of the application for the duration of a single transaction.The coordinator agent may be created if

f th t l t b d i th t th ti f

11

IBM Software Group


Monitoring and Diagnostic Best Practice

12

IBM Software Group


Top 10 Monitoring List for WebSphere

Servlets and EJBs1. Average Response Time

2. Number of Requests

3. Concurrent Requests

4. Web Server Threads

5. Web Container Thread Pool 6. Data Source Connection Pool

7. Java Virtual Machine Memory

HTTP

JDBC

IIOP

Web Server Web Container EJB Container Data Source

8. CPU 9. I/O10. Paging

DB

ORB



HTTP

Clients

Network

5. ORB Thread Pool

Because of the sheer magnitude of monitors and tuning parameters, knowing where to start, what to monitor and which component to tune first is hard. If you experience performance problems, use TPV and walk through the top ten monitoring items checklist

Consider the following:If you recognize something out of the ordinary, for example, an over-utilized thread pool or a JVM that spends 50% in garbage collection at peak load time, then concentrate your tuning actions there firstPerform your examination when the system is under typical production level load and make note of the observed valuesAlternatively, save TPV sessions to the file system, where the monitoring data will be stored for recurring analysisKeep in mind that one set of tuning parameters for one application will not work the same way for anotherUse the Performance Advisors together with TPV

13

IBM Software Group


PMI Data

WebSphere Command

Beans

TradeDatabase

MessageServer

EJB Container

TradeSession

EJB

Entity 2.0 EJBs

QuoteCMP

AccountProfileCMP

AccountCMP

OrderCMP

HoldingsCMP

Pub/Sub

Web Container

Trad

eAct

ion

Trad

eSer

vice

s

TradeDirectJDBC

AuditorClient

Message EJBsTradeBroker

MDBStreamer

MDB

Topic

Queue

Web ServicesClient

TradeServletsTrade

Servlets

WebClient

JAX/RPC SOAPServlet

Web Services

TradeServletsTrade

JSPs

Joein action!

Web Applications Servlet/JSP response time Servlet/JSP requests

Servlet Session Manager Avg # of active HTTP sessions Session size

EJB Module EJB response time EJB requests

JDBC Connection Pools % pool in use Avg. wait time JDBC times

JVM Memory Free memory # of garbage collection calls (JVMPI)

Thread PoolsWeb container and ORB Active threads % pool maxed

Java Transactions # active transactions Avg duration

J2C Connection Pools # of managed connections Total # of connectionsObject Request Broker (ORB) Object reference lookup time # of requests

Web Services Gateway # of sync requests/responses # of async requests/responses

System Data CPU Utilization Memory usage

Workload Management # of requests # of updates

Dynamic Cache Memory cache size # of hits/misses

Web Services # of loaded web services # of requests delivered/processed

JVMIMS, CICS

1. There is a lot of data and we continue to create new data points2. The need for tools/advisors become more important/relevant3. Perf data is collected from the JDK, WAS runtime & appls

The Performance Monitoring Infrastructure (PMI) uses a client-server architecture. The server collects performance data from various WebSphere Application Server components. A client retrieves performance data from one or more servers and processes the data. (http://publib.boulder.ibm.com/infocenter/ws51help/topic/com.ibm.websphere.base.doc/info/aes/ae/cprf_requestmetrics.html)

Performance Monitoring Infrastructure (PMI) provides server-side monitoring and a client-side API to retrieve performance data. PMI maintains statistical data within the entire WebSphere Application Server domain, including multiple nodes and servers. Each node can contain one or more WebSphere Application Servers. Each server organizes PMI data into modules and submodules.

14

IBM Software Group


WebSphere PMI Metrics of Interest

WebSphere Command

Beans

TradeDatabase

MessageServer

EJB Container

TradeSession

EJB

Entity 2.1 EJBs

QuoteCMP

AccountProfileCMP

AccountCMP

OrderCMP

HoldingsCMP

Pub/Sub

Web Container

Trad

eAct

ion

Trad

eSer

vice

s

TradeDirectJDBC

AuditorClient


MDBStreamer

MDB

Topic

Queue

Web ServicesClient

(JAX/RPC)

TradeServletsTrade

Servlets

WebClient

JAX/RPC SOAP

ServletWeb Services

TradeServletsTrade

JSPs

Joein action!

WebSphere Command

Beans

TradeDatabase

MessageServer

EJB Container

TradeSession

EJB

Entity 2.1 EJBs

QuoteCMP

AccountProfileCMP

AccountCMP

OrderCMP

HoldingsCMP

QuoteCMP

AccountProfileCMP

AccountCMP

OrderCMP

HoldingsCMP

Pub/Sub

Web Container

Trad

eAct

ion

Trad

eSer

vice

s

TradeDirectJDBC

AuditorClient


MDBStreamer

MDB

Topic

Queue

Web ServicesClient

(JAX/RPC)

TradeServletsTrade

Servlets

WebClient

JAX/RPC SOAP

ServletWeb Services

TradeServletsTrade

JSPs

Joein action!

Web Applications Servlet/JSP response time Servlet/JSP requests

Servlet Session Manager Avg # of active HTTP sessions Session size

EJB Module EJB response time EJB requests

JDBC Connection Pools % pool in use Avg. wait time JDBC times

JVM Memory Free memory # of garbage collection calls (JVMPI)

Thread Pools Web container and ORB Active threads % pool maxed

Java Transactions # active transactions Avg duration

J2C Connection Pools # of managed connections Total # of connections

Object Request Broker (ORB) Object reference lookup time # of requests

Web Services Gateway # of sync requests/responses # of async requests/responses

System Data CPU Utilization Memory usage

Workload Management # of requests # of updates

Dynamic Cache Memory cache size # of hits/misses

Web Services # of loaded web services # of requests delivered/processed

JVM

Available zOS 6.0Available as of 6.0Available zOS 5.0.2Available as of 5.0.2Available zOS 5.0.1Available as of 5.0

Available zOS 6.0Available as of 6.0Available zOS 5.0.2Available as of 5.0.2Available zOS 5.0.1Available as of 5.0

DCS # of messages sent/receivedTime spent synchronizing

HA Manager# of local groups# of bulletin-board subjects

For your reference a subset of the key ones and in which release they become availalble.

15

IBM Software Group


PMI Predefined Statistic Sets (New in V6)

None All statistics are disabled

Basic Statistics specified in J2EE 1.4, and top statistics like CPU usage and

live HTTP sessions are enabled Enabled by default; only 1-4% overhead Provides basic performance data about runtime and application

components Extended

Basic set plus key statistics from various components like WLM and Dynamic caching; only 1-4% overhead

All All statistics are enabled; dangerous could have 35% overhead

Use the Custom setting to define your own statistic set Enable or disable statistics individually

16

IBM Software Group


Monitor WebSphere with TPV Tivoli Performance Viewer (TPV) externalizes PMI metrics

and is part of WebSphere

This chart shows a view from the Tivoli Performance Viewer whichdisplays the PMI data- this tool ships as part of WebSphere Application Server. For those of you who used WAS 4.0, remember that TivoliPerformance Viewer is the new name for Resource Analyzer. You can view the data in tabular form or in graphical form. On the graph you can also select which counters you want displayed. This particular view is showing that there is a large number of discards from the prepared statement cache. If the cache were set to a larger size, you should see the discard rate reduced and some performance gains as a result.

17

IBM Software Group


Performance Advisor in TPV

For details, go to http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=/com.ibm.websphere.express.doc/info/exp/ae/tprf_enablingtpvadvisor.html

For a demo, go to http://publib.boulder.ibm.com/infocenter/ieduasst/v1r1m0/topic/com.ibm.iea.was_v6/was/6.0/Performance/WASv6_TPVPerformanceAdvisor.viewlet/WASv6_TPVPerformanceAdvisor_viewlet_swf.html?dmuid=20061227195912256394

This is a screen shot from the Performance Advisor in Tivoli performance viewer. The section at the top is a blown up version from the screen shot underneath. The advisor was run against an internal application, and as you can see we received advice on how to set the configuration for many of our Websphere parameters. Note the highlighted message when you could be click on that msg.

[This particular customer doesnt have any simple config problems; all of his warnings are of a 2a type]

Tuning strategyRun the performance advisorsDo what they suggest

18

IBM Software Group


Performance Advisor in TPV

You get more details. This message specifically tells you that the rate of discards from the preapred statement cach is high and suggests to the user that the size of the cache be increased.

19

IBM Software Group


DB2 Snapshot Monitoring

Snapshots Most common DB2 performance diagnostic

Point-in-time data on database activity

Usually cumulative

Small volume of data produced

Often practical for ongoing monitoring

Which ones are useful?

20

IBM Software Group


S

Tables

A

A

Tablespaces

A

A

A

A

A

Mem

ory pools

S

S

S

CPU

utilization

S

Lock detail

A

A

A

A

SQL stm

t activity

A

A

A

Log usage

P

P

Dynam

ic SQL

A

A

A

A

A

Pkg/Sect/Cat

cache

A

A

A

A

Application

state

A

A

A

A

A

Row

s read/selected

A

A

A

A

Agents

S

S

S

S

S

S

S

S

Bufferpool &

I/O

PPdatabase

get snapshot for

dynamic sqllocks

tablespacestables

applicationsbufferpools

all

all

P

PP

PP

on

bufferpoolsPPapplicationsPPdatabasesPdatabase manager

Sorts

Lock summ

ary

Overhead: 3-5% overhead for most settings, 10% for all including SQL snapshot, more if system is stressed (A always collected S collected only when monitor switch is ON P collected when switch is ON, partially collected when switch is OFF. )

What you can get

how to get it

and some prettyuseful

things

Monitor DB2 with DB2 Snapshot

This chart gives us a summary of what we can get (across the top) and how to get it (down the left side.)

An A in the chart means the indicated information is always collected, regardless of the state of monitor switches.P means the data is partially collected (but to get all the information, you need to have the appropriate switch turned on)S means that the data is only available if the switch is on.

The green ellipses indicate especially useful areas, and are what were going to focus on in the following slides.

21

IBM Software Group


DB2SystemMonitor Object (new for WAS v6.0)

The com.ibm.db2.jcc.DB2SystemMonitor interface is used for collecting system monitoring data for a connection

WebSphere application generates monitor database activities: DB system monitor data is broken up into the following four

areas: ServerTimeMicro time spent in the DB2 server NetworkIOTimeMicros time spent in network (send/receive), and

DB2 server CoreDriverTimeMicros time spent in driver, network, and DB2

server ApplicationTimeMillis time spent in application, driver, network,

and DB2 server Back end independent.

See http://publib.boulder.ibm.com/infocenter/db2luw/v8/index.jsp?topic=/com.ibm.db2.udb.doc/ad/rjvjdb2o.htm

22

IBM Software Group


Provide WebSphere Client Information to DB2 For WAS 5, set client id, workstation name, appl name, and accounting

string as custom properties of datasource sent at first physical connect.

db2 get snapshot for applications on .

TP Monitor Client User ID:WebSphere5-Userid

TP Monitor Client Workstation Name: WebSphere5-Wrkstn TP Monitor Client Application Name: WebSphere5-Appl TP Monitor Client Accounting String:

WebSphere5-Acct

23

IBM Software Group


Provide WebSphere Client Information to DB2

DB2 Connection annotation (WAS v6.0) Pass client info to DB2 on each application connection

Can pass client info explicitly within application By calling an IBM proprietary API, setClientInformation(Properties), on

the com.ibm.websphere.rsadapter.WSConnection object

More likely usage is to pass client info implicitly by setting The WebSphere Trace group WAS.clientinfo=all=enabled Via the Admin Console: Troubleshooting > Logs and Trace > pick the

server > Change log detail level > Configuration or Run time > Trace Specification

See http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=/com.ibm.websphere.express.doc/info/exp/ae/rdat_impclientinfo.html

24

IBM Software Group


Provide WebSphere Client Information to DB2

DB2 Connection annotation (WAS v6.0)

For debugging purpose,

Set the WebSphere trace group WAS.clientinfopluslogging

This passes client info implicitly to DB2 and to log and trace important application activities in WebSphere: SQL Strings that are run (such as, select userId from tabl1

where id=? for update). Start, commit, and rollback of transactions. EJB calls (such as, Create, Remove, findByPrimaryKey, and so

on)

See http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=/com.ibm.websphere.express.doc/info/exp/ae/rdat_impclientinfo.html

25

IBM Software Group


Request Flow, Error Logs and traces Web Server Logs (1)

/logs/access.logs, /logs/error.log

Plug-in Logs (1) Native.log

Application Server Logs (2+3) SystemOut.log, SystemErr.log

Components (2+3) WebSphere Diagnostic Trace

DB2 JDBC Driver (4) JDBC, CLI, or universal driver trace

DB2 Server db2diag.log instance.nfy or Windows event log

DB2 trace

browser

Java App

HTTP Server

1a

1b

1c

1b+

Data Provider

EJB Cont-ainer

Application Server

WebCont-ainer

JDBC Drivers

2

3

3a

4

DB2

When there are performance or functional problems, it helps to check the various error logs and traces.

Usually start with the WebSphere SystemOut.log and the DB2 db2diag.log to see if there are any errors.

The WebSphere Diagnostic Trace is the equivalent of DB2 Trace for WebSphere. It can filter by runtime component, package name, or on message type: entry/exit, event, info, debug. The output is sent to circular in-memory buffer or trace file.

In addition, it can also control tracing of application components. Application must make use of WebSphere's JRAS implementation either directly or through Jakarta Commons Logging mechanism. The trace can filter by application package name. The benefit is that the control of application logging is then an extension of controlling the runtime logging.

26

IBM Software Group


Log files Interpretation

By looking at the log files, you can always reduce down to at least where the problems are!!

Where is the problem? or Where is NOT the problem? When in doubt, start with the WebSphere SystemOut.log and

the DB2 db2diag.log Usually reports the error in stack trace or Exception format

Good to trace back where the problem FIRST occurred WebSphere log can capture DB2 JDBC driver trace WebSphere traces includes DB2 Correlater to correlate db2diag log

transaction entries with WAS traces

27

IBM Software Group


Enabling DB2 JDBC JCC Trace from WebSphere

Go to Resources > JDBC Provider > Data Sources > Additional Properties > Custom Properties:

Specify traceLevel -1 to TRACE_ALL

Leave the traceFile property blank to embed the JCC trace points inside the WebSphere trace automatically.

The trace setting is enabled or disabled dynamically the next time the datasource is used, which should help when trying to narrow down a problem

Go to Troubleshooting > Logs and Trace > pick the server > Diagnostic Trace > Trace Specification: Specify RRA=all=enabled:WAS.database=all=enabled

Notice here that you specify two trace strings separated by ':', one for the WebSphere Application Server resource adaptor, and one for the database (JDBC driver) since

See the following articles for more details on DB2 JDBC drivers and tracing http://www.ibm.com/developerworks/db2/library/techarticle/dm-0512kokkat/ http://www.ibm.com/developerworks/db2/library/techarticle/dm-0506fechner/

28

IBM Software Group


Trace Correlator ExampleWebSphere Trace Correlator for DB2

DB2 jcc trace

29

IBM Software Group


DB2 server side main log (db2diag.log):2003-05-06-22.17.44.791070 Instance: db2inst1 Node: 000PID: 2706 (db2agent (ICMNLSDB)) TID:8192 Appid: NF00001.G6D7.00F59AE79B0Faccess plan manager sqlra_sqlC_dump Probe: 25 Database: ICMNLSDB

DB2 Trace (server side)3571 mbt_scb DB2 common communication sqlccgetapplid cei (3.3.43.10.2.1)

pid 1188018 tid 1 cpid -1 node 0 sec 0 nsec 16431127 probe 10marker name: PD_SQLT_MARK_APPIDDescription: Correlator identifier (TCP/IP connection, JDBC type 4) bytes 26appID: NF00001.G6D7.00F59AE79B0F

Universal JDBC driver trace (client side)[ibm][db2][jcc][time:1050540951783][thread:main][Connection@8385e3] Database product

version: SQL08012[ibm][db2][jcc][time:1050540951783][thread:main][Connection@8385e3] Driver name: IBM DB2

JDBC Universal Driver [ibm][db2][jcc][time:1050540951783][thread:main][Connection@8385e3] Driver version: 1.3.7 Test

Build[ibm][db2][jcc][time:1050540951783][thread:main][Connection@8385e3] DB2 Correlator:

NF00001.G6D7.00F59AE79B0F[ibm][db2][jcc][time:1050540951783][thread:main][Connection@8385e3] END TRACE_CONNECTS

30

IBM Software Group


Log Trace Analyzer Eclipse based tool included as part of the AC Tool kit in DeveloperWorks

http://www.ibm.com/developerworks/autonomic/overview.html

Import logs from Apache HTTP Server, IBM HTTP Server, WAS Appl Server, DB2 Server

Correlate logs based on correlation id (WAS, JCC, DB2) or time,

Log and trace analyzer is included as part of theAC Tool kit in http://www-128.ibm.com/developerworks/autonomic/overview.html?Open&ca=daw-prod-logandtrace

31

IBM Software Group


Configuration

Best Practice

32

IBM Software Group


General For WebSphere

Do NOT use DriverManager when developing Websphereapplications

use DataSource, JNDI lookups, and datasource reference Use connection pool (default) Use PreparedStatement cache to minimize the need to re-

prepare SQL statements internally For DB2

Use jcc type 4 driver for remote access, and jcc type 2 driver for local access

Use Config Advisor to do your initial configuration Use sufficient DB2 global dynamic cache to minimize the

need to re-prepare SQL statements internally Use STMM (DB2 9) to manage memory allocation (e.g. global

dynamic cache)

Connection object cachingds.getConnection() returns a connection object from the WAS connection object pool or creates a new object (up to WAS max)

PreparedStatement object cachingcon.PrepareStatement() returns an already prepared statement object (no DB2 flow) from WAS statement cache whenever possible.If not found in cache, statement will be prepared(DB2 prepare flow) an kept in the cache (up to WAS max)

EJB CachingCaching data within the EJB to avoid trips to the backend database. (Timer validation)

33

IBM Software Group


WebSphere Queuing Network WebSphere establishes a queuing network Includes the network, Web server, Web container, EJB

container, and data source

Represents a queue of requests waiting to use that resource

WebSphere Application Server

MaxClients for UNIXThreadsPerChild for Windows

Browser

HTTPServer

(MaxClients=75)

Web Container

(MaxThreads=50)

EJBContainer

(MaxThreads=50)

Data Source(MaxCon=25) Database

EJB Client

100Requests

75Requests

50Requests 25

Net

wor

k

Web Server

Connection pool

ORB thread pool maximum size

Thread pool maximum sizeHTTP transport channel: Maximum persistent requests

WebSphere Application Server establishes a queuing network, which is a group of interconnected queues that represent various components. There are queues established for the network, Web server, Web container, EJB container, ObjectRequest Broker (ORB), data source, and possibly a connection manager to a custom back-end system. Each of these resources represents a queue of requests waiting to use that resource. Queues are load-dependent resources. As such, the average response time of a request depends on the number of concurrent clients.As an example, think of an application, consisting of servlets and EJBs, that accesses a back-end database. Each of these application elements reside in the appropriate WebSphere component (for example servlets in the Web container) and each component can handle a certain number of requests in a given time frame.A client request enters the Web server and travels through WebSphere components in order to provide a response to the client.

34

IBM Software Group


WebSphere Queuing Network and DB2 Agents Queues are used in each layer of a WAS environment Aggressive queue size can lead to resource saturation.

Slow response time in WAS or DB2 could drive even more connections into DB2 and make the whole system even slower. Set reasonable limits to WAS connection pool (MaxCon) and DB2 agents (maxagent)

For WebSphere:HTTP Servers MaxClient

> App Servers MaxThread> Data Sources MaxCon

For DB2:maxagent > sum of MaxCon from all WAS nodesmaxappl >= 1.2 to 2x maxagent (to account for indoubts)num_poolagents > sum of MinCon from all WAS nodes

In general do not enable concentrator unless the WebSphere nodes have many idle connections and DB2 is memory bound

Use WebSpheres TPV and DB2s Get Snapshot for Database to monitor and adjust

MAXAGENTSThe maximum number of physical DB2 server agents Set this high enough to handle maximum of all WAS connections

MAXAPPLSThe maximum number of concurrent applications that can be connected to a database

NUM_INITAGENTNumber of agents started when DB2 instance is started

NUM_POOLAGENTSSpecifies maximum size of the server agent pool

35

IBM Software Group


Database Connection Pooling Use the Type 4 (pure Java) JDBC driver, if possible

Type 4 Universal JDBC Drivers (JCC) 10-30% faster than the legacy/CLI JDBC driver

Type 4 JDBC drivers generally perform faster than Type 2 JDBC drivers. Type 4 drivers best suited for client/ server environment. Type 2 for local access.

Connection pool tuning Min 1, Max 30 good starting point for tuning

Poor database or network performance often indicated by: Many used connections Many threads waiting for the database connection pool

Poor overall response time

Prepared statement cache Monitor via Tivoli Performance Viewer Increase pool size if you see cache evictions

36

IBM Software Group


XA Transaction Optimization

Dont use XA datasources if you dont need to There is some overhead over non-XA datasources even if transactions

only use a single datasource since it invokes an extra request for transaction start and end

DB2 optimization for XA For legacy JDBC driver (and CLI and embedded SQL),

set DB2 config parm:

DB2 UPDATE DBM CFG USING TP_MON_NAME=WAS

For jcc driver, the optimization is built-in

37

IBM Software Group


JMS

A message-driven bean (MDB) is an enterprise bean that allows J2EE applications to process messages asynchronously.

Acts as a JMS message listener, The messages may be sent by any J2EE component--an application client, another enterprise bean, or a Web component--or by a JMS application or system that does not use J2EE technology.

MDB Performance tips Change the default Cloudscape database to a remote DB2 database for

better performance

Expect up to 50% higher throughput with remote DB2 compared to local DB2

Tune the default thread pool

Deliver batches of messages to each MDB endpoint to improve performance

Adjust the JDBC connection pool size and preparedStatementCache size

38

IBM Software Group


JMS

Data store Performance tuning Sufficiently increase the JDBC connection pool to cope with peak

loads

Place database logs on a fast RAID array to reduce the write latency

Consider putting the SIBxxx tables into a tablespace with 32KB pages

Adjust the column width of the VARCHAR column to 32032 bytes.

Use One-Phase Commit optimization for XA: On WAS, when using CMP with both DB2 datasource and JMS, avoid

2-phase commit by using the same DB2 datasource for JMS data Choose DB2 instead of FileStore (default in WAS 6.1) for JMS data, and

then choose 1-phase JMS optimization

39

IBM Software Group


Session Database Application-related

Enable session persistence (java.io.Serializable)

Reduce size of the session object

Invalidate unneeded sessions

System-related Use larger DB2 page size

Tune session cache size

Create additional application server instances

Increase available memory

Tune session timeout interval

Consider using multi-row session in database persistence

Tune session database connection pool

Tune the database as usual

40

IBM Software Group


WebSphere Transaction Time-Out WebSphere transaction time-out

The intention is to time-out cases where WebSphere hangs or loops Does not time-out SQL query Only time-out a transaction after a query returns

Several alternatives to time-out long running query Use DB2 Governor to force transaction that exceeds CPU or elapse

time limit In the JDBC (non-EJB) application, setQueryTimeout on the query In the datasource definition, set custom property

blockingReadConnectionTimeout (DB2 FP12).

41

IBM Software Group


DB2 Client Side Time-Out DB2TCP_CLIENT_CONTIMEOUT and

DB2TCP_CLIENT_RCVTIMEOUT registry variables Used by non-jcc driver (e.g. embedded SQL, CLI, and legacy

.app JDBC driver) to time-out a connection if the server is down or overloaded.

For jcc driver, the equivalent properties are loginTimeout (FP9) and blockingReadConnectionTimeout (FP12).

On WebSphere, these two can be setup as custom properties for the datasource.

See details of DB2 registry variables at technote

http://www.ibm.com/support/docview.wss?uid=swg21231084

Use of DB2 Governor To force transaction that exceeds CPU or elapse time limit

42

IBM Software Group


DB2 Server Side Transaction Time-Out

DB2CHECKCLIENTINTERVAL registry variable Check for client terminating the connection before returning result

(e.g. a user kills an SQL application) Does not help if client machine terminates abruptly where TCP

cannot respond; OS keepalive value needed to handle this situation

See TCP/IP keepalive settings and related DB2 registry variables at technote

http://www.ibm.com/support/docview.wss?uid=swg21231084

43

IBM Software Group


Application

Best Practice

44

IBM Software Group


JDBC/SQLJ Best Practices

Use setFetchSize() to optimize SELECT performance Use Batch instead of individual UPDATE/INSERT statements Use stored procedure to minimize even more network flow Turn off auto-commit Fetch Data into their own primitive data types

e.g. getTimestamp 3x faster than getString if fetching Timestamp

Use reusable statements whenever possible; use parameter markersand avoid literals

SQLJ for static complex SQL statements to avoid prepare time

Use the following Statement methods for creating, executing, and removing a batch of SQL updates: addBatchexecuteBatchclearBatch

45

IBM Software Group


EJB Best Practices EJBs represent managed access to DB resource Sets the size and policies of its EJB cache to avoid unnecessary

SQL requests

Avoid applying higher level of access intent than necessary on EJBs(e.g. Set read-only access intent for read-only access!!)

Define index on primary key for tables where its Entity Beans mapping has a findByPrimaryKey method

Use optimistic concurrency read-ahead hints for CMR fields Other enhancement to minimize number of request flow to DB2:

Use batch updates Use Deferred create Use Partial Update Invoked as JVM option:

Dcom.ibm.ws.pm.batch=true Dcom.ibm.ws.pm.deferredcreate=true Dcom.ibm.ws.pm.grouppartialupdate=true

46

IBM Software Group


Understanding the JDBC vs DB2 Isolation Levels

TRANSACTION_SERIALIZABLE isolation level is the highest, most restrictive, isolation level. It prohibits dirty reads, non-repeatable reads, and phantom reads.

TRANSACTION_REPEATABLE_READ isolation level is the second highest isolation level. It prohibits dirty reads and non-repeatable reads, but allows phantom reads. Dont confuse this with the DB2 RR.

TRANSACTION_READ_COMMITTED isolation level prohibits dirty reads only. TRANSACTION_READ_UNCOMMITTED isolation level permits dirty reads, non-

repeatable reads, and phantom reads.

WebSphere changed default from CS (WAS 4) to RS (WAS 5, 6). You can override the default with a custom property webSphereDefaultIsolationLevel for non-CMP appls. Other methods are via resource reference or access intent.

Uncommitted Read (UR)TRANSACTION_READ_UNCOMMITTED

Cursor Stability (CS) TRANSACTION_READ_COMMITTED

Read Stability (RS)TRANSACTION_REPEATABLE_READ

Repeatable Read (RR)TRANSACTION_SERIALIZABLE

DB2 Isolation LevelJDBC Isolation Level

See http://www-1.ibm.com/support/docview.wss?rs=180&uid=swg21224492 on how to change the default isolation level with difference methods.

47

IBM Software Group


Whats Coming for DB2/WebSphere

Integration

48

IBM Software Group


Heterogeneous DB2 pooling

Heterogeneous pooling is the ability to share one connection pool among multiple similar WebSphere data sources Core properties will have to be identical (e.g. username, host,

port, ..)

Can, significantly, reduce database overhead by reducing # of physical connections to the database.

Additional optimization will also eliminate 2-phase commit to the use of these datasources within the same transaction when the same connection is being used The Get use close pattern must be used.

49

IBM Software Group


DB2 Trusted Context

Identity propagation which is useful for accountability. Doesnt require creating and destroying of connection (hence, good

performance)

Better security (fine grained security) - no longer need to give one user access to all tables in the database.

50

IBM Software Group


DB2 SQLj usability enhancements in WebSphere

Allow for SQLj profiles to be created, customized and bound during SQLj application install

Allow for SQLj profiles (.ser files) to be modified for existing installed applications in WebSphere from the WebSphere admin console/WSAdmin.

51

IBM Software Group


Resources

52

IBM Software Group


Additional Resources Sign up to receive weekly technical My support emails

www.ibm.com/software/support/einfo.html

WebSphere Application Server Performance Web Site http://www.ibm.com/software/webservers/appserv/performance.html Performance guides includes papers describing how to maximize performance

Redbooks http://www.redbooks.ibm.com Best Practices for High-Volume Web Sites DB2 UDB/WebSphere Performance Tuning Guide (and ITSO course) IBM WebSphere V6.0 Performance, Scalability, and High Availability WebSphere

Handbook (and ITSO course) WebSphere Version 6 Application Development Handbook

WebSphere Application Server InfoCenter http://www.ibm.com/software/webservers/appserv/infocenter.html WebSphere Application Server V6 Tuning Guide

53

53

Peter ShumIBM Toronto Laboratory

[email protected]

Session F13DB2 WebSphere Integration Best Practice

DB2_WebSphere_BestPractice

Documents

java platform

java web

j2se java

ibm operatesibm

java technology

standard javabased interface

j2ee application

jndi java naming