Top Banner
ibm.com/redbooks Redpaper Managing an SOA Environment with Tivoli Budi Darmawan Pradeep Nambiar Prem Lall Ravinder Gummadavelli Discusses SOA performance and availability management Describes mediation scenarios with ESB and message broker Integrates ITCAM and other Tivoli solutions
228
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
  • 1. Front coverManaging an SOAEnvironment withTivoliDiscusses SOA performance andavailability managementDescribes mediation scenarioswith ESB and message brokerIntegrates ITCAM andother Tivoli solutions Budi DarmawanPradeep Nambiar Prem LallRavinder Gummadavelliibm.com/redbooks Redpaper

2. International Technical Support OrganizationManaging an SOA Environment with TivoliApril 2008 REDP-4318-00 3. Note: Before using this information and the product it supports, read the information in Notices on page vii.First Edition (April 2008)This edition applies to: IBM Tivoli Composite Application Manager for WebSphere V6.1, 5724-L62 IBM Tivoli Composite Application Manager for Web Resources V6.2, 5724-S32 IBM Tivoli Composite Application Manager for Response Time Tracking V6.1 FP2, 5724-L99 IBM Tivoli Composite Application Manager for Response Time V6.2, 5724-C04 IBM Tivoli Composite Application Manager for SOA V6.1 FP2, 5724-M07 IBM Tivoli OMEGAMON XE for Messaging V6.1 WebSphere Enterprise Service Bus and WebSphere Process Server V6.1 WebSphere Message Broker V6.0 WebSphere Services Registry and Repository V6.0.2 Copyright International Business Machines Corporation 2008. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADPSchedule Contract with IBM Corp. 4. Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix The team that wrote this paper. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Chapter 1. Introduction to SOA management. . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Introduction to SOA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 SOA application principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 SOA constructs and components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 SOA governance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.1 Web Services life cycle governance . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4.2 Web Services life cycle management . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.5 SOA management considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.6 Users of SOA management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.6.1 Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.6.2 Middleware or application subject matter expert . . . . . . . . . . . . . . . . 111.6.3 Performance analyst . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.6.4 System administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.6.5 Enterprise system management architect . . . . . . . . . . . . . . . . . . . . . 141.6.6 Web Services application developer . . . . . . . . . . . . . . . . . . . . . . . . . 151.6.7 Business executives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.7 Management needs for the SOA environment . . . . . . . . . . . . . . . . . . . . . 161.7.1 Web Services metric data collection . . . . . . . . . . . . . . . . . . . . . . . . . 161.7.2 Web Services troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.7.3 Displaying data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.7.4 Mediation management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Chapter 2. Tivoli application management products . . . . . . . . . . . . . . . . . 19 2.1 ITCAM for SOA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.1.1 Product features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.1.2 Product components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.2 ITCAM for WebSphere and ITCAM for J2EE . . . . . . . . . . . . . . . . . . . . . . 262.2.1 Architecture and interconnection. . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.2.2 The managing server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.2.3 J2EE and WebSphere data collectors . . . . . . . . . . . . . . . . . . . . . . . 302.2.4 Tivoli Enterprise Monitoring Agent . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Copyright IBM Corp. 2008. All rights reserved.iii 5. 2.3 ITCAM for Response Time Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.3.1 The management server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.3.2 Store and forward agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.3.3 Management agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.3.4 Tivoli Enterprise Monitoring Agent . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.4 OMEGAMON XE for Messaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.4.1 WebSphere MQ configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422.4.2 WebSphere MQ monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.4.3 WebSphere Message Broker monitoring . . . . . . . . . . . . . . . . . . . . . 46 Chapter 3. Basic SOA and Web Services management. . . . . . . . . . . . . . . 49 3.1 Basic monitoring concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2 Performance metric of Web Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.3 Generating events and alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.4 Managing Web Services response time . . . . . . . . . . . . . . . . . . . . . . . . . . 553.4.1 Execution environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563.4.2 Creating Rational Performance Tester script . . . . . . . . . . . . . . . . . . 563.4.3 Defining Web Response Monitor policies . . . . . . . . . . . . . . . . . . . . . 593.4.4 Reports generated from the policies . . . . . . . . . . . . . . . . . . . . . . . . . 633.4.5 Tivoli Enterprise Portal workspaces . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.5 Debugging performance of Web Services. . . . . . . . . . . . . . . . . . . . . . . . . 69 3.6 Understanding Web Services calling pattern . . . . . . . . . . . . . . . . . . . . . . 743.6.1 Turning on the content logging for a Web Service . . . . . . . . . . . . . . 743.6.2 Using the Log Assembler tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.7 Working with Web Services filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 3.8 Web Services life cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Chapter 4. Advanced SOA management . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 4.1 Mediation and SOA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.2 Enterprise Service Bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 4.3 Maintaining Web Services continuity. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1044.3.1 Register TraderDBServices in the registry . . . . . . . . . . . . . . . . . . . 1064.3.2 Developing managed SCA mediation with ITCAM for SOA . . . . . . 1104.3.3 Deploying the mediation application . . . . . . . . . . . . . . . . . . . . . . . . 1264.3.4 Verifying the service invocation with mediation. . . . . . . . . . . . . . . . 132 4.4 Service monitoring automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1364.4.1 Automation principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1374.4.2 Update service metadata utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1384.4.3 ITCAM for WebSphere and ITCAM for Web Resources situations. 1394.4.4 ITCAM for SOA situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1474.4.5 Verifying situation automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 4.5 Using managed message logger. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1554.5.1 Viewing the message data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156iv Managing an SOA Environment with Tivoli 6. 4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158Chapter 5. Managing an SOA application in a business context . . . . . . 1595.1 Solution overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1605.2 Tivoli EIF probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1615.3 Defining situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1625.4 Designing business services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1645.5 Defining service level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1745.6 Getting the business status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175Appendix A. The Trader application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179Application components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 Portal interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Front-end J2EE Web application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Java desktop application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Back-end implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Back-end J2EE servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 WebSphere Enterprise Service Bus mediation . . . . . . . . . . . . . . . . . . . . . 190 WebSphere Message Broker mediation . . . . . . . . . . . . . . . . . . . . . . . . . . 191Software requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Runtime environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Development environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195Appendix B. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 System requirements for downloading the Web material . . . . . . . . . . . . . 198 How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205How to get IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209Contents v 7. vi Managing an SOA Environment with Tivoli 8. NoticesThis information was developed for products and services offered in the U.S.A.IBM may not offer the products, services, or features discussed in this document in other countries. Consultyour local IBM representative for information on the products and services currently available in your area.Any reference to an IBM product, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product, program, or service thatdoes not infringe any IBM intellectual property right may be used instead. However, it is the usersresponsibility to evaluate and verify the operation of any non-IBM product, program, or service.IBM may have patents or pending patent applications covering subject matter described in this document.The furnishing of this document does not give you any license to these patents. You can send licenseinquiries, in writing, to:IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.The following paragraph does not apply to the United Kingdom or any other country where suchprovisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATIONPROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS ORIMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimerof express or implied warranties in certain transactions, therefore, this statement may not apply to you.This information could include technical inaccuracies or typographical errors. Changes are periodically madeto the information herein; these changes will be incorporated in new editions of the publication. IBM maymake improvements and/or changes in the product(s) and/or the program(s) described in this publication atany time without notice.Any references in this information to non-IBM Web sites are provided for convenience only and do not in anymanner serve as an endorsement of those Web sites. The materials at those Web sites are not part of thematerials for this IBM product and use of those Web sites is at your own risk.IBM may use or distribute any of the information you supply in any way it believes appropriate withoutincurring any obligation to you.Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products and cannot confirmthe accuracy of performance, compatibility or any other claims related to non-IBM products. Questions onthe capabilities of non-IBM products should be addressed to the suppliers of those products.This information contains examples of data and reports used in daily business operations. To illustrate themas completely as possible, the examples include the names of individuals, companies, brands, and products.All of these names are fictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.COPYRIGHT LICENSE:This information contains sample application programs in source language, which illustrate programmingtechniques on various operating platforms. You may copy, modify, and distribute these sample programs inany form without payment to IBM, for the purposes of developing, using, marketing or distributing applicationprograms conforming to the application programming interface for the operating platform for which thesample programs are written. These examples have not been thoroughly tested under all conditions. IBM,therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. Copyright IBM Corp. 2008. All rights reserved.vii 9. TrademarksThe following terms are trademarks of the International Business Machines Corporation in the United States,other countries, or both: Redbooks (logo)DB2 OS/400 z/OS ETE Rational Alerts IBM Redbooks AIXIMS RACF Cloudscape MQSeriesTivoli Enterprise Console CICS MVS Tivoli DataPowerOMEGAMONWebSphereThe following terms are trademarks of other companies:SAP NetWeaver, SAP, and SAP logos are trademarks or registered trademarks of SAP AG in Germany andin several other countries.Oracle, JD Edwards, PeopleSoft, Siebel, and TopLink are registered trademarks of Oracle Corporationand/or its affiliates.ITIL is a registered trademark, and a registered community trademark of the Office of GovernmentCommerce, and is registered in the U.S. Patent and Trademark Office.Enterprise JavaBeans, EJB, Java, JavaBeans, JDBC, JMX, JNI, JRE, JVM, J2EE, Solaris, Sun, Sun Java,and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, othercountries, or both.Internet Explorer, Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in theUnited States, other countries, or both.UNIX is a registered trademark of The Open Group in the United States and other countries.Linux is a trademark of Linus Torvalds in the United States, other countries, or both.Other company, product, or service names may be trademarks or service marks of others.viiiManaging an SOA Environment with Tivoli 10. Preface Service-oriented architecture (SOA) is a major new trend for application architecture. It allows you to build applications as components as defined by using a Web Services Description Language (WSDL) file. You can implement applications on multiple servers, even on multiple platforms. You can easily modify application components and workflow logic in execution by allowing a flexible application structure. The use of enterprise service bus (ESB) masks the implementation of the client side and the server side. ESB allows you to implement different servers without needing to modifying the client. Or, multiple clients can use the same server implementation. The highly flexible and distributed nature of SOA-based applications is its primary strength and the source of its appeal. However, when problems arise, this flexible nature also causes a greater challenge in pinpointing the source of a problem. SOA also requires a disciplined management effort to ensure that operational changes do not disrupt overall system availability. The IBM Tivoli Composite Application Manager (ITCAM) family of products is designed to assist you in managing distributed applications, including SOA-based applications. However, the overall management of a complete SOA management solution requires the use of several tools that work together. Each tool addresses a different aspect of the application. This paper illustrates the management needs for SOA-based applications and demonstrates how Tivoli products can address your application environment needs. The overall solution that we use includes ITCAM for SOA, ITCAM for WebSphere, ITCAM for Response Time Tracking, OMEGAMON XE for Messaging, and the Tivoli Business Service Manager solution to address various needs in SOA-based application management. The intended audience for this IBM Redpaper publication cis any services specialist who implements a performance management solution for an SOA-based environment.The team that wrote this paper This paper was produced by a team of specialists from around the world working at the International Technical Support Organization, Austin Center. Copyright IBM Corp. 2008. All rights reserved.ix 11. Budi Darmawan is a project leader at the International Technical Support Organization, Austin Center. He writes extensively and teaches IBM classes worldwide on all areas of Tivoli and systems management. Before joining the ITSO eight years ago, Budi worked in IBM Indonesia services as a technical lead and solution architect. His current interests are Java programming, availability management, business service management, and z/OS management. Pradeep Nambiar is a Worldwide Technical Evangelist in the IBM Tivoli Business Automation Sales Enablement group. He has over 19 years experience in the IT industry in various areas ranging from graphics systems, networked graphics, IBM Component Broker/WebSphere Application Server system management, business application architecture, design, and development. He is an IBM Certified SOA Solution Designer, IBM Certified WebSphere Enterprise Developer, and IBM Certified Solution Developer in XML and Related Technologies. His current focus is on application management and the automation family of products, including SOA management from IBM Tivoli. He is based in Austin, TX. Prem Lall is a Software Engineer currently assigned to the ITCAM for SOA project where he specializes in the field of Web Services management. He has had over 15 years experience in the IT field. During his 11 years at IBM, he has helped design and implement a variety of software products. He has expertise in front-end, middleware, and back-end development with an emphasis on e-commerce. Among other things, he created end-to-end online banking solutions for IBM clients in the Integrion consortium, he has been part of the WebSphere Application Server development team, and helped create an extensive SOA-based e-File application for the IRS that is currently used by numerous businesses across the country. He holds a Masters of Science Degree in Pure and Applied Mathematics from California State University, Northridge, CA. He also worked as an Actuary, and he has worked in the Atmospheric Physics Division of the NASA Goddard Space Flight Center. Ravinder Gummadavelli is a Software Engineer with IBM Systems Technology Group, in the USA. He has over 10 years of experience in the IT Systems Design and Development field. He holds a Masters in Technology degree in Electrical Engineering from REC, Warangal, India, and a Masters of Science degree in Electrical Engineering from Auburn University, Auburn, AL. His areas of expertise include Systems Design, Development, and SOA. His current interests include SOA and IBM Virtualization offerings. Thanks to the following people for their contributions to this project: Karen Durston, Mark Anderson, Jayne Regan IBM Software Groupx Managing an SOA Environment with Tivoli 12. Become a published authorJoin us for a two- to six-week residency program! Help write a book dealing withspecific products or solutions, while getting hands-on experience withleading-edge technologies. You will have the opportunity to team with IBMtechnical professionals, IBM Business Partners, and Clients.Your efforts will help increase product acceptance and customer satisfaction. Asa bonus, you will develop a network of contacts in IBM development labs andincrease your productivity and marketability.Find out more about the residency program, browse the residency index, andapply online at:ibm.com/redbooks/residencies.htmlComments welcomeYour comments are important to us!We want our papers to be as helpful as possible. Send us your comments aboutthis paper or other IBM Redbooks publications in one of the following ways: Use the online Contact us review IBM Redbooks publications form found at: ibm.com/redbooks Send your comments in an e-mail to: [email protected] Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400Preface xi 13. xii Managing an SOA Environment with Tivoli 14. 1Chapter 1. Introduction to SOA management In this chapter, we explain service-oriented architecture (SOA) and walk you through managing an SOA environment. We divide this discussion into: Introduction to SOA on page 2 SOA application principles on page 2 SOA constructs and components on page 4 SOA governance on page 5 SOA management considerations on page 7 Users of SOA management on page 9 Management needs for the SOA environment on page 16 Copyright IBM Corp. 2008. All rights reserved.1 15. 1.1 Introduction to SOA A service-oriented architecture (SOA) is an architecture that describes a system that is composed of discrete services that interact with clients and accomplish various tasks. Each service contains operations that perform a small unit of work that corresponds to a high level definition of a given task. These services can perform simple tasks, such as accessing data from a database and returning data to a client, or can be part of a workflow that represents a more complex task. A service usually either provides information or facilitates a way to modify it in a certain way. Although Web Services technology is commonly used to implement an SOA, it is not required; in other words, a service does not have to use Web Services constructs or technology. However, most SOAs rely on the standards, practices, and tools that are available with Web Services. Regard SOA as an approach to build distributed systems that deliver application functionality as Web Services to user applications. Properly used, SOA principles can provide a framework for matching business needs with realistic solutions. Web Services is the programmatical interface to a capability that complies with standard protocols, providing the interface technology and delivering platform independency and loose coupling of the transport. SOA is potentially wider in its scope of governing the policies, rules, and common services that enable logical service bus structure for use by authorized consumers internal and external to the enterprise regardless of implementation technology. Also, SOA enables the design and quality of service that can be reused and that conforms to functional and nonfunctional service level agreements (SLAs). Web Services are the foundational interoperability technology, and SOA is the application of the interoperability that implies considerable change in business and IT practices. This magnitude of business and technology change requires a certain level of management in order to successfully reap the benefits from the investment that has been made for the change. In this chapter, we look into the SOA structure and the management requirements that lead to the need for this paper to define the SOA system management environment.1.2 SOA application principles You can deploy an SOA-based application incrementally and slowly integrate it into an existing environment. Developers have designed SOA scenarios to be2 Managing an SOA Environment with Tivoli 16. used together or adopted incrementally. There are several well-known SOApatterns, including: Service creation Service connectivity Interaction and collaboration services Business Process Management Information as a serviceYou can apply the SOA design, governance, security, and management acrossall of these scenarios.Applications in a typical silo architecture build their functionality on top of anexisting application stack. In an SOA-based environment, the logical boundarybetween Web Services consumer and provider is defined by a business function,not by application boundaries (typical for silo architectures): Separation of Web Services interface from its implementation One of the most important aspects of SOA is that it separates a Web Services implementation from its interface. A meta-language, such as Web Services Definition Language (WSDL), can be very helpful, because it describes a business interface that a Web Services provider can expose to a client application and other Web Services while concealing the Web Services implementation. A WSDL document acts as a server-side descriptor that defines the Web Services operations and the messages they use so the client will know how to invoke the Web Service. Web Services consumers view a Web Services simply as an endpoint that supports a particular request format or contract. Web Services consumers are not concerned with how the Web Services executes their requests; they only expect that it will do so according to the defined interface. Implementations can use anything from Java 2 Platform, Enterprise Edition (J2EE) entities to older code running in a mainframe environment. Loose coupling The exposed interface is purposely loosely coupled from the Web Services provider so that implementations can be modified or even replaced (swapped in and out as desired in a plug and play manner), which increases the ability to reuse existing function. Reuse promotes increased performance, reliability, and Quality of Service (QoS), because a common interface can be exposed to many clients regardless of what is underneath. Reusable components do not have to be retested as often as well. Individual Web Services are also loosely coupled, having little or no dependencies upon each other. Web Services must also be stateless (information or state are not preserved from one request to another).Chapter 1. Introduction to SOA management3 17. Business impactClient applications in an SOA do not contain business logic; they consumeWeb Services instead. Therefore, they become much smaller and easier toimplement. A shorter time-to-market for new products is an important resultfor business development. SOA principles in general can help form a strongerconnection between the business needs that define the architecture and thesoftware and information systems that implement it. Web Services can becreated according to an abstract model that conforms to the needs of thebusiness. If done correctly, an abstraction can drive the concrete veryeffectively.1.3 SOA constructs and components Standards have been developed to ensure Web Services technologies conform to common principles. Support for Web Services interoperability, Web Services security, sending attachments using Web Services, and Web Services Management have been defined by organizations, such as W3C, OASIS, and so on. Standards help ensure that Web Services products sold by different software vendors conform to common agreed upon expectations. For example, a user must be able to expect that a Web Services client written using Microsoft .Net can invoke a Web Services written using Apache Axis deployed on the WebSphere Application Server without any complications if these tools conform to Web Services standards. With the advent of application server technology, Web Services can be distributed across many machines and environments, making it easier to perform scalability, clustering, and load balancing across your enterprise (SOA can ease the transition away from existing systems towards modern application servers and middleware). For example, COBOL and C++ applications that use messaging software, such as IBM MQSeries, can be phased out in favor of Java applications that use SOAP/Java Message Service (JMS) and message-driven beans (MDBs). SOA-related constructs, such as enterprise service buses (ESBs), business processes, and so on, can add further structure and flexibility to your architecture by providing routing, mediation, and flow management functions. Just as you can hide implementations from a client, transport/protocol layers and message formats can be similarly concealed by the inclusion of an enterprise service bus. SOAP/Java Message Service (JMS), SOAP/HTTP, Remote Method Invocation (RMI)/Internet Inter-ORB Protocol (IIOP), XML/MQSeries, local Java calls, and so on are all supported and transparent to the user. Often, people implement an enterprise service bus by technologies that are found in certain4 Managing an SOA Environment with Tivoli 18. middleware/infrastructure products, for example, WebSphere Enterprise ServiceBus, DataPower, WebSphere Message Broker, and so on.You can combine discrete Web Services into composite business processes toaccomplish more complex business objectives. As part of a business process,individual Web Services can be viewed as activities within a workflow. Thisfacilitates the creation of a business process that can be described by ameta-language called Business Process Execution Language (BPEL). BusinessProcess Execution Language is similar to Web Services Definition Language(WSDL) in that it provides a static definition of a Business Process. Just as withWeb Services, the potential for reuse is quite high after these Business Processdefinitions are created.In recent years, a new method has been introduced to model Web Services in anSOA called the Service Component Architecture (SCA). SCA components aredesigned to separate implementation details from any business logic so that youcan put together an integrated application without knowing its implementationdetails and make each component interoperate with any other SCA component.Issues, such as security, transactions, and so forth, are resolved in a seamlessmanner across SCA components. SCA components are often used in businessprocess modeling, because they provide a great deal of flexibility.A mediation is a special type of SCA module. You can insert mediations betweenloosely coupled Web Services. Introducing mediations between Web Servicesprovides added function for processing messages that are being passedbetween these Web Services. Mediations intercept and modify messages thatare passed between existing Web Services and clients that want to use thoseWeb Services. Mediations are ideal for deployment in an environment thatcontains an enterprise service bus, because they can reroute and examine WebServices traffic.1.4 SOA governanceIt is important to consider both governance and management requirements whenplanning an SOA.SOA governance is an extension of IT governance that focuses on the life cycleof Web Services and composite applications in an organizations SOA. SOAgovernance is related to establishing policies within the context of the activitiesand constructs associated with SOA that are similar to those that exist formanaging and controlling other aspects of IT. Chapter 1. Introduction to SOA management5 19. Initially, the concept of SOA governance was applied narrowly to the development and use of Web Services (validating Web Services adherence with specific standards or managing Web Services in the SOA runtime environment). Today, SOA governance spans SOA architecture, as well as the governance of Web Services, across the entire implementation life cycle. Architecture governance and Web Services-level life cycle governance are the two of the main components of SOA governance. For the purposes of this discussion, we will concentrate mainly on the latter.1.4.1 Web Services life cycle governance The goal of Web Services life cycle governance is to define:Decision rights for the development, deployment, and management of newWeb ServicesMonitoring and reporting processes for capturing and communicatinggovernance results There are four phases of the Web Services life cycle:Modeling: This phase involves incorporating business requirements andobjectives into your business design so that the design becomes aspecification of business processes that achieve goals and considerassumptions.Assembly: This phase centers around the information systems that will beused to assemble the business design. Typically, an enterprise architectworking with a business analyst can convert the business design into a setof business process definitions that are composed of activities. The requiredWeb Services are derived from the activity definitions and businessprocesses from the business process definitions.Deployment: This phase involves creating the environment that will hostcomposite Web Service-based applications and then deploying thoseapplications there. This phase includes identifying resource dependenciesfor the application, as well as operational conditions, requirements, andconstraints that impact the successful deployment and running of theapplications.Management: This phase addresses the maintenance of the operationalenvironment and the policies that govern the deployed applications. Thisphase includes monitoring the performance of Web Services requests andresponses and developing a recovery strategy for failures (detecting andquarantining failures and logging them, rerouting traffic around failures andrecovering work affected by them, correcting problems and restoring the state6 Managing an SOA Environment with Tivoli 20. of the system). The Management phase also includes tuning the operational environment to conform to changes in the business design.Because SOA applications are so loosely coupled, they introduce newgovernance challenges. But with the proper standards, practices, and processesin place, businesses can reap the full benefit of service orientation. Effective SOAgovernance helps business and IT teams better identify how to achieve mostbusiness goals. It also empowers employees by clearly defining their roles andresponsibilities.1.4.2 Web Services life cycle managementAfter you implement the SOA governance framework, you use it in the model,assemble, deploy, and manage phases within the SOA life cycle. With respect tothe operational aspects of implementing SOA governance, Web Services lifecycle management addresses how Web Services will be developed, deployed,and managed.Web Services life cycle management focuses on the development anddeployment of Web Services, while SOA governance supplies the decisionrights, processes, and policies for those activities. After a Web Services isdeployed, there must be management strategies in place to control and monitorthe Web Service. Web Services life cycle management is subject to the businessdesign created within the governance stage that ensures that reuse and costreduction are achieved.1.5 SOA management considerationsImplementing SOA-based applications introduces new IT managementchallenges. Because Web Services development tools make it easy to createservices within the SOA framework, there is a danger that services canproliferate and become uncontrollable within the SOA enterprise, if the growth isnot managed properly. When systems are composed of multiple independentbusiness processes, the relationships between these processes and theapplications executing in the IT layer are not always obvious. For example,consider verifying the correctness of a workflow in a system or locating aperformance bottleneck. While these actions are difficult in smaller systems andmight become impractical to manage as the size and complexity increases inlarger systems.Simply stated, SOA management includes solutions and software for managingand monitoring composite applications and the infrastructure that supports itacross the entire architecture. Chapter 1. Introduction to SOA management 7 21. Considerations associated with managing an SOA environment include:SOA management is needed on the following scenarios: Defining your Web Services in a way that they can be easily managed asresources (Service Creation) Defining how Web Services relate to each other (Service Connectivity andInteraction) How to manage your Web Services within their deployment environment(Governance/Management)The information that is needed from the management system: Capturing data that can be used to evaluate whether nonfunctional andquality of service requirements as defined by business needs aresupported (reliability, scalability, cost, and so on) Defining which Web Services/Activities to group into BusinessProcesses/Workflows to accomplish a business goal (CollaborationServices) Define service level agreements based on data that can be easilymonitored (average response time, maximum message size, and so on) Using standards whose enforcement can be monitored Evaluating whether your Web Services are secure (Security)The management environment itself must be properly evaluated.Considerations about the management technology are: How to display Monitored and Registered information to users (using aconsole, such as the Tivoli Enterprise Portal (TEP) or the Tivoli EnterpriseConsole (TEC) What technology and products to use to monitor various resources(examples include Simple Network Management Protocol (SNMP), JavaManagement Extensions (JMX), Java API for XML-Based RPC(JAX-RPC), Java API for XML-Based Web Services (JAX-WS), and so on) How can you manage infrastructure that employs tiers and clustering.Ensuring that both Web Services providers and consumers can bemonitored. Also, resolving how to display interactions if only certain thingsare managed (how do you display unmanaged clients and Web Servicesthat are part of the same flow) What are some of the various user types that you will need monitor andadminister your applications (administrators, operators, and so on) There are number of metrics that you must monitor for a holistic application health view in an SOA environment.8 Managing an SOA Environment with Tivoli 22. These metrics include: Service response time Service request message size Service faults Real-time service performance metrics Application server health where the service is deployed Health of dependent components, such as database, messaging resources, and so on Service performance metrics for historical purposes Service request and response message dataThese metrics help measure service level agreements (SLAs) for applications.You can use these metrics to debug performance bottlenecks in applications.You can also use these metrics to automatically take corrective actions or initiatefailover steps to maintain service availability when the primary service providergoes down or is not performing to meet the SLAs.IBM Tivoli Composite Application Manager (ITCAM) for SOA and other ITCAMofferings provide all the necessary metrics to keep the applications in an SOAenvironment healthy and resilient.1.6 Users of SOA managementThere are several types of users that use SOA management and need to interactwith the SOA-based environment. In this section, we discuss the access patternsand the needs of those users in relation to SOA management.With any management software, a number of various personnel can be potentialusers. You can have multiple user roles, and each role can be defined in part bythe users current job responsibilities. For example, an application developer anda performance tester most likely are interested in viewing varying pieces ofinformation at different times and thus might use the same console to viewdifferent information. Depending on their responsibilities, they might even havedifferent levels of access to various views. Ultimately, the roles andresponsibilities that are defined for various users will depend greatly on how acompany defines their architecture and how they want to manage it.Chapter 1. Introduction to SOA management 9 23. These are possible user roles for SOA management. The primary users are likely to be common to most organizations. There can also be secondary and provisional users as well:Note: We consider the user roles for managing the performance and theavailability of an SOA-based application. We do not include securitymanagement and its roles in this paper.Primary users: These users are the most common in most organizations.They have a critical need to use the SOA management tools for theirday-to-day jobs: Operator Middleware and application subject matter expertSecondary users: These users have supportive roles that require occasionalaccess to the SOA management tools. Although the tools are not a necessityfor them to perform their work, the tools greatly enhance their productivity: Performance analyst System administrator Web Services application developerProvisional users: These users also need access to SOA management onlyas the need arises. These roles use SOA management rarely, and the toolsdo not perform a critical role: Enterprise system management architect Business manager or executive1.6.1 Operator An operator or system operator is the first level of IT support personnel that might detect system, application, or performance problems. The operator or system operator job is directly related to ensuring the health of the IT environment and attaining the appropriate service level for IT operation. The primary interaction for an operator with the SOA management environment relates to:Monitoring for potential problems and correcting themEnsuring that they adhere to the system SLAProviding initial troubleshooting of a problem, and, if possible, fixing itEscalating a problem to the next level of support or subject matter expert forresolution if necessary10 Managing an SOA Environment with Tivoli 24. To accomplish these objectives, an operator can perform monitoring by using system management tools. The tools can provide views that the operator can use to detect color-coded conditions or troublesome values in a measurement. The tools can also provide automatic monitoring that generates events or alerts for the operator to address. In most environments, the operator must rely on alerts and events that happen on the environment instead of trying to navigate the tools to uncover problems. Using the Tivoli tools, IBM Tivoli Monitoring provides a facility for the operator to get alerts and navigate views in order to problems in the Tivoli Enterprise Portal. An alert might be sent to Tivoli Enterprise Portal to signal that a problem has occurred. On other occasions, uncovering the problem might require investigation by the operator. In either case, an operator must examine metric data to see if the operator can make a preliminary diagnosis of the problem. Event monitoring is represented by situations. ITCAM provides situations that come predefined (ready to use) or that can be modified by using the situation editor. Breaching an SLA can fire a situation to alert the operator of the problem. The situation event console displays information about these situations. Figure 1-1 shows the situation event console. Figure 1-1 Situation event console An operator can also look at the results of queries that are performed against Tivoli Enterprise Monitoring Agent. These queries can be either predefined or created by using the Tivoli Enterprise Portal query editor. The system or product administrator loads the predefined queries into the Tivoli Enterprise Portal Server. An administrator will often use the query editor to define and create queries for display on the Tivoli Enterprise Portal.1.6.2 Middleware or application subject matter expert The subject matter experts on the application or the middleware perform the in-depth problem determination. They might respond to problems that were initially uncovered by an operator. For a composite application and specifically for the SOA-based application, they must be able to see and trace theChapter 1. Introduction to SOA management 11 25. application beyond just the SOA interface and into the component level or a breakdown of the transaction. Subject matter experts on the application or the middleware perform these in-depth diagnostics from several sources, such as:Investigating the Web Services flow from one application server to anotherRerouting and modifying mediation and the Web Services flowCollecting response time breakdowns using correlation trackingPerforming a method trace for the J2EE application The subject matter experts on an application or the middleware perform these functions by using a combination of ITCAM solutions, such as:ITCAM for SOAITCAM for WebSphere and ITCAM for J2EEITCAM for Response Time Tracking1.6.3 Performance analyst A performance analyst inspects reports about availability and response time for various applications, including applications deployed in an SOA environment. If the metrics indicate that a threshold is close to being reached, the performance analyst consults a trend analysis of how the application has been performing over time. The conclusion can indicate a sudden spike in activity or a increasing trend over time, which might mean you need to expand your capacity. The performance analyst collects this information from various data sources from the monitoring system historical data. The Tivoli solution in the IBM Tivoli Monitoring environment is based on Tivoli Data Warehouse. The Tivoli Data Warehouse collects historical data from various Tivoli Enterprise Monitoring Agents. This data can be reported and analyzed by using the reporting tools that report into a DB2 database.1.6.4 System administrator The system administrator is a user with administrative privileges that performs the day-to-day tasks of maintaining the management system. The system administrator tasks include granting access to users, implementing a monitoring solution, extending the monitoring solution by using a standard procedure, and so on.12 Managing an SOA Environment with Tivoli 26. The system administrator has the authority to modify the system, such asinstalling a new feature or a patch to the monitoring system, thus, allowinginteraction with the management of the SOA solution.The system administrator can install, configure, and maintain all the tools that asubject matter expert needs. A system administrator can also configure, start,and stop agent processes, or perhaps even reconfigure the entire monitoringenvironment.System administrator interactions include: Managing situations, workspaces, and actions in Tivoli Enterprise Portal that are related to the SOA application Administering users, such as Tivoli Enterprise Portal users. Using the Administer Users window for setting authorities to specific features, specifying access to applications, and specifying access to Navigator views. Selecting the features in Tivoli Enterprise Portal to provide access to each user and to set the specific permissions granted to each user (see Figure 1-2 on page 14) Chapter 1. Introduction to SOA management13 27. Figure 1-2 Administer user dialog1.6.5 Enterprise system management architect The system management architect interacts with the business side of the company and is familiar with the companys business processes. An architect must understand the model for the SOA-based application, including Web Services, SCA, and business process choreography to model the architecture of the business. The architect observes as the model is built and then implemented into system management. The architect must oversee the system management implementation based on the business model and ensure that the management model is kept up-to-date. The architect must review the rollout of the new application and the change of the mediation rule to ensure that the monitoring model is still current and meaningful.14 Managing an SOA Environment with Tivoli 28. The architect utilizes the WebSphere Services Registry and Repository to manage the Web Services life cycle and can use the discovery through Tivoli Common Object Repository to monitor new services and the new model. You can do this by displaying the ITCAM for SOA Services Overview view to see the difference between which Web Services have been observed and which Web Services are registered. If you notice that certain observed Web Services are not registered, then you can update the registry.1.6.6 Web Services application developer The application developers are responsible for coding Web Services applications. After the developers Web Services are deployed, the developers must be available to fix problems. Depending on the severity of the problem, the developers usually have a small amount of time in which to complete the fix, and then they run tests to verify that the problem has been resolved. A typical scenario is a new mediation that needs to be added to manage traffic for an existing Web Service, because the system performance has been unexpectedly affected. The system management tools can assist the developer to identify the potential bottlenecks and performance problems in test system. The developers need to use the tools to test fixes for possible performance bottlenecks before the fixes go into production. After the build with the fix is installed, the developers observe whether the performance problem has been alleviated.1.6.7 Business executives The business executives care about their business processes and the applications that support the business. The management solution must allow the business executives to see the application health based on either the SOA performance or another metric. The business executives need an interface with minimum technical detail. The business executives need a simple but meaningful view of the business process health and possibly the SLA attainment. Chapter 1. Introduction to SOA management 15 29. 1.7 Management needs for the SOA environment Based on the user roles and the users needs in 1.6, Users of SOA management on page 9, the management needs for the SOA environment are:Understanding the current performance of the SOA application in real time inorder to proactively identify any potential outage and problemCollecting historical trends of SOA performanceBuilding structure about the calling pattern of the Web ServicesUnderstanding the structure and life cycle of Web ServicesPerforming deep dives and diagnoses on the SOA-based applicationModifying the routing and data analysis of the SOA Web Services callsShowing the business impact of Web Services call performance problems oroutagesManaging security of the SOA-based application. We do not discuss securitymanagement in this paper. See Understanding SOA Security Design andImplementation, SG24-7310. In a production environment, it is vital to have sufficient information for the management system as long as collecting this information does not adversely affect the managed environment. To manage an SOA-based application, you need to have information about the Web Services contained within it and the environment on which they are deployed. You can observe data on Web Services by monitoring it or from static definitions, such as WSDL documents that define a Web Service. Data on Services, ports, and operations is exposed, as well as details about the server (deployment environment) upon which the data runs (application servers, such as WebSphere Application Server, WebLogic, JBoss, DataPower, and so on). As with other software components, you must gather information throughout the Web Services life cycle (see section on 1.4.2, Web Services life cycle management on page 7).1.7.1 Web Services metric data collection You can collect metrics throughout the life of a Web Service. These metrics are usually numeric information that you can use to indicate or calculate the health and performance of Web Services. You can collect these metrics by using a polling process or instrumenting the application to report the metrics. Metrics that are collected at the Web Services operation level include average response time, average message size, number of messages, number of faults,16 Managing an SOA Environment with Tivoli 30. and so on. The metrics can provide valuable information about SOA and Web Services. You need to be able to view real-time data, as well as historical data, corresponding to a specific time interval. You can collect metric data by using a management API, such as Java API for XML-Remote Procedure Call (JAX-RPC) handlers, which make Web Services into manageable resources. Data collection using the JAX-RPC handler happens when when either of the following events occurs:A client application invokes a Web service, which is referred to as aclient-side interception.The Web Services request is received by the hosting application server,which is referred to as a server-side interception. The data can be stored in log files for later analysis, sent to a management server, or loaded into a repository.1.7.2 Web Services troubleshooting Troubleshooting Web Services includes monitoring the health of the infrastructure that supports the Web Services, such as the underlying middleware. Think of Web Services as manageable resources in the system management environment. In this way, you can determine whether policies are being enforced in SOA and whether SLAs are achieved or not. Data on throughput, availability, workload, transactions, and so on can be gathered to help you determine if your current topology is optimal, given the demands placed on your enterprise. Business processes are managed indirectly in that the Web Services that make up the business processes are managed resources. You might also consider a SOA management standard, such as Web Services Distributed Management (WSDM) from Oasis. See: http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsdm1.7.3 Displaying data You must be able to display Web Services entities, such as the deployed server, operation, namespace, relationship, and other attributes, to users. You can present this information in a tabular or graphical form, and you need to provide drill-down capability for the user to see details about the Web Service. A user must also be able to define events and alerts to monitor an SOA application. These alerts allows users to be notified when a certain conditionChapter 1. Introduction to SOA management 17 31. occurs. Alerts allows the user to not have to monitor a particular view all the time.1.7.4 Mediation management Mediation is a special type of SCA module. You can insert mediations between loosely coupled Web Services. Introducing mediations between Web Services provides added function for processing messages that are being passed between these Web Services. Mediations intercept and modify messages that are passed between existing Web Services and clients that want to use those Web Services. Managing mediations includes the ability to filter mediations or reroute Web Services calls based on monitoring metrics. You can perform this function on a mediation platform, such as WebSphere Enterprise Service Bus or WebSphere Message Broker.18 Managing an SOA Environment with Tivoli 32. 2Chapter 2. Tivoli application management products in this chapter, we provide an overview of the various Tivoli solutions that you can use to manage a service-oriented architecture (SOA) environment. Tivoli has a set of solutions to manage composite application. In a way, you can regard SOA as a special composite application environment. We discuss: ITCAM for SOA on page 20 ITCAM for WebSphere and ITCAM for J2EE on page 26 ITCAM for Response Time Tracking on page 33 OMEGAMON XE for Messaging on page 41 Copyright IBM Corp. 2008. All rights reserved.19 33. 2.1 ITCAM for SOA in this section, we describe IBM Tivoli Composite Application Manager (ITCAM) for SOA V6.1. We discuss:Product features on page 20Product components on page 212.1.1 Product features ITCAM for SOA manages service-oriented architecture (SOA). It can monitor, manage, and control the Web Services layer of the IT architecture, while drilling down to the application or resource layer to identify the source of bottlenecks or failures and to pinpoint services that take the most time or use the most resources. ITCAM for SOA:Provides service monitoring views in Tivoli Enterprise Portal. ITCAM for SOAworkspaces consist of data collector-based workspaces: Performance Summary: Shows the response time information for WebServices calls as viewed from the client or the server Message Summary: Shows the message statistics, including the volumeand size of message information Fault Summary: Shows failure analysis for Web Services callsOther workspaces for each agent are: Service Management Agent Environment: Provides a summary of theWeb Services metrics for all data collectors Service Management Agent: Shows the agent configuration summary,data collectors, monitoring profiles, and filters Mediation Configuration: Shows configuration entries for mediation onService Component Architecture (SCA) Message arrival: Shows the message arrival rate and events based on themessage arrival critical situationLeverages Tivoli Enterprise Portal situations to check thresholds. ITCAM forSOA provides predefined situations that you need to tailor. The predefinedsituations concern: Number of messages received by a service within a time window Size of the messagesProvides basic mediation support with the ability to filter or reject WebServices call messages from a particular client or service. It can log requestand response messages for analysis.20 Managing an SOA Environment with Tivoli 34. Offers heterogeneous platform coverage: Support for IBM WebSphere Application Server, CICS Transaction Server, Microsoft .NET, JBoss, BEA WebLogic, and other SOA clients and servers Target IBM Enterprise Service Bus platforms: WebSphere Application Server Versions 5.x and 6.x and WebSphere Business Integration Server Foundation V5.1.1 Displays a list of services and operations that are monitored in the environment Leverages Tivoli Enterprise Portal workflow and policy editor for threshold-triggered action sequences Offers the ability to include services-layer views in Tivoli Enterprise PortalThe context-rich views and inter-workspace linkages in Tivoli Enterprise Portalenables users to drill down to IT resources to identify Web Services bottlenecksand failures. By providing built-in and extensible alerts, situations, and workflows,users can create powerful automated mediation scenarios using the TivoliEnterprise Portal.The service metrics, alerts, and automation workflows that are provided byITCAM for SOA and other Tivoli products can be displayed in Tivoli EnterprisePortal with the cross-workspace linkages to provide a rich and multilayeredsource of information. This information can help to reduce the time and skills thatare required for problem root-cause analysis and resolution.ITCAM for SOA includes the Web Services Navigator, a plug-in to IBM RationalApplication Development and other Eclipse-based tools. It provides a deepunderstanding of the service flow, patterns, and relationships for developers andarchitects. The Web Services Navigator uses data from the IBM TivoliMonitoring V6.1 Tivoli Data Warehouse or from the ITCAM for SOA log filesusing the Log Assembler tool.In Version 6.1, IBM Tivoli Composite Application Manager for SOA contains anew component for mediation service management based on SCA. It enablesyou to modify several of the mediation service settings dynamically. Mediation isa facility that sits between Web Services requester and Web Services providerthat allows manipulation of Web Services messages, includes format translation,filtering, and enrichment.2.1.2 Product componentsITCAM for SOA manages Web Services. Web Services can be viewed as aremote processing facility that is defined through the use of Web ServicesChapter 2. Tivoli application management products 21 35. Definition Language (WSDL). Usual access uses SOAP over HTTP. Internally, Web Services are implemented using the Java API for XML-based Remote Procedure Call (JAX-RPC). ITCAM for SOA installs itself as the JAX-RPC handler to capture and manage Web Services calls. ITCAM for SOA consists of these logical components:Web Services data collector that acts as the JAX-RPC handler and interceptsWeb Services calls to collect statistical information and write to a log file.Tivoli Enterprise Monitoring Agent that collects information from all of the datacollectors on a machine and forwards them to Tivoli Enterprise MonitoringServer. We discuss the data collectors and Tivoli Enterprise Monitoring Agentin Monitoring agent data collector on page 22.An Eclipsed-based viewer that processes log files that are generated by theWeb Services data collector. It generates visual representations of variouscharacteristics of monitored Web Services. See IBM Web ServicesNavigator on page 24.Mediation SCA tools that enable partial monitoring of SCA within WebSphereEnterprise Service Bus. See Managing SCA mediation on page 25. Monitoring agent data collector ITCAM for SOA works with several application server environments:IBM WebSphere Application Server V5.1.0.5 with PQ89492, V6.0, and V6.1IBM WebSphere Business Integration V5.1.1.1IBM WebSphere Process Server V6.0.1IBM WebSphere Enterprise Service Bus V6.0.1IBM CICS Transaction Server V3.1 and laterBEA WebLogic Server V8.1.4Microsoft .NET V1.1 with Service Pack 1 and V2.0JBoss V4.03WebSphere Community Edition V1.0 and its service packsSAP NetWeaver V6.40 with Service Pack 9 or later service packsIBM WebSphere DataPower SOA Appliance Firmware V3.5.0.5 or later Figure 2-1 on page 23 shows the ITCAM for SOA data collection conceptual architecture.22 Managing an SOA Environment with Tivoli 36. ITCAM for SOAMonitoring agentApplication ServerconfigurationWeb Services handler orData Data collectorTivoli Enterprise extension collectoradapter Monitoring Agent log Tivoli EnterpriseManagement Server Tivoli Enterprise Portal ServerFigure 2-1 ITCAM for SOA structureThe monitoring agent data collector is implemented as a JAX-RPC handler orservice extension that is installed into the application servers that host themonitored Web Services. The handler is given control when either of thefollowing events occurs: A client application invokes a Web service, which is referred to as a client-side interception. The Web Services request is received by the hosting application server, which is referred to as a server-side interception.The monitoring agent records and collects monitored information into one ormore local log files. The information is then transferred to Tivoli EnterpriseMonitoring Server and can be archived into a historical database for laterretrieval with IBM Web Services Navigator.ITCAM for SOA V6.1 focuses on the SOAP engine of IBM WebSphereApplication Server, WebSphere Service Integration Bus, Microsoft .NETFramework, and BEA WebLogic.The Web Services data collector supports both Java 2 Platform, EnterpriseEdition (J2EE) application client and server container environments, becauseJAX-RPC handlers are supported only by these environments. The WebServices must be compliant with JSR-109 specifications.To ensure the proper operation of the JAX-RPC handler, verify that the clientapplications are written according to the conventions at the following location:http://www.jcp.org/aboutJava/communityprocess/final/jsr109/ Chapter 2. Tivoli application management products23 37. IBM Web Services Navigator IBM Web Services Navigator is an Eclipsed-based tool that is used to visualize Web Services in an SOA environment. It provides a graphical display of:Web Services transaction flowsService topologyFlow patterns Figure 2-2 illustrates Web Services Navigator concepts. MetriclogDatacollector TDWwarehouseMetricTivoli Enterprise log Data Monitoring Agent collector Web ServicesNavigator MetriclogDatacollectorLog AssemblerCombinedmetric log MetriclogDatacollector Figure 2-2 Web Services Navigator The Web Services Navigator is a log-browsing tool intended for offline analysis of SOA Web Services. The Web Services Navigator provides four primary views:Statistic tables: Message statistics Per-message statistics, including requestor, provider, send/receive time, and message size Invocation statistics Response time, network delay, message size, and more for each Web Services invocation Transaction statistics Statistics for aggregated transactions, including elapsed time, number of faults, number of machines that this transaction involves, and number of invocations comprising this transaction24 Managing an SOA Environment with Tivoli 38. Pattern invocation statisticsStatistics for discovered patterns, including operation names, number ofoccurrences, response times, and message sizesNote: To see the message content from the ITCAM for SOA metric log:1. Set a monitor control higher than none for any or all of the Web Services being monitored.2. Include the subsequent xxxx.content.log when running Log Assembler. Service topology view This view is a graphical representation of the monitored Web Services that displays aggregated information and details about the relationships between Web Services. Transaction flows view The transaction flows view displays Universal Markup Language (UML) style sequence diagrams. The transaction flow shows a chronological view of each transaction, the flow between the various Web Services over time, and the topology and statistics for each transaction. You can zoom in on the view to see the details of individual transactions. Flow pattern view The flow pattern view is a visual representation of the aggregated pattern of transactions represented in the log file. The view also represents each pattern as a distinct sequence of Web Services calls and displays the frequency of each pattern.Managing SCA mediationWebSphere Process Server and WebSphere Enterprise Service Bus introduce anew way to model services in an SOA, which is called the Service ComponentArchitecture (SCA). SCA is designed to separate business logic from itsimplementation so that you can focus on assembling an integrated applicationwithout knowing implementation details.There is a special type of SCA component, which is called a mediation. In anSOA, where services are loosely coupled rather than connected directly to eachother, mediations can be inserted between the services, where they canintercept and process messages that are passed between the services.Mediations can process these messages and take appropriate actions, such asreroute, log, or transform a message, or create a notification or an event.IBM Tivoli Composite Application Manager for SOA provides the ability todynamically enable and disable the deployed mediation functions. This facility is Chapter 2. Tivoli application management products 25 39. available for applications in the WebSphere Enterprise Service Bus or WebSphere Process Server runtime environment. The invocation is provided in a new workspace in Tivoli Enterprise Portal called the Mediation Configuration workspace. The actions are:ConfigureMediation_610DeletePrimitiveProperty_6102.2 ITCAM for WebSphere and ITCAM for J2EE The IBM Tivoli application management solution for J2EE application servers comes in the form of ITCAM for WebSphere and ITCAM for J2EE. These two products share the same managing server. ITCAM for WebSphere and ITCAM for J2EE observe and report on the health of J2EE-based applications. They track the progress of applications as they traverse through J2EE application servers, middleware adapters and transports, and database calls, and on to back-end systems, such as CICS or IMS, to extract business data or to invoke mainframe business processes. Tracking applications produces request traces, where the events in a requests life are recorded and stored in a monitoring repository database. ITCAM for WebSphere and ITCAM for J2EE capture the CPU and the elapsed internal times when events are called and when they are exited, measuring as far down as the CPU times consumed and the elapsed internal times charged to individual methods in J2EE classes. The methods or events taking the most time are marked as an applications parts that deserve attention for runtime improvement studies and code optimizations. ITCAM for WebSphere manages and monitors WebSphere-based application servers, while ITCAM for J2EE manages and monitors the following J2EE containers:JBossTomcatSAP NetWeaverBEA WebLogic ServerOracle Application ServerApache Web ServerSun Java System Web ServerMicrosoft IISWebSphere Application Server CE ITCAM for WebSphere and ITCAM for J2EE do not need modification of any J2EE or mainframe application code. The data collectors use the following principal data sources: Java Virtual Machine Tool Interface (JVMTI) interfaces26 Managing an SOA Environment with Tivoli 40. and primitives. ITCAM for WebSphere also uses WebSphere Performance Management Interface (PMI) and z/OS System Measurement Facility (SMF) 120 records. The monitoring data is collected and analyzed to offer a wealth of information about the health of J2EE applications and their servers. These products collect and report many system-level performance metrics about J2EE application servers. The status of the servers and their resources (particularly at vital checkpoints, such as CPU utilization), memory usage, and the status of internal components, such as database connection pools, JVM thread pools, EJB usage, and request processing statistics, can be extremely important in locating real-time problems with J2EE applications. ITCAM for WebSphere and ITCAM for J2EE bring attention to these critical indicators with real-time, graphical displays of their values and their trends over spans of time.2.2.1 Architecture and interconnection ITCAM for WebSphere and ITCAM for J2EE are distributed performance monitoring applications for application servers. The components are connected through TCP/IP communication. The central component of ITCAM for WebSphere and ITCAM for J2EE, the managing server, is its heart and brain. It collects and displays various performance information from application servers.Chapter 2. Tivoli application management products 27 41. The application servers run the data collector, which is a collecting agent that runs in the application server and sends monitoring information to the managing server. These data collectors operate independently of each other. Figure 2-3 shows the overall architecture of ITCAM for WebSphere and ITCAM for J2EE. Browser interface ITCAM for WebSphereITCAM for J2EE Managing Server I Tivoli Enterprise Monitoring Agente ris nt rp te ge Tivoli Enterprise En ng A li iApplication servers with vo or Ti onitManagement ServerData collectorsMand Tivoli Enterprise Portal Server Web Servers Figure 2-3 ITCAM for WebSphere and ITCAM for J2EE architecture The application monitor consists of two major parts: the managing server and the data collectors. A data collector agent runs on each monitored application server, whether J2EE, CICS, or IMS, and communicates essential operational data to the managing server. Unique sampling algorithms maintain low CPU and network processing while providing application-specific performance information. The managing server consists of several Java-based components that provide the environment to collect and present management data.2.2.2 The managing server ITCAM for WebSphere and ITCAM for J2EE use one common managing server that controls and coordinates data collectors for J2EE, CICS, and IMS servers28 Managing an SOA Environment with Tivoli 42. that run applications. The difference between ITCAM for J2EE and ITCAM forWebSphere is the platform support for the data collectors. These data collectorscan run independently.The managing server uses the following software: Managing server database (DB2 UDB or Oracle on Sun Solaris) for the relational data repository WebSphere Application Server to run the visualization engine Web console application An optional Web server, such as IBM HTTP Server The managing server overseer components, which are a set of Java-based processesThe overseer components are the controlling logic for the managing server. Forthe overseer components: Kernels control the managing server. There are always two copies of the kernels running on an ITCAM for WebSphere and ITCAM for J2EE managing server for redundancy and failover. The kernels register components as they join the managing server, periodically renew connections and registrations with components and data collectors, and collect server and component availability information. Publishing servers receive application and system event data from the data collectors, gather and compute request-level information about performance metrics such as response times, and implement the trap monitoring and alerts features. Archive agents receive monitoring data from the publish servers and store the monitoring data in the repositories of ITCAM for WebSphere and ITCAM for J2EE. The global publishing server collects information from the publish servers and correlates all parts and pieces of multiserver requests, such as requests from J2EE servers to execute CICS or IMS programs. The message dispatcher is a conduit for messages from ITCAM for WebSphere and ITCAM for J2EE using e-mail and Simple Network Management Protocol (SNMP) facilities.Chapter 2. Tivoli application management products 29 43. The visualization engine is a Web-based GUI with access to graphics, ITCAMfor WebSphere and ITCAM for J2EE performance reports, real-time views ofdifferent slices of monitoring data, and ITCAM for WebSphere and ITCAM forJ2EE internal commands and event-driven functions. The visualizationengine runs on a WebSphere Application Server. Figure 2-4 shows the conceptual relationship among the components. Snapshot trafficPublish traffic Global Publish Server (SAM)Publish Server (PS) Kernel (KL) Visualization EngineProvide services on:Provide services on: Message Dispatcher- Lookup-Administration (MD)- Registration-Availability - Recovery-Problem DeterminationArchive Agent (AA) - Configuration -Performance Management OCTIGATE databaseFigure 2-4 Kernel components At the managing server, monitoring data is prepared for real-time displays within the monitoring console and is inserted into the OCTIGATE data repository. These are extremely resource-intensive operations. Having this processing in the managing server isolates this from other the application servers, thus reducing the footprints of ITCAM for WebSphere and ITCAM for J2EE in the monitored systems. This design also helps keep the data collectors processing at levels low enough for 24x7 production system monitoring. Data from the data collectors is collected by the publishing server and then stored in the OCTIGATE database by the archive agent. The visualization engine reads the database to present data through the Web console, and snapshot information, such as lock analysis and in-flight transactions, is retrieved directly from the data collectors.2.2.3 J2EE and WebSphere data collectors The data collectors run inside the application servers. They use native system services, and they are tailored for the particular environments where they30Managing an SOA Environment with Tivoli 44. execute. The data collectors for z/OS systems are written to take advantage ofservices on z/OS, such as MVS Cross-Memory Services and address spacefencing, which are not available on distributed systems.Data collectors are configured as a multithreaded process. They consist of thefollowing agents: Command agent The command agent collects requests from other components for information about Enterprise Java Bean (EJB) invocations, database connection pools, thread pools, stack traces, memory analyses, and heap dumps. Event agent The event agent provides data to the publish servers according to polling frequencies. This data includes system initialization data, application request-level data, and application method-level data. Secondary collector The optional secondary collector provides support for displaying data in Tivoli Enterprise Portal for collecting WebSphere Application Server and other J2EE application server performance metrics. This component communicates with Tivoli Enterprise Monitoring Agent using a TCP/IP port.Collectively, these agents and other data collector routines unleash the probes,package the monitoring data into Java formats for the managing server, anddeliver the data to the managing server.The data collectors send the probes into the application servers to analyze theapplications performance. The probes collect monitoring data and feed it totransport routines that in turn route the data to the managing server. Themanaging server processes it for display in the Web console and for storage inthe OCTIGATE repository, which relieves the processing burden of ITCAM forWebSphere and ITCAM for J2EE from the application servers as much aspossible. The data collectors and probes are not designed to analyze or interpretdata, but to collect it and route it as quickly as possible to the managing serverwhere the analysis is performed.The data sources that are employed by ITCAM for WebSphere and ITCAM forJ2EE are: Java Virtual Machine Tools Interface (JVMTI) garbage collection data, method trace, stack trace, CPU time, and heap dump Java Management Extensions (JMX) system resources System Management Facilities (SMF) system resources (z/OS only) PMI system resources (WebSphere only) Chapter 2. Tivoli application management products 31 45. OS services, platform CPU, and its environmentByte Code Modification (BCM) instrumentation of some classes The data collector in the J2EE server runs as a custom service called am in a distributed architecture and - for z/OS architecture. Figure 2-5 shows the conceptual data collector structure of the distributed WebSphere data collector. WebSphere JVMTIJMXPMICustom Service bcmPublish data am KYNTivoli Enterprise To TEMSMonitoring Agent Figure 2-5 J2EE data collector structure2.2.4 Tivoli Enterprise Monitoring Agent The ITCAM for WebSphere and ITCAM for J2EE Tivoli Enterprise Monitoring Agent can forward monitoring information to Tivoli Enterprise Monitoring Server for monitoring using Tivoli Enterprise Portal. There is an additional component at Version 6.1, Tivoli Enterprise Monitoring Agent for Web Servers. Web server monitoring no longer uses the managing servers polling agent, but uses Tivoli Enterprise Monitoring Agent for Web Servers instead. The existing Tivoli Enterprise Monitoring Agent for WebSphere and J2EE provides application server performance information, while the new Tivoli Enterprise Monitoring Agent for Web Servers displays Web server performance information.32 Managing an SOA Environment with Tivoli 46. 2.3 ITCAM for Response Time TrackingITCAM for Response Time Tracking Version 6.1 is an evolution from IBM TivoliMonitoring for Transaction Performance V5.3. It inherits the major componentsand functions of IBM Tivoli Monitoring for Transaction Performance V5.3.Figure 2-6 shows the ITCAM for Response Time Tracking components.RTT RTT RTTRTT Management Store and Store and management server forward agent forward agentagentFIREWALL FIREWALL RTT RTT RTTRTTTivoli managementmanagement managementmanagementEnterpriseagent agentagent agent ManagementAgentFigure 2-6 ITCAM for Response Time Tracking componentsBasically, ITCAM for Response Time Tracking is controlled from themanagement server. The management server provides a centralized repositoryof policy, configuration, and data for the ITCAM for Response Time Trackingenvironment.The rest of ITCAM for Response Time Tracking consists of the managementagents. The management agents perform performance and response-time datacollection on behalf of the management server. The agent can perform responsetime collection from an application server or perform robotic simulation of atransaction for measuring response time. The management agent functions as asingle agent that can have different monitoring components deployed on it toperform various functions.The management server and management agent typically operate in anunrestricted port environment. When there is a firewall between them, theyrestrict the port usage to communicate. The firewall requirement typicallyrequests that they use a single communication port to talk back and forth. Thisis where the store and forward agent comes in. It bundles the management Chapter 2. Tivoli application management products 33 47. communication between the management server and management agent to use a single port to pass through the firewall. The store and forward agent can be cascaded, so in this sense, there can be a chain of store and forward agents to pass through multiple layers of firewalls. A special management agent resides on z/OS machines. The management agent on z/OS machines has the transaction server component activated to receive performance information from the CICS and IMS data collector. In ITCAM for Response Time Tracking V6.1, information from the management server can be forwarded to Tivoli Enterprise Monitoring Server for display in Tivoli Enterprise Portal. Use the Tivoli Enterprise Monitoring Agent for ITCAM for Response Time Tracking to have this capability. We discuss the components of ITCAM for Response Time Tracking in the following sections:The management server on page 34Store and forward agent on page 36Management agents on page 37Tivoli Enterprise Monitoring Agent on page 412.3.1 The management server The ITCAM for Response Time Tracking management server consists of a J2EE enterprise application that accesses a DB2 repository using JDBC. The management server runs on WebSphere Application Server. The application server can be installed on a stand-alone WebSphere Application Server or on a clustered environment. Figure 2-7 shows a stand-alone management server. Management server WebSphere Application Server DB2 Figure 2-7 Stand-alone management server34 Managing an SOA Environment with Tivoli 48. Figure 2-8 shows the clustered management server. It consists of WebSphereEdge Server for load balancing, the management server on several clusteredWebSphere Application Server Node Deployment systems, and the databaseinstalled on a separate database server.WebSphere Node Deployment ClusterWebSphereApplication Server WebSphere Edge Server DB2WebSphereApplication ServerFigure 2-8 Clustered management serverClustered management server benefits include: Separating the servers reduces the processing burden of a single machine. Clustered management server allows failover for failure in WebSphere Application Server, so the other management server in the cluster can take over the work.Some disadvantages are: Additional communication traffic between machines More difficult setup; refer to IBM Tivoli Composite Application Manager for Response Time Tracking V6.0: Installing a Management Server in a WebSphere Cluster Environment, SC32-1804, for installation instructionsOverall, regardless of the management server types, the management serverprovides the following functions: Managing management agents and their deployed components; management agents must sign in to the management server and retrieve all required policies when it is started initiallyChapter 2. Tivoli application management products35 49. Storing policies for management agent operation, including discoverypolicies, listening policies, and playback policies, which are maintained usingthe Web interface or the new command-line interfaceManaging a schedule repository; note that the management agent performsthe schedule applicationPerforming data collection from various management agents and storingthem in its repository.Maintaining users and roles for accessing the Web interfaceServing the Web interface2.3.2 Store and forward agent The store and forward agent acts as an intermediary between the management server and the management agent. Figure 2-9 shows its overall processing. 1976 Store and ForwardRTT RTTagent Management80 Management Caching server9081 Proxy agent Figure 2-9 Store and forward agent This agent consolidates communication from and to management agents and uses a single port to communicate to the upstream component. The store and forward agent can be cascaded. It uses IBM WebSphere Caching Proxy, which is part of WebSphere Edge Server V2.0. The caching proxy optimizes the connection with the management server. The default port, to which the management agent must connect, is 9446 for Secure Sockets Layer (SSL) or 80 for non-SSL. You can have multiple store and forward agents chained to get to the management server through multiple layers of firewalls. Figure 2-10 on page 37 shows this concept.36 Managing an SOA Environment with Tivoli 50. Store andStore and Management Managementforward80forward80 Serveragent agentagentFigure 2-10 Multiple store and forward agents2.3.3 Management agentsThe management agent runs in a Java virtual machine on the managed server. Ittypically performs the following functions: Starting and stopping the management components Collecting monitoring and schedule information from the management server Informing the management components about what to perform Caching response time data in the temporary directory Uploading response time data as requested by the management server, either at regular collection time or on demandThe management agent behavior is based on the Application ResponseMeasurement (ARM) specification. The management component monitorsmeasure response time and report the times using the ARM specification to theARM agent process (tapmagent executable). The ARM agent process storesresponse time information on physical disk. The management agent uploads theresponse time information to the management server at the regular interval.There is a slight difference between the distributed management agentarchitecture and the z/OS-based management agent.Chapter 2. Tivoli application management products37 51. Distributed management agent Conceptually, Figure 2-11 illustrates the processing of the management agent. ARM agent QoS Apache Web Management reverseResponseagentproxyMonitorClient J2EE Generic STI client Applicationmonitoring Window Tracker component Figure 2-11 Management agent The existing management components are:The Generic Window component allows investigation of a Windows-basedapplication to use with Rational Robot application, which enables the Robot tointeract with a native Windows application, a Java application, or abrowser-based application. You can only deploy this Generic Windowcomponent on a Windows system.Synthetic Transaction Investigator (STI) simulates user interaction with aWeb browse