PPM - All About Integration - casupport.broadcom.com · SOAP is a industry standard, describing the exchange of XML based messages in heterogeneous environments SOAP is a remote procedure

All About IntegrationXOG XML Open Gateway

Lars Seibert, Engineering Services Architect

July 2009

Terms of This Presentation

This presentation was based on current information and resource allocations as of October

2009 and is subject to change or withdrawal by CA at any time without notice.

Notwithstanding anything in this presentation to the contrary, this presentation shall not serve

to (i) affect the rights and/or obligations of CA or its licensees under any existing or future

written license agreement or services agreement relating to any CA software product; or (ii)

amend any product documentation or specifications for any CA software product. The

development, release and timing of any features or functionality described in this presentation

remain at CA‟s sole discretion. Notwithstanding anything in this presentation to the contrary,

upon the general availability of any future CA product release referenced in this presentation,

CA will make such release available (i) for sale to new licensees of such product; and (ii) to

existing licensees of such product on a when and if-available basis as part of CA maintenance

and support, and in the form of a regularly scheduled major product release. Such releases

may be made available to current licensees of such product who are current subscribers to CA

maintenance and support on a when and if-available basis. In the event of a conflict between

the terms of this paragraph and any other information contained in this presentation, the

terms of this paragraph shall govern.

2 October 11th, 2009 [XOG - Integrations] Copyright © 2009 CA

For Informational Purposes Only

Certain information in this presentation may outline CA‟s general product direction. All

information in this presentation is for your informational purposes only and may not be

incorporated into any contract. CA assumes no responsibility for the accuracy or completeness

of the information. To the extent permitted by applicable law, CA provides this document “as

is” without warranty of any kind, including without limitation, any implied warranties or

merchantability, fitness for a particular purpose, or non-infringement. In no event will CA be

liable for any loss or damage, direct or indirect, from the use of this document, including,

without limitation, lost profits, lost investment, business interruption, goodwill, or lost data,

even if CA is expressly advised of the possibility of such damages.


Session Abstract

> This session is dedicated to the technology available, be

it Web Services (XOG), Business Processes or GEL

scripting, to get your data moving in and out of Clarity

as your business demands.

Agenda

> XOG Overview / XOG usage

> XOG Client

> XOG Example

> XOG Via GEL

> Error trapping with GEL

> XOG Via WSDL and Java

> XOG Tips / Common Issues

> Documentation / Community Assistance

> XOG – XML Open Gateway

> XOG is the SOAP based CA Clarity PPM web service interface

that can be used to:

Import/Export data

Move configuration data between Clarity environments

Initiate processes in Clarity

Execute NSQL queries via the Query API

via XML files


XOG Overview

> SOAP (Simple Object Access Protocol)

SOAP is used for data exchange in distributed

environments

SOAP is a industry standard, describing the exchange of

XML based messages in heterogeneous environments

SOAP is a remote procedure call (RPC) protocol based on

HTTP and XML


XOG Overview

XML (Extensible Markup Language)

XML is an open standard produced by the W3C committee

XML is used for the representation of arbitrary data

structures in a textual format

XML contains no binary data therefore it is human

readable

XML supports all the languages of the world via Unicode


XOG Overview

> Conclusion

XOG is based on standards produced by the W3C

committee

XOG can be used to import or export data from any

platform supporting these standards

Therefore we call it the Clarity XML Open Gateway


XOG Overview

> What kind of „data‟ can be Xogged?

Content

– Data such as objects, views, filters, lookups etc…

Instance

– Data such as specific projects, resources, timesheets,

etc…


Typical XOG usage

> How do our customers typically use XOG? Here are a few

examples:

Importing resources out of a resource management

system such as PeopleSoft and into Clarity

Exporting financial transactions out of Clarity and

into an external billing system

Content pack XOG – between Clarity Environments


Typical XOG usage

> Migration of object customisations between Clarity

environments, for example:

> See KB TEC482079 - General Migration XOG Guidelines


DEV PRODUAT

Develop Custom Portlet

Deploy into Production

User Testing

XOG XOG

Content XOG

> From the command line

> In the XOG Shell

> In a process with a custom GEL Script

> In a Web service call via WSDL

> In a high level programming language like Java


Different ways to use XOG

> The XOG Client communicates with the Clarity PPM server via HTTP SOAP protocol to:

Log in

Start an authenticated session

Execute requests to „read‟/‟write‟ data

Log out

End session

> Can be run via command line


XOG Client

> Where do we get the XOG Client for the command line or the XOG shell from and how is it installed?

1. In the Clarity Admin Tool, select Client Downloads

2. Download and install the XML Open Gateway Client directly to your

workstation.

You‟ll want to make sure you have your Clarity user configured with the

XOG Administration global right as well as the rights required for your

XOG activity.

15 July 28, 2009 [XOG - The Basics] Copyright © 2009 CA

XOG Client

> Folder Structure

In the folder structure created by the XOG install, the two

main folders you will need are \bin and \xml.

The \bin folder contains the properties file needed for

modifying your XOG settings.

The \xml folder contains the xml files used to tell XOG

what data to read as well as example files for XOG writes.

– For example, prj_projects_read.xml reads project data from

Clarity.

16 July 28, 2009 [XOG - The Basics] Copyright © 2009 CA

XOG Client

> Getting started

For example, you‟ve added a new project in your

DEV environment that now needs to be moved into

your PROD environment

1. In your XOG installation XML folder, locate the correct

project instance „read‟ file to use

2. For our example, we‟ll be using the prj_projects_read.xml

file. Open this file and save it as a new copy for your test

(prj_projects_readTEST.xml)


XOG Example

> Setting up the XOG Read File

The xml read files provided in the installation XML folder,

give examples of how to set the filter section per item.

Instance related data could be Xogged out by filtering on

the ID of the instance or by applying filters to read

multiple instances.

Arguments can also be utilised in some read xml files to

specify which items associated with an instance you would

like to XOG (a project‟s tasks but not resources, for

example).

Once you have modified the XML to suit your needs, you

are ready for XOGing.


XOG Example

> Setting up the XOG Read File

In our example, since we know exactly which project we

want to „read‟ out, you‟ll want to use the projectID filter in

order to indicate you are filtering your data based upon a

specific projectID

<Filter name="projectID“ criteria="EQUALS“>PJ0000043</Filter>


XOG Example

> Filter section

This is what the filter section of our „read‟ file looks like

before we perform our modifications: 

Note the open and close XML comment tags




XOG Example

> Example of our completed XOG „read‟ XML:

<?xml version="1.0" encoding="UTF-8"?>

<NikuDataBus xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:noNamespaceSchemaLocation="../xsd/nikuxog_read.xsd">

<Header version="6.0.11" action="read" objectType="project"

externalSource="NIKU">

<args name="include_tasks" value="true"/>

<args name="include_dependencies" value="true"/>

<args name="include_subprojects" value="true"/>

<args name="include_resources" value="true"/>

<args name="include_custom" value="true"/>

</Header>

<Query>

<Filter name="projectID" criteria="EQUALS">PJ00000043</Filter>

</Query>

</NikuDataBus>


XOG Example

> Now that we have a modified „read‟ file, how do you

run the XOG Client on the command line?

You can invoke XOG in the following ways:

a) From the command line you can type in the parameters

required to import and export data directly on the command

line

b) You can store the parameters in a .properties file and call

the file from the XOG Client.


XOG Example – Invoking XOG

> Running XOG directly from the command line

xog –servername=clarityserver –input=../xml/prj_projects_readTEST.xml

-output =../xml/prj_projects_writeTEST.xml –user=admin –password=nimda

This method is normally only used in batch scripts

> Running XOG from the command line with a propertyfile

Modify test.properties file in the \bin folder

servername=clarityserverinput=../xml/prj_projects_readTEST.xml output=../xml/prj_projects_writeTEST.xml username=admin password=nimda

Execute XOG with the following command in the /bin folderxog –propertyfile test.properties



> Running XOG from the command line with a propertyfile

Some things to remember when using the test.properties

file…

– Set the servername = the server you are reading from or

writing to

– Check the settings for portnumber and sslnumber. They are

set to default and commented out .

– Set the input = the read or write xml file you want to use

– Set the output = whatever you want to name the output

file. By default, the properties file sets the name to out.xml

and the location to the bin folder.

– Set the username and password to a user with appropriate

rights (be sure there are not spaces at the end of the

username or password)



> The XOG shell

Enter the following command in the /bin folderxog

Login> login admin/nimda@clarityserver

Set output> output ../xml/prj_projects_writeTEST.xml

Invoke XOG> call ../xml/prj_projects_readTEST.xml



GEL - Generic Execution Language

> The Power of GEL

FTP

Query multiple data sources

Read and write files

Invoke XOG


> Using XOG in a custom GEL script

Load the needed libraries<gel:script

xmlns:SOAP-

ENV="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:core="jelly:core"

xmlns:gel="jelly:com.niku.union.gel.GELTagLibrary"

xmlns:soap="jelly:com.niku.union.gel.SOAPTagLibrary"

xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope"

xmlns:xog="http://www.niku.com/xog"

xmlns:xsd="http://www.w3.org/2001/XMLSchema"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">


XOG Via GEL

> Using XOG in a custom GEL script (continued)

Login<soap:invoke endpoint="http://clarityserver/niku/xog" var="auth" >

<soap:message>

<soapenv:Envelope

xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope"

xmlns:xog="http://www.niku.com/xog">

<soapenv:Header />

<soapenv:Body>

<xog:Login>

<xog:Username>admin</xog:Username>

<xog:Password>nimda</xog:Password>

</xog:Login>

</soapenv:Body>

</soapenv:Envelope>

</soap:message>

</soap:invoke>


XOG Via GEL


Get the session id<gel:set asString="true"

select="$auth/SOAP-ENV:Envelope/SOAP-

ENV:Body/xog:SessionID/text()"

var="sessionID" />


XOG Via GEL


Set XML<gel:parse var="XOGin">

<NikuDataBus xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:noNamespaceSchemaLocation="../xsd/nikuxog_read.xsd">

<Header version="6.0.11" action="read" objectType="project"

externalSource="NIKU">

<args name="order_by_1" value="name"/>

<args name="order_by_2" value="projectID"/>

<args name="include_tasks" value="true"/>

<args name="include_dependencies" value="true"/>

<args name="include_subprojects" value="true"/>

<args name="include_resources" value="true"/>

<args name="include_custom" value="true"/>

</Header>

<Query>

<Filter name="projectID" criteria="EQUALS">PJ00000043</Filter>

</Query>

</NikuDataBus>

</gel:parse>


XOG Via GEL


Invoke XOG<soap:invoke endpoint="http://clarityserver/niku/xog" var="XOGout">

<soap:message>

<soapenv:Envelope

xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"


<soapenv:Header>

<xog:Auth>

<xog:SessionID>${sessionID}</xog:SessionID>

</xog:Auth>

</soapenv:Header>

<soapenv:Body>

<gel:include select="$XOGin"/>

</soapenv:Body>

</soapenv:Envelope>

</soap:message>

</soap:invoke>


XOG Via GEL


Close XOG connection<soap:invoke endpoint="http://clarityserver/niku/xog" var="Result">

<soap:message>

<soapenv:Envelope

xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"


<soapenv:Header>

<xog:Auth>

<xog:SessionID>${sessionID}</xog:SessionID>

</xog:Auth>

</soapenv:Header>

<soapenv:Body>

<xog:Logout/>

</soapenv:Body>

</soapenv:Envelope>

</soap:message>

</soap:invoke>


XOG Via GEL

XOG Via GEL Via Process

> GEL Scripts can be run in a process custom script action

Use a process with Start Conditions rather than a trigger

– Stay in support compliance

– Insure business rules are maintained

Automate manual data entry processes


Capturing Errors in GEL

> Be sure to add check points to capture errors

Use a Choose function to test for a successful response

Log response of success or failure

> Example 1 Login

Test for a valid session ID


Capturing Errors in GEL<soap:invoke endpoint="${xogURL}/niku/xog" var="auth">

<soap:message>

<soap-env:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"


<soap-env:Body>

<xog:Login>

<xog:Username>${xogUser}</xog:Username>

<xog:Password>${xogUserPassword}</xog:Password>

</xog:Login>

</soap-env:Body>

</soap-env:Envelope>

</soap:message>

</soap:invoke>

<gel:set asString="true“ select="$auth/SOAP-ENV:Envelope/SOAP-ENV:Body/xog:SessionID/text()

"var="sessionID"/>

<core:choose>

<core:when test="${sessionID == null}">

<gel:log level="ERROR">ERROR: Login to Clarity XOG failed. Check username and password.

</gel:log>

</core:when>

<core:otherwise>

<gel:log>SUCCESS: Login to Clarity XOG successful.</gel:log>

</core:otherwise>

</core:choose>


Capturing Errors in GEL

> Example 2: Use XOG output to log success and failures

response

Scenario: XOG is initiated from GEL script within a process

Process starts on update to project

XOG closes milestones if all previous tasks are complete

Record XOG output and number of records affected


Capturing Errors in GEL<gel:set asString="true“ select="$result/SOAP-ENV:Envelope/SOAP-ENV:Body/XOGOutput/Status/@state"

var="XOGOutcome"/>

<gel:set asString="false“ select="$result/SOAP-ENV:Envelope/SOAP-ENV:Body/XOGOutput/Statistics" var="stats"/>

<gel:set asString="true” select="$result/SOAP-ENV:Envelope/SOAP-ENV:Body/XOGOutput/Records" var="vrecords"/>

<core:switch on="${XOGOutcome}">

<core:case value="SUCCESS“>

<gel:log>SUCCESS: XOG to update Clarity milestones is successful.

Total = <gel:expr select="$stats/@totalNumberOfRecords"/>.

Inserted Records = <gel:expr select="$stats/@insertedRecords"/>.

Updated Records = <gel:expr select="$stats/@updatedRecords"/>.

Failure Records = <gel:expr select="$stats/@failureRecords"/>

</gel:log>

</core:case>

<core:case value="FAILURE">

<gel:log level="ERROR">FAILED:**${proj_unique_name}** XOG to update milestones failed.

Total = <gel:expr select="$stats/@totalNumberOfRecords"/>.

Inserted Records = <gel:expr select="$stats/@insertedRecords"/>.

Updated Records = <gel:expr select="$stats/@updatedRecords"/>.

Failure Records = <gel:expr select="$stats/@failureRecords"/>

</gel:log>

</core:case>

<core:default>

<gel:log level="ERROR">FAILED: to find the Clarity Project information! </gel:log>

</core:default>

</core:switch>


XOG Via WSDL

> WSDL? – Web Service Descriptor Language

> Third-party SOAP Integration Toolkits

Apache AXIS 1.3

MS Visual Studio (.NET Framework 2.0)

SOAP Sonar (For educational purposes)

– Download from www.crosscheck.com


XOG Via WSDL

> How to access WSDL Definitions?

http://clarityserver/niku/wsdl

> Types of WSDL

Object - XOG

Invoke action (Can invoke a process definition)

Query (NSQL to read data)


XOG Via WSDL

> Export data using NSQL

Every NSQL query has a WSDL

Filterable

Only export what‟s needed

Efficient fast performance


> Using XOG in a high level language like Java

It would exceed this presentation to show the necessary

coding for Java, therefore we will only describe the steps

which have to be implemented:

– A SOAP connection has to be established

– With a XOG login you get a session id

– Create a XML document with a tool like JDOM

– Invoke XOG with the session id and the XML document

– Close the XOG connection


XOG Via Java

> Some things to remember:

Not everything in Clarity is XOGable, so when

migrating data it is important to design a “migration

plan”.

Dependencies will „read‟ out and appear in the output

„write‟ file.

– For example, if you are XOGing out an object and in

that object is an active attribute that uses a certain

lookup, both the active attribute and the lookup will

be Xogged out and appear in the „write‟ output file.


XOG Tips

> In addition…

Always XOG between like-to-like Clarity environments

– You cannot XOG „read‟ out of Clarity 8.1 FP03 and „write‟ into

Clarity 12.0.1

Keep your XOG Client current even after SP upgrades

When you perform a „read‟ using XOG, the output file is

formatted to be a „write‟ xml file. This means that if

you XOG out a project, the output xml file will be XOG-

ready as a write xml file.


XOG Tips

> Special Characters…

Users will frequently copy/paste from text editors which

allow special formatting (such as MSWord) into Clarity text

fields. If any of these characters are outside of the basic

ASCII character set used by XML , the XOG read and/or

write will fail with an “invalid XML” error message. But

unfortunately will not tell you where your invalid XML is…

> Wrong XOG permissions

> Inconsistent XOG and/or Clarity versions


Common Issues

> For further and more detailed reading enjoyment…

Clarity Project & Portfolio Manager Integration Guide

Review KB articles by searching CA support site.

Keywords = Clarity XOG

TEC482079 (General Migration Guidelines)

Example Script:


Documentation

Community Assistance

> http://caforums.ca.com/ca/

Clarity is the MOST ACTIVE CA board

> XOG/GEL/WSDL is the MOST ACTIVE topic on CA Clarity

forums

> “How to” questions are answered by community experts

and Clarity Support/Services/Development

> Lots of sample code shared


http://caforums.ca.com/ca/

Questions?

PPM - All About Integration - casupport.broadcom.com · SOAP is a industry standard, describing the exchange of XML based messages in heterogeneous environments SOAP is a remote procedure

Documents