Top Banner
© 2009 by Actuate, IBM, and Sybase, Inc., made available under the EPL v1.0 | March 23, 2009 Using and Extending the Data Tools Platform Brian Fitzpatrick (Sybase) Linda Chan (Actuate) Brian Payton (IBM)
73

DTP

Feb 01, 2016

Download

Documents

gpzunino

Using and Extending the Data Tools Platform
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: DTP

© 2009 by Actuate, IBM, and Sybase, Inc., made available under the EPL v1.0 | March 23, 2009

Using and Extending the Data Tools Platform

Brian Fitzpatrick (Sybase)

Linda Chan (Actuate)

Brian Payton (IBM)

Page 2: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Welcome!

Goals for this tutorial:Get a working knowledge of DTP ToolingGet a glimpse into how to enable or extend database support for a particular vendorGet familiar with some DTP APIsUse ODA to extend DTP for a non-database data sourceSee how to expand the SQL Query Model and Parser for a particular DBAnd finally see how DTP is being leveraged in other Eclipse projects

Page 3: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Before we begin…

What do you hope to get out of the talk today?

What are your goals? What do you know already about DTP?

Page 4: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Beginning DTP

What is the Data Tools Platform (DTP)?

Page 5: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Important Concepts in the Tooling

Connection Profiles & Driver Definitions

Page 6: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Available Perspectives for DTP

Database Development

Database Debug

Page 7: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Basic Components

Data Source Explorer (DSE)

SQL Scrapbook

SQL Results View

Page 8: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Creating a Database Connection in DTP

In the Data Source Explorer, right-click on “Database Connections” and select “New…”Specify your database type, click NextIf no driver available, click the New Driver Definition button

Select a driver template, set your jar paths, click OKCustomize database properties, click FinishRight-click on new Connection Profile and select “Connect”

Page 9: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Expanding and Exploring the Connected Profile

Once you connect to a profile, it starts to load the model a level at a time

So you can expand and see catalogs, schemas, tables, procedures, and so on – to represent what your database (and the DTP implementation for that database) supports

Page 10: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Using the SQL Scrapbook to Execute Queries

Right-click on a database Connection Profile and select “Open SQL Scrapbook”

Type your query, select it, and select “Execute Selected Text”

Page 11: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

How to Use the SQL Query Builder from the SQL Scrapbook

To create more complicated queries, you can use the SQL Query Builder (SQB), which simplifies query creation by allowing you to click, drag, and drop to customize the SQLTo open the SQB, right-click and select “Edit in SQL Query Builder”Right-click where it says “To add a table…” and select “Add Table…”Select one or more tables touse in the query and click OKThen you can select columns, specify conditions, and so on

Page 12: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Executing a Query from the SQB

Once you’ve customized your query in the SQB, right-click in the SQL text area at the top and select “Run SQL” to show the results

Page 13: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Interacting with Stored Procedures and UDFs

In SQL, you can define and call stored procedures in user-defined functions if your database supports them. DTP allows you to call them as well.

In a Derby database, you can call the SQLTABLES procedure:call SYSIBM.SQLTABLES(null, 'APP', null, null, null);

Page 14: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Stored Procedures and UDFs continued…

You can also create and then use user-defined functions…

In a Derby database, try this:create function getSystemProperty(name varchar(128)) returns varchar(128) language java external name 'java.lang.System.getProperty' parameter style java no sql;

Page 15: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Creating Tables and Modifying Content

In addition to simply running SQL, you can use DTP tools to create tables and view/modify their data

In the Data Source Explorer, connect to and drill down into a database connection profile. On the “Tables” folder in the tree, right-click and select “New Table…”

The New Table wizard creates DDL that you can then execute to create the table.

Page 16: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Creating Tables and Modifying Content, continued

Execute the table DDL by right-clicking and selecting “Execute All” – this will run the SQL and create the table.

Then right-click on the “Tables” node in the tree, right-click and select “Refresh” to repopulate the table list.

Once you have a table, you can edit its content by right-clicking and selecting “Data->Edit”, which opens a tabular window for editing table data that you can use for data entry

When you’re done, right-click in the editor and select Save from the popup menu to persist your changes

Page 17: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Creating Tables and Modifying Content, continued

Another way of loading data is by CSV file. To load a table froma CSV file, right-click on a table and select Data->Load…

Specify the input file, the delimiter, and the string delimiter, and click Finish.

Page 18: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Tooling Exercise…

Set up a new Derby connection profile and connect to a local database

Connect to the connection profile

Create a new Product table with three columns – product_id (varchar), name (varchar), inventory (number)

Create a new Orders table with three columns – order_id (varchar), date (date/time), product_id (varchar), number_ordered (number)

Import data from the Products.csv and Orders.csv files into your new tables

In the SQL Scrapbook, create a query (in the text editor or the SQB) to correlate information from the Product and Orders tables to get a readable report of what products were ordered on a given date (product name, date, number ordered)

Page 19: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Extending DTP

Now that you have a bit of knowledge about what DTP can do for you… How do you leverage and extend it?

A few possible avenues…Use DTP components in your own applications and simply leverage what’s already thereExtend DTP into new Data Sources and Databases (“enabling”new databases, adding custom items to the tree, adding custom SQL syntax to what DTP already knows, etc.)Take advantage of DTP APIs to utilize core DTP functionality under the covers

Page 20: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

“Enabling” a New Database

Rather than spend a long time on this topic, I’ll direct you to a series of blog posts I wrote while I worked through “enabling” SQLite for DTP Enablement. Not all databases will require all of this, but it’s not difficult to extend the existing frameworks

Driver Framework: http://fitzdtp.blogspot.com/2008/06/dtp-driver-framework-repost.htmlCustom Driver Template: http://fitzdtp.blogspot.com/2008/06/how-do-you-add-your-own-custom-driver.htmlAdding a Custom Catalog Loader: http://fitzdtp.blogspot.com/2008/08/dtp-sqlite-support-continued-on-to.htmlCreating the SQLite Connection Profile (minus UI): http://fitzdtp.blogspot.com/2008/08/creating-actual-sqlite-connection.htmlCreating the SQLite Connection Profile UI Bits: http://fitzdtp.blogspot.com/2008/08/creating-sqlite-connection-profile-ui.html

Page 21: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Leveraging DTP APIs

Though we have many cool tools in DTP, you may find that you don’t need or want to use them. In these cases, you may still be able to take advantage of DTP APIs.

There are too many APIs to really talk about in the time allowed, so I’m just going to hit upon a few of the high-level ones

In this section, we’re going to use DTP APIs to create a connection profile and write a utility class that takes a file, loads it into memory, and loads a table from the file

Consider that APIs could also be used to map/replicate data fromone database to another via a combination of JDBC and DTP

Page 22: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

APIs for Creating Connection Profiles…

There are two types of connection profiles…persisted and transient.

Page 23: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Accessing Existing Connection Profiles (Persisted)

Access to existing connection profiles (persisted or transient) is done using the ProfileManager class (org.eclipse.datatools.connectivity.ProfileManager)

If you know the name of your connection profile, you can find itpretty easily

IConnectionProfile myProfile = ProfileManager.getInstance().getProfileByName(“MyProfile”);

Page 24: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Creating a Transient Connection Profile

In the second case, where you may not want to use the whole connection profile UI framework but want to leverage other components, you use the Transient API

These are the typical JDBC properties you think of…String profileUserName = "";String profilePassword = "";String profileURL = "jdbc:derby:C:\\DerbyDatabases\\MyDB;create=true";String profileDriverClass = "org.apache.derby.jdbc.EmbeddedDriver";String profileJarList = "C:\\Derby10.2.2\\db-derby-10.2.2.0-bin\\lib\\derby.jar";

Then there’s the DTP-specific properties you need…String providerID =

"org.eclipse.datatools.connectivity.db.derby.embedded.connectionProfile";String vendor = "Derby";String version = "10.1";

Page 25: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Transient Profile, continued

So once you have your properties, you can do this:Properties baseProperties = new Properties();

baseProperties.setProperty( IDriverMgmtConstants.PROP_DEFN_JARLIST, jarList );baseProperties.setProperty(IJDBCConnectionProfileConstants.DRIVER_CLASS_PROP_ID,

driverClass);baseProperties.setProperty(IJDBCConnectionProfileConstants.URL_PROP_ID, driverURL);baseProperties.setProperty(IJDBCConnectionProfileConstants.USERNAME_PROP_ID, userName);baseProperties.setProperty(IJDBCConnectionProfileConstants.PASSWORD_PROP_ID, password);baseProperties.setProperty(IJDBCConnectionProfileConstants.DATABASE_VENDOR_PROP_ID,

vendor);baseProperties.setProperty(IJDBCConnectionProfileConstants.DATABASE_VERSION_PROP_ID,

version);baseProperties.setProperty( IJDBCConnectionProfileConstants.SAVE_PASSWORD_PROP_ID,

String.valueOf( true ) );ProfileManager pm = ProfileManager.getInstance();IConnectionProfile icp = pm.createTransientProfile(providerID, baseProperties );

Page 26: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Transient Profile, continued…

So the upside to the Transient profile is that you don’t have to pre-define it in the UIThe downside is that you have some additional properties to figure out.

This is *new* API for Galileo. There are other changes to help with getting the correct vendor and version in Galileo so you can hand it a JDBC connection you managed yourself, it gets the properties it needs, and then returns the correct vendor, and version.For now, you can use the provider ID of the Generic JDBC profile(org.eclipse.datatools.connectivity.db.generic.connectionProfile)

Page 27: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Using your Connection Profile reference

Now that you have an IConnectionProfile instance, what can you do with it?If it’s not connected, you can connect to it:

IConnectionProfile profile = ProfileManager.getInstance().getProfileByName("myprofile");

IStatus status = profile.connect();if (status.equals(IStatus.OK)) {

// success} else {

// failure :(if (status.getException() != null) {

status.getException().printStackTrace();}

}

If you get an exception, you can react accordingly…

Page 28: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Using your Connection Profile reference…

So now that you have a connected profile, you can get the JDBC connection from the ProfileConnectionManager:

public java.sql.Connection getJavaConnectionForProfile(IConnectionProfile profile) {

IConnection connection =ProfileConnectionManager.getProfileConnectionManagerInstance().getConnection(profile, "java.sql.Connection");

if (connection != null) {return (java.sql.Connection) connection.getRawConnection();

}return null;

}

Page 29: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Using the Connection…

From there, you can do anything you might have done with any other JDBC connection:

try{Statement st = connection.createStatement();String statement = “select * from mytable”;int val = st.executeUpdate(statement);

}catch (SQLException s){

// exception}

Page 30: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Other good API bits…

You can also get the SQL model from a database connection. First, you have to get the ConnectionInfo class, which acts as a bridge between the JDBC connection and the SQL models

IManagedConnection managedConnection = ((IConnectionProfile)profile).getManagedConnection("org.eclipse.datatools.connectivity.sqm.core.connection.ConnectionInfo");

if (managedConnection != null) { try {

ConnectionInfo connectionInfo = (ConnectionInfo) managedConnection.getConnection().getRawConnection(); if (connectionInfo != null) {

Database database = connectionInfo.getSharedDatabase();// do something with the database reference…

}} catch (Exception e) {

e.printStackTrace();}

}

This is how to get the SQL Model’s root Database node for your connection profile

Page 31: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Using the Database reference…

Once you have the Database reference, you can drill into your database and get the abstracted SQL structure, which you can then use to develop more flexible SQL calls for a variety of reasons…

Imagine being able to connect to two databases and providing a tool that allows you to map data between disparate tables…Imagine writing tooling or a set of APIs that could be used to inject data via DTP’s abstracted modelsImagine abstracting a data source class for a given table or set of tables that would hide SQL complexity and allow a developer to more easily add/remove/edit data via simple callsThe sky’s the limit!

Page 32: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

API Exercise…

Using the base plug-in provided, in the action’s run method, dive into the connection profile and print a few things to the console for the Product table in the App schema of our Derby database…

MyDatabase.getCatalogs() returns an EList of catalogs. Derby has a dummy Catalog object, so you’ll have to get it and then go deeper ( (Catalog)MyDatabase.getCatalogs().get(0) )MyCatalog.getSchemas() returns an EList of schemas you can iterate through until you find APPOnce you find the Schema, MySchema.getTables() returns an EList of tables you can iterate through until you find ProductMyTable.getColumns() returns an EList of columns…And so on… simply work down the hierarchy and show a list of the columns in the Product table

Page 33: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

In Summary… Tools + API = Flexibility

DTP’s tooling provides useful database and data source functionality that’s also extendable and reusable in your own code

And DTP’s APIs allow you to go deeper than the tooling to do many cool things with JDBC, the SQL model, and your own tools/APIs

Page 34: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

More DTP Information

Main DataTools site at http://www.eclipse.org/datatoolsNewsgroup - news://news.eclipse.org/eclipse.dtpMailing list – http://dev.eclipse.org/mailman/listinfo/dtp-devWiki - http://wiki.eclipse.org/index.php/Data_Tools_Platform_ProjectBugzilla for bug or enhancement requests

Feel free to ask questions!

Page 35: DTP

© 2009 by Actuate, IBM, and Sybase, Inc., made available under the EPL v1.0 | March 23, 2009

DTP Open Data Access (ODA) Framework

Extend DTP for Non Relational Database Data Sources

Linda Chan – Actuate® Corporation

Page 36: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Topics

What is Open Data Access1 (ODA)?

Overview – ODA Framework

Enabling non-RDBMS data source with an ODA Data Provider

Demo/Exercise: creating ODA provider plug-ins

1 Actuate® patent pending

Page 37: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Open Data Access Framework

Enables data access to any data source• Provides an abstraction for accessing heterogeneous data

sources• Highly scalable data retrieval and easy end-user experience• Built using familiar Eclipse extension points

Extends & customizes Applications Systems• Applications can provide custom data driver and query builder

for accessing their data

Provides proven framework• BIRT’s built-in ODA data connectors• Commercial products’ ODA data connectors

Page 38: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Overview – Open Data Access Architecture

ODA Data Driver

ODA Query Builder

Application Client ToolsBIRT Report Designer

Data Sources

Application Server

ODA RuntimeInterfaces

ODA DesignInterfaces

Data Access & QueryDefinition

Complete control of data access and branding

Run-Time Data AccessDesign-Time Query Builder

Complete control of data access and branding

Page 39: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Overview – Open Data Access Framework

Run-timePublic API defines a set of Java™ interfaces that allow heterogeneous data to be accessed in a generic way

Design-timeAllows data source-specific user interface to be embedded in any ODA-compliant consumer application

ODA Consumer HelperBuilt-in support for any ODA-compliant consumer application to consume data from any ODA data provider

Page 40: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

ODA Run-time Framework

Stable run-time API since DTP 0.9

Integrates with DTP Connection Profile and JDBC™ Databases framework

Implementation wraps data source-specific APIs

Run-time plug-in implementsextension points

org.eclipse.datatools.connectivity.oda.dataSourceorg.eclipse.datatools.connectivity.connectionProfile

ODA run-time API interfacesorg.eclipse.datatools.connectivity.oda package

Page 41: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

ODA Run-time API Interfaces

Defines the primary run-time operations of an ODA data provider to access and retrieve data from a data source

Java interfacesJDBC-like, extended to support additional capabilities of non-RDBMS data sources Emphasis on scalable data retrieval

Main run-time operationsData Source Connection – IConnection

Establishes a live connection to any type of data sourceObtains provider capabilities for each type of data set queriesCreates one or multiple concurrent data set-specific queries

Page 42: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

ODA Run-time API Interfaces

Main run-time operations (cont’d)Data Set Query – IQuery, IAdvancedQuery

Prepares and executes a data set-specific query text command, e.g. XPath, MDX, SQL, Stored Procedure callsHandles one or more sets of data rows, i.e. result sets, retrieved by a single data set query

Query Parameters – IParameterMetaData, IParameterRowSet

Provides run-time metadata of parameters specified in a prepared queryHandles scalar and complex input/output parameters

Result Sets – IResultSet, IResultSetMetaData

Fetches tabular data rows Allows sequential or concurrent access to multiple result sets

Page 43: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

ODA Design-time Framework

Eclipse Modeling Framework (EMF) model-based design-time interfaces

communicate connection properties, query and parameter definitions to an ODA consumer application

Integrates with DTP Data Source Explorer view

Page 44: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

ODA Design-time Framework

ODA Consumer Designer (e.g. BIRT Data Source Wizard) Customized Wizard Page

contributed by ODA Data Providers

ODA Design Session model

Page 45: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

ODA Designer UI Plug-in

Implements extension points

org.eclipse.datatools.connectivity.connectionProfileorg.eclipse.ui.propertyPagesorg.eclipse.datatools.connectivity.oda.design.ui.dataSourceorg.eclipse.datatools.connectivity.ui.connectionProfileImage(optional)

customizes ODA Designer UI pagesorg.eclipse.datatools.connectivity.oda.design.ui.wizardspackage

Communicates its Data Source and Data Set design definitions in an ODA Design Session model

Page 46: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

ODA Design Session model

Model based on Eclipse Modeling Framework (EMF)org.eclipse.datatools.connectivity.oda.design package

Allows customized data source and query builders to design the slice of data to access at run-time

Communicates connection information, query and parameter definitions to an ODA consumer applicationTransient Objects

root element: org.eclipse datatools.connectivity.oda.design.OdaDesignSession

ODA design-time consumer application, e.g. BIRT Report Designer, initiates an ODA design session

Consumes a data provider’s UI page contributionsAdapts an edited data access design to host-specific design components Provides persistent services for editing designs

Page 47: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Build a Custom ODA Data Provider

From scratchPDE New Plug-in Project Wizard

Demo / Exercise

Extends an existing ODA providerdriverBridge extension point

org.eclipse.datatools.connectivity.oda.consumer.driverBridge

Page 48: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Build a Custom ODA Data Provider – from Scratch

PDE New Plug-in Project WizardDemo / Exercise

Page 49: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Extends an Existing ODA Data Provider

Modify/enhance behavior of an existing ODA data providerorg.eclipse.datatools.connectivity.oda.consumer.driverBridge extension point

Ideal for minor enhancementspreserves existing ODA extension IDSupports “chained” bridges

Page 50: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

DriverBridge Extension Point

ODA Consumer Application(e.g. BIRT)

ODA Run-time Driver

ODA RuntimeInterfaces

oda.consumer.helper

Driver Bridge

driverType driver class name or first matching interface name

Driver BridgebridgeId oda.dataSource extension id

Page 51: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Enhance an Existing ODA Data Provider

DriverBridge extension implementsExtension Point

org.eclipse.datatools.connectivity.oda.consumer.driverBridge

Sample driverBridge extension element<bridge

driverType="org.eclipse.birt.report.data.oda.jdbc.OdaJdbcDriver"bridgeId="org.eclipse.birt.report.data.testjdbc">

</bridge>

Bridge driver is handed its underlying driver instanceIDriver.setAppContext( Map )

Key: OdaDriver.ODA_BRIDGED_DRIVERValue: underlying driver instance

May use own API between Bridge and underlying Driver instances Sample code attached in Bugzilla 135246

Page 52: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

ODA Data Providers

Out-of-the-Box ODA Data ProvidersCSV data file

org.eclipse.datatools.connectivity.oda.flatfile* plug-insXML data

org.eclipse.datatools.enablement.oda.xml* plug-insWeb Services

org.eclipse.datatools.enablement.oda.ws* plug-insJDBC Drivers – SQL Query Builder & SQL Textual Editor

org.eclipse.birt.report.data.oda.jdbc* plug-ins

DTP Incubation ProjectECore ODA Data Provider (Bugzilla 132958)

org.eclipse.datatools.enablement.oda.ecore* plug-insSeparate incubator download package

Welcomes contribution

Page 53: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

What is New in ODA – DTP 1.7 Release

ODA API EnhancementsJava Object data typeAvailability of Data Set Property and Input Parameter Values prior to Query PreparationCancel a Query ExecutionConnection-level Locale SettingConsumer Resource Identifiers

http://wiki.eclipse.org/Galileo_DTP_ODA_API_Changes_Spec

Dynamic Query Result Specification API – Experimental APIFiltering Result ProjectionRow Ordering

Page 54: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Resources

ODA Overview documentlink on DTP Connectivity home page

http://wiki.eclipse.org/Connectivity

Developers' Guide for Connectivity FrameworksDTP Help Contents – Javadoc™ and Extension Points’ SchemaPresentation slides

https://www.eclipsecon.org/submissions/2009/view_talk.php?id=686

Page 55: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Open Data Access Framework

Discussions, Feedbacks, Q&As

Page 56: DTP

© 2009 by Actuate, IBM, and Sybase, Inc., made available under the EPL v1.0 | March 23, 2009

Using and Extending the Data Tools Platform

SQL Query Model and Parser

Brian Payton (IBM)

Page 57: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Agenda

Introduction and backgroundCharacteristicsWhat is it good for?Tour of the Query Model and Parser components

Query ModelQuery ParserParser FactoryPost-parse ProcessorsSQL Source WriterQuery Model Helpers

Extending the Query Model and ParserFun with the interactive tester

Page 58: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Introduction and background

The Data Studio Query Model and Parser is a system for representing, analyzing, manipulating, and generating SQL query statements in client-side database applications.

Originally created for the SQL Assist component of IBM® DB2 Control Center, later used in IBM® WebSphere® Studio Application Developer, IBM Rational® Application Developer, and IBM Data Studio Developer.

Donated by IBM to the Eclipse Data Tool Platform project 2006 and used in the DTP SQL Query Builder component.

Page 59: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Characteristics

Deep coverage of SQL DML statements (Query, Insert, Update, Delete)But DML only!Extensible, componentized system

Extension points to register extensions for different databasesParser grammar inheritance

Model is based on EMF technologyParser is based on LPG (LALR Parser Generator) technology

SourceForge project, but available in Eclipse through Orbit

Think of the model as an SQL AST (Abstract Syntax Tree) on steriods!

Page 60: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

What is it good for?

Provides underlying support for the SQL Query BuilderMakes “round-trip” editing possible

Can do SQL Syntax validation outside of the databaseEnables query analysis

Page 61: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Tour of the Query Model and Parser components

Query ModelRepresent the essential structure of a query statement

Query ParserValidate SQL source syntax and generate the query model

Parser FactorySet of methods used by the parser to generate the model

Post-parse ProcessorsValidate semantics and modify the model after parsing

SQL Source WriterGenerate SQL source from the model

Model HelpersSet of helpful methods to analyze or modify the model

Page 62: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Query ModelRepresents the essential syntactic structure of a query statement as an object model

Covers DML statements

SELECT, INSERT, UPDATE, DELETE

Implemented using EMF (Eclipse Modeling Facility)

Derived from object model diagrams created using Rational Rose

Extends and complements the SQL ModelSQL Model is the foundation for the Data Studio tool setSQL Model models the database catalog while the Query Model models database statements

Page 63: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Query Model - plugins

org.eclipse.datatools.modelbase.sql.queryBase query model

org.eclipse.datatools.modelbase.sql.xml.querySQL/XML extension

Depends on:org.eclipse.datatools.modelbase.sql

Page 64: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Query Model – EMF Model

Java code is generated directly from the Rose model using EMF (Eclipse Modeling Facility)

Each Rose diagram class becomes a Java class

Model classes extend EMF EObject class

EMF handles details of getters, setters, collections, notifications

Each Rose diagram class becomes two EMF classes: an interface class and an implementation (Impl) class

Generated code can be customized, and changes won't be lost when regenerated

Page 65: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Query Parser

Parses a SQL DML statement and generates an instance of the Query Model

Implemented using LPG (LALR Parser Generator) from IBM Research

Parser is generated from BNF-like grammar files

Like the Query Model, is actually an extendable set of parsers: base SQL, SQL/XML

Base SQL query grammar derived from ISO SQL 2003

Page 66: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Parser Factory

Set of methods used by the parser to create a Query Model instance

Each parser plug-in has a parser factory class

Used in the “action” clauses of the LPG grammar file rules

In turn the parser factory methods call model factory methods tocreate individual model elements

Model factory code is generated by EMF, parser factory methods are hand-coded

Page 67: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Post-parse Processors

Query Parser provides a mechanism to allow clients to modify a generated Query Model after the Parser Factory creates it but before it is returned as a parse result“Post-parse processors” are attached to the parser, register what model elements they are interested in, and after the parse get achance to modify the modelTwo default post-parse processors are provided:

DataTypeResolverTries to determine datatypes of value expressions in the model using available information

TableReferenceResolverReplaces place-holder table references in generated model with table objects from the SQL Model

Page 68: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

SQL Source Writer

Generates SQL source from a Query Model instanceTogether with the Query Parser, provides “round-trip” capability from SQL to model and backLike the model and parser, is configurable and extendableWalks the model tree and uses reflection to find the right source writer method for each model element

Page 69: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Query Model Helpers

Query Model Helpers are a set of classes and methods that provide useful utility functions for the Query Model

DataTypeHelperJoinHelperStatementHelperTableHelperValueExpressionHelper

Used both for analyzing and manipulating the model. Examples: Find information, such as all table references or all effective result columnsModify the model, such as remove a table and all column references for that table from a query

Packaged in org.eclipse.datatools.modelbase.sql.query

Page 70: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Extending the Query Model and Parser

Process:Analyze the grammar feature you want to add. Is it specific to one database type, or common to several?Modify the appropriate to add the new feature

Getting this right is not easy! Modeling is hard work.Regenerate the EMF modelAdd Parser Factory methods for the new model elementsModify the appropriate parser grammar to cover the new syntaxUpdate the SQL Source Writer to generate SQL for the new syntaxUpdate the Query Model Helper classes as neededUpdate the JUnit tests for the parserMake sure it all works!

Page 71: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Fun with the interactive tester

Along with JUnit tests, the test plugins for the Query Parsers include an interactive testing tool

In org.eclipse.datatools.sqltools.parsers.sql, it's SQLQueryParserInteractiveTest

A command-line tool, works with the Eclipse console viewType in a SQL statement, and it displays the generated model elements, then the SQL generated back out from the model

Page 72: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

DTP Adopters

Eclipse ProjectsBusiness Intelligence and Reporting Tools (BIRT)

SQL Query BuilderDatabase EnablementConnection Profile ManagementOpen Data Access

http://www.eclipse.org/birtWeb Tools Project (WTP) – EJB or Dali JPA Tools development

http://www.eclipse.org/webtools

Other Adoptershttp://www.eclipse.org/datatools/community.php

Page 73: DTP

DTP Tutorial | EclipseCon 2009 | © 2009 by Actuate, IBM, and Sybase, Inc.; made available under the EPL v1.0

Legal Notices

Actuate is a registered trademark of Actuate Corporation and/or its affiliates in the U.S. and certain other countries.

IBM, Rational, Rational Rose, and WebSphere are registered trademarks of International Business Machines Corporation in the United States, other countries, or both.

Sybase and Adaptive Server are registered trademarks of Sybase, Inc. or its subsidiaries.

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.