CVSQL 2 The Design. System Overview System Components CVSQL Server –Three network interfaces –Modular data source provider framework –Decoupled SQL parsing.

CVSQL 2

The Design

System Overview

System Components

• CVSQL Server– Three network interfaces

– Modular data source provider framework

– Decoupled SQL parsing and execution

• Client utility– Uses new XML-RPC network interface

• JDBC driver– Uses updated CVSQL2 XML network interface

CVSQL Server

CVSQL Server• Networking component

– CVSQL1 and CVSQL2 XML protocol support

– XML-RPC protocol support

• SQL parser– Parses the SQL query string

• SQL engine– Executes the query

• Data Provider modules– CVS log data provider

– Palantir event provider

• Server configuration and control

Networking

• CVSQL XML protocol support– Legacy software support

– CVSQL2 extension – “protocol 2.0”• response byte count report

• used only on client's explicit request

• XML-RPC support– Uses available third-party XML-RPC C++ implementation

(XmlRpcPP)

• Desired network interface selection is done via configuration file– Only one network interface active at any moment

SQL Parser

• Completely rewritten from ground-up

• Modular– Configurable lexical analyzer

– Recursive-descent syntax parser

• Validates the string representation of SQL query and transforms it into SQLQuery object

• SQLQuery object contains:– SQLQuerySelection – description of selected columns

– SQLQueryTables – a list of data sources for the query

– SQLQueryExpression – a selection filter for the data

– GroupBy, OrderBy – data ordering and grouping

SQL Engine

• Executes the SQL query

• Works on ResultSet objects– join, filter

• The result is also a ResultSet object

• Supported data types:– String, DateTime, Integer, Decimal, Null

• Database independent– Table list and structure provided by the SQLDatabase object

Data Providers

• Gather data from external sources– CVS log, Palantir event log

• Present the data as ResultSet objects– Unnamed objects – only the SQLDatabase object knows table

names and their references to table providers

• Have intimate knowledge of data structure– data fields, column types

• Register upon program start– module type, reference to object factory

• Created by database separately for each table

• Take additional options to control data gathering

Server Configuration

• Server configuration file example:

<?xml version="1.0"?>

<cvsql>

<authentication>

<user login="jdoe” info="John Doe" pwd="1f6fc7c0d324b0e4a7d7c278e2b3a5f4"/>

</authentication>

<server type="cvsql" port="9090"/>

<!-- <server type="xmlrpc" port="80"/> -->

<databases>

<database path="/etc/cvsql/db/cvsql2.xml"/>

</databases>

</cvsql>

Database Configuration

• Database configuration file example:

<?xml version="1.0"?>

<cvsql_database name="cvsql2" log="/var/log/cvsql/cvsql2.log">

<users>

<user login="senko"/>

</users>

<table name="users" type="cvsql" path="/var/lib/cvs/cvsql2/">

<option name="table" value="users"/>

<option name="ttl" value="60"/>

<option name="cvs" value="/usr/bin/cvs"/>

</table>

</cvsql_database>

The Client

• Simple command-line utility

• Operation:1) Get the query string input from the user

2) Send the query to the server using XMLRPC

3) Obtain the result in XML format and present it back to the user

● Uses third party XML-RPC library (XmlRpcPP)

JDBC Driver

• CVSQLAPI– Contains all I/O logic and XML parsing – Send() CVSQLServerResponse – Throws CVSQLException

• CVSQLException for I/O or general error

• CVSQLProtocolException for XML errors .

• CVSQLServerResponse– Encapsulates the three important XML

elements • resp / result • definition

• values

JDBC Driver (cont'd)

• Handling XML Chunks– CVSQL XML protocol sends chunks, not

documents – Multiple root elements

• resp for non-data/error

• db for row-data

– Proposed solution • Protocol "upgradability"

• InputStream wrapper

JDBC Driver (cont'd)

• Protocol Upgradability– Server assumes v1.0 protocol for compatibility – Client requests protocol upgrade

• Sends "protocol 2.0" command

• Occurs at any time

• Protocol switch takes effect AFTER server's response

JDBC Driver (cont'd)

• XML Protocol 2.0– Root XML elements prefixed with

• XML element byte length in ASCII decimal • ASCII newline (0x0a)

– Example >>> protocol 2.0 <<< <resp>

<number>0</number> <text>OK</text>

</resp> >>> login user <<< 41

<resp> ...

</resp>

JDBC Driver (cont'd)

• InputStream Wrapper– XMLElementInputStream extends InputStream – Knowledge of expected XML chunk length

(data-length) – Remembers bytes read from input stream

since last reset() (read-bytes) – Behavior

• InHeader read() returns bytes from the XML processing header (<?xml version=...)

• InData && read-bytes < data-length read() calls are passed directly

• InData && read-bytes >= data-length read() returns EOF (-1)

JDBC Driver (cont'd)

• CVSQLResultSet – Version 2.0 adds constructor for static

ResultSet, not created from server response – Allows for use within CVSQLDatabaseMetaData

and CVSQLResultSetMetaData classes • CVSQLDriver

– Accepts URLs containing sub-protocol "cvsql" – Example

jdbc:cvsql@server:port

– Version 1.0 code needs to be verified safe with malformed URLs

JDBC Driver (cont'd)

• CVSQLStatement – Result set data is obtained during the call to execute. – In version 1.0, protocol exceptions are not detected until

a call to getResultSet – In version 2.0, exceptions are thrown in execute for bad

queries as well as I/O errors .

• CVSQLDatabaseMetaData – Version 1.0 used static values for limited meta-data

support– Version 2.0 uses the CVSQL Server v2.0 DESCRIBE

command to obtain database info – Limited only by data returned by DATABASE query

JDBC Driver (cont'd)

• CVSQLConnection – Version 1.0 needs review and rework – Work carefully with the Properties object to

avoid RuntimeExceptions

Project Status

• Requirements defined– except for Palantir – still in the research phase

• System design completed

• We're ready to code– work already started on SQL parser and engine

• Overall status: On Track! ;-)

• Questions?

Thank you

CVSQL 2 Team