CMPE 226 Database Systems October 28 Class Meeting Department of Computer Engineering San Jose State University Fall 2015 Instructor: Ron Mak mak.

CMPE 226

Database SystemsOctober 28 Class Meeting

Department of Computer EngineeringSan Jose State University

Fall 2015Instructor: Ron Mak

www.cs.sjsu.edu/~mak

http://www.cs.sjsu.edu/~mak

Computer Engineering Dept.Fall 2015: October 28

CMPE 226: Database Systems© R. Mak

2

Extract, Transform, and Load (ETL)

http://uploads.webflow.com/54e3cc87305a0f0a0665f71f/5537351f5f1065d401cde83c_etl-elt-architecture.jpg





3

Extract, Transform, and Load (ETL), cont’d

http://www.stonebranch.com/common/images/etl-tipping-point.jpg





4


http://superdevelop.com/wp-content/themes/shopperpress/thumbs/ETL.jpeg





5


http://edpflager.com/wp-content/uploads/2014/05/ETL.png





6

Data Virtualization

An approach to data management that allows an application to retrieve and manipulate data without requiring technical details about the data, such as how it is formatted or where it is physically located.

An agile data integration approach organizations use to gain more insight from their data, respond faster to ever changing analytics and BI needs and save 50-75% over data replication and consolidation.

https://en.wikipedia.org/wiki/Data_virtualization

http://www.compositesw.com/data-virtualization/







7

Data Virtualization, cont’d

Differs from traditional ETL. The original data remains in place. A real-time process accesses the data sources.

Reduces the risk of data errors. Reduces the workload of extracting,

transforming, and loading data that may never be used by an application.

Provides applications a “virtual view” of the data. Applications can treat the disparate and

heterogeneous data as a single relational database.



8

Data Virtualization Functionality

Abstraction Abstract the technical aspects of stored data,

such as location, storage structure, API, access language, and storage technology.

Virtualized data access Connect to different data sources and make them

accessible from a common logical data access point.

Transformation Transform, improve quality, reformat, etc.

source data for consumer use.https://en.wikipedia.org/wiki/Data_virtualization





9

Data Virtualization Functionality, cont’d

Data federation Combine result sets from across

multiple source systems.

Data delivery Publish result sets as views and/or data services

executed by client applications or users when requested.






10

Cisco Information Server (CIS)

The Cisco Information Server (CIS) forms the core of the Cisco Data Virtualization Platform.






11

Install the CIS Studio

Studio is the client-side application that connects to the CIS server. It’s configured to run only on Windows. Even though it’s a Java application.

Download and install Studio. See http://www.cs.sjsu.edu/~mak/CMPE226/CIS Unzip the 2.54 GB zip file, which includes both the

CIS server and the client Studio app. Install only Studio. See the Installation Guide.

http://www.cs.sjsu.edu/~mak/CMPE226/CIS

http://www.cs.sjsu.edu/~mak/CMPE226/CIS



12

CIS Team Accounts

Each team has a CIS account.Team User

namePassword

Alpha Squad alpha alpha@226

Apollo apollo apollo@226

Blade Runner blade blade@226

Data Crunchers cruncher cruncher@226

Database Predators

predator predator@226

Data Nerds nerd nerd@226

Elite elite elite@226

GangOfFour gang gang@226

Oceans3 ocean ocean@226

Sappers sapper sapper@226

Snowlions snow snow@226

Swagg swagg swagg@226

Team Include include include@226



13

Logging into CIS

To log in, start the client-side Studio application.

Domain: compositeServer: 130.65.159.87 Port: 9400

Windows only!



14

Demo: Cisco Information Server

This demo is described in the Getting Started Guide.

Query: Why did customer Landmark Systems not receive product Widget 5 on the promised date of February 10, 2003?



15

Demo: Cisco Information Server, cont’d

ViewOrder View the activities of an order. Obtain product status, order date,

and customer contact information.

ViewSales Obtain information about the purchase department’s

interactions with the customer and the supplier.

ViewSupplier Use the sales department’s record

and noted the lead time for product delivery.



16

Assignment #8

Each team do the CIS demo described in the Getting Started Guide.

Create your sources folder under My Home, not under Shared. You can named the sources folder anything you like. Each team member can have a separate folder.

Generate the following screen shots.



17

Assignment #8, cont’d



18




19




20




21


Tips

On page 19 of the Getting Started Guide, after you’ve clicked the Create & Introspect button, you may need to click the Refresh Resource List button.

On page 23, the XML file is located on the server in /opt/Composite_Software/CIS_6.2.6/examples



22


Tips, cont’d

On page 26, theCreate Link and Targetis the button circled inred:



23


Email a zip file containing your screen shots to [email protected].

Due: Wednesday, Nov. 4 at 11:59 PM.

NOTE: We are also creating MySQL accounts for you on the same server. If we accomplish that in the next couple of days, you should upload your database dump file to the server via FTP, recreate your database on the server, and import it into CIS as another data source.

mailto:[email protected]



24

Extra Credit!

Convert Studio to run under Bash on Linux or the Mac. Studio is a Java application. Convert the Windows batch file studio.bat

(in the CIS bin directory) to a Bash script.

Zip all the required directories, jar files, xml files, property files, etc.

Unzip and run Studio on Linux or Mac OS X.



25

Extra Credit! cont’d

Create the same screen shots as for Assignment #8.

Individual work only!

Up to 25 points added to the midterm score of up to 10 students, chosen in the order of midterm scores (lowest scores first).

Due Friday, Nov. 13.

NOTE: No guarantees that this conversion is even possible!



26

Project Presentations

We lose two days: Nov. 11 Veterans’ Day Nov. 25 Day before Thanksgiving.

Originally scheduled for project presentations: Nov. 18 Dec. 2 (last day for this class)

We need a new plan for doing project presentations!

Two weeks apart!

CMPE 226 Database Systems October 28 Class Meeting Department of Computer Engineering San Jose State University Fall 2015 Instructor: Ron Mak mak.

Documents

source data

heterogeneous data

original data

data management

data replication

data structures cmpe

mak computer engineering

different data sources