Applying Concepts of Software Reuse to the Implementation of Data Warehouse ETL Systems Jiayang Zhou November 2001.

Applying Concepts of Software Reuse to the Implementation of Data Warehouse ETL Systems

Jiayang ZhouNovember 2001

Data Warehouse Loader sd&m

Applying concepts of software reuse to Data Warehouse Loader

Fundamental of software reuse

Description of Data Warehouse Loader

Implementation of Data Warehouse Loader

Reuse Analysis of Data warehouse Loader

Summary

Data Warehouse Loader sd&m

Applying concepts of software reuse to Data Warehouse Loader

Fundamental of software reuse

Description of Data Warehouse Loader

Implementation of Data Warehouse Loader

Reuse Analysis of Data warehouse Loader

Summary

Data Warehouse Loader sd&m

Definition of Software reuse

Software reuse is a process of implementing or updating

software systems using existing software assets.

software components objects software requirements analysis

design model domain architecture database schema

codedocumentation manual standardstest scenario

plan

Data Warehouse Loader sd&m

Importance of Software reuse

Increases productivity

Enhances quality

Saves cost

Reduces software development schedules

Reduces maintenance

Enhances standardization

Increases portability

Data Warehouse Loader sd&m

Fundamental of software reuse

Description of Data Warehouse Loader

Implementation of Data Warehouse Loader

Reuse Analysis of Data warehouse Loader

Summary

Applying concepts of software reuse to Data Warehouse Loader

Data Warehouse Loader sd&m

Architecture of data warehouse application

Data Warehouse Loader sd&m

Features of data warehouse

A data warehouse is a

Subject-oriented

Integrated

Time-variant

Non-volatile

collection of data for management’s decision making process.

The role of Data Warehouse Loader

Extraction: extract raw data from different source system

Transformation: transform data in order to be fit into the architecture of data warehouse

Loading: import data into data warehouse

Data Warehouse Loader sd&m

The requirements of Data Warehouse Loader

Adaptability to any data source systems

Adaptability to any target data warehouse

High operation speed

Ability to deal with large amount of data

Flexibility for data transformation

Portability

Data Warehouse Loader sd&m

The reuse consideration of Data Warehouse Loader

Data Warehouse Loader sd&m

Fundamental of software reuse

Description of Data Warehouse Loader

Implementation of Data Warehouse Loader

Reuse Analysis of Data warehouse Loader

Summary

Applying concepts of software reuse to Data Warehouse Loader

Data Warehouse Loader sd&m

Architecture of Data Warehouse Loader

Data Warehouse Loader sd&m

Loader-engine

Data Warehouse Loader sd&m

Loader-interface

• Extraction-interface: access to data source

• Transformation-interface: different transformation scheme

• Database-interface: access to data warehouse for updating and retrieving

• Record-interface: construct an object structure for each database record

Data Warehouse Loader sd&m

Data Warehouse Loader sd&m

Format of intermediate file

INFO header

FORMAT header

DATA body

Data Warehouse Loader sd&m

Fundamental of software reuse

Description of Data Warehouse Loader

Implementation of Data Warehouse Loader

Reuse Analysis of Data warehouse Loader

Summary

Applying concepts of software reuse to Data Warehouse Loader

Data Warehouse Loader sd&m

Reuse development of Data Warehouse Loader

Thorough examination of application requirements

Good documentation

Choosing of Java as programming language

Concrete separation of specification from a particular application

Data Warehouse Loader sd&m

Advantages of workflow architecture

Easy maintenance

Module reuse

Temporally decoupling

Manageable and controllable programming

Re-composition of operation sequence

Data Warehouse Loader sd&m

Re-composition of operation sequence

Data Warehouse Loader sd&m

Appling concepts of software reuse

Code reuse

Adaptability

Modularity

Interface

Data Warehouse Loader sd&m

Reuse architecture of Data Warehouse Loader

Data Warehouse Loader sd&m

Package overview of Data Warehouse Loader

Data Warehouse Loader sd&m

Fundamental of software reuse

Description of Data Warehouse Loader

Implementation of Data Warehouse Loader

Reuse Analysis of Data warehouse Loader

Summary

Applying concepts of software reuse to Data Warehouse Loader

Data Warehouse Loader sd&m

START-MDB project

Data Warehouse Loader sd&m

Java and C

C

• fast

• migration between different systems

• difficulty for flexibility

Java

• slow and large RAM consumption

• portability and platform-independence

• easy for flexibility

• object-oriented feature

Data Warehouse Loader sd&m

Thank you very much for your attention!

[email protected]