PROJECT OMNIGLEAN Team Members: Kenny Trytek Derek Woods Abby Birkett Joe Briggie Advisor: Simanta Mitra Client: Kingland Systems.

PROJECTOMNIGLEANTeam Members:Kenny TrytekDerek WoodsAbby BirkettJoe Briggie

Advisor:Simanta Mitra

Client:Kingland Systems

PROBLEM STATEMENT

• Large companies have many layers of corporate hierarchy.

• Financial and data records sometimes conflict between

various layers/entities.

• Accurate and comprehensive company records are needed.

• There is a need for “Data Mastering”, to take multiple

conflicting sources of data and

• determine the reality of the matter in conflict.

CONCEPT SKETCH

Client User

Analyst User

OmnigleanInternet

Harvest & store

Figure 2.1.1: Concept Sketch

FUNCTIONAL REQUIREMENTS

• System shall autonomously traverse publicly available websites

• System shall parse information from downloaded file in

portable document format (pdf)

• System shall store parsed information in a flat file

• System shall allow user the ability to create, modify, and delete

records

• System shall maintain a normalized database

• System shall expose functionality through web services

NON-FUNCTIONAL REQUIREMENTS

• System shall support up to 250 concurrent users

• A single run of system shall complete execution in less

than six hours

• System shall be easily extensible to include more

websites than originally specified

• System shall be completed by May of 2011

CONSTRAINTS AND TECHNICAL CONSIDERATIONS

• System shall interact with a third party library to facilitate

database interaction

• The database may not be available at all times

• Using SVN to manage code

• Using the spiral design process

MARKET SURVEY

• Omniglean provides a unique combination of access to freely available FDIC and FFIEC data through a data mastering suite.

• Omniglean provides access to the mastering capability through web services, to enable a rapid delivery of functionality to customers of Kingland Systems, as well as analysts located in different geographical areas.

POTENTIAL RISKS AND MITIGATION

• External format change

• External availability

• New technologies

• Web services

• Not enough time for testing and debugging

• Team members

COST AND RESOURCE ESTIMATE

Item Cost

Reporting

Poster Materials $50.00

Report Materials $50.00

Labor @ $20.00/hr

Kenny Trytek $3,560

Abby Birkett $3,220

Joe Briggie $3,160

Derek Woods $3,360

Total $13,100

PROJECT MILESTONES

• Complete modules related to harvesting and

transforming the data.

• Complete web services and user interface modules

• Integrate all modules successfully

FUNCTIONAL DECOMPOSITION

• Harvester – Gathers data

• ETL – Transforms data

• DAL – Database access layer

• Web services – Exposes data to external users

• User interfaces

SYSTEM DIAGRAM

Flat FileDatabase

ETL Tool

Normalized

Kingland Data Analyst UI

DAL

No Conflicts?

External Client UI

Web Svcs.

WWW Data Scraper Tool

HTML ParserPDF Parser

CreateRead

UpdateDelete

HARVESTERScraper

Flat File

(XML)

World Wide Web

Parser

PDF Parser

HTML ParserstartGatheringData() - Returns the XML

document populated with data from the site.

getLogFile() - Returns the log file that is either being written to or has been written to this session.

stopGatheringData() - Stops all current harvester operations and writes an error to the log file.

ETL TOOL ETL Tool DALFlat File(XML)

• loadFFIEC() - This method will load the data from the XML file into the FFIEC table.

• loadFDIC() - This method will load the data from the XML file into the FFDIC table.

• createORGANIZATION() - This method will take the information from both the FFIEC table and the FDIC table and put it into the ORGANIZATION table.

DATA ACCESS LAYER

　　　 DAL

Database

User InterfaceETLTool

Add()Find()Update()Delete()

Organization- A class that creates and maintains a connection to the ORGANIZATION table

GetConnection()CloseConnection()

OrganizationService- This class will allow CRUD functionality with an Organization object.

Find(String organizationId)Add(...)Delete(String organizationId)Update(String organizationId, ...)

WEBSERVICES

Unauthenticated

Authenticated

Read()

LogOut()

Write()

Update()

Delete()

Read()

LogIn()

• Create()• Read() • Update()• Delete()• login()• logout()

Allows remote users to access the database throughthe Internet

TECHNOLOGY PLATFORM

• SQL Server 2008

• Visual Studio 2010 development environment

• Microsoft Windows operating system

• WSDL and SOAP for web services

TEST PLAN

The team will be testing the system in three phases.

The first phase is testing the individual modules, the

second phase will be testing the integration of the

modules, and the final phase will be testing the

system as a whole.

PROTOTYPINGWe have begun prototyping. The harvester is able to

traverse the necessary websites easily. The ETL can read in

XML files. The user interface has been mocked up.

CURRENT PROJECT STATUS

Activity Start Date

End Date

Sept 13-19

Sept 20-26

Sept 27-Oct 3

Oct 4-10

Oct 11-17

Oct 18-24

Oct 25-31

Nov 1-7

Nov 8-14

Nov 15-21

Nov 22-28

Nov 29-Dec 5

Dec 6-12

Dec 13-19

Dec 20-26

Dec 27-Jan 2

Jan 3-9

Jan 10-16

Jan 17-23

Jan 24-30

Jan 31- Feb 6

Feb 7-13

Feb 14-20

Feb 21 -27

Feb 28 – Mar 6

Mar 7-13

Mar 14-20

Mar 21-27

Mar 28 – Apr 3

Apr 4-10

Apr 11-17

Apr 18-24

Apr 25 – May 1

Project Plan Presentaion

09/21/10 09/27/10

Project Plan Rough Draft

09/21/10 10/05/10

Project Plan Final Draft

10/05/10 10/12/10

Prototyping 10/05/10 12/10/10

Design Document Rough Draft

10/05/10 11/15/10

Design Document Final Draft

11/15/10 12/03/10

Testing Phase 01/31/11 04/04/11

System Completion

12/10/10 04/04/11

End Product Documentation

03/21/11 04/04/11

Project Poster 03/14/11 03/28/11

Project Presentation

04/25/11 04/25/11

Buffer 04/11/11 4/24/11

TASK RESPONSIBILITY

Kenny Trytek – Team Leader, responsible for the

harvester

Derek Woods – Developer, responsible for the ETL and

UI

Abby Birkett – Developer, responsible for database

and DAL

Joe Briggie – Developer, responsible for web services

PLAN FOR NEXT SEMESTER

• Continue prototyping

• Continue to meet with client to be sure we are

meeting expectations

• Develop a more thorough test plan

QUESTIONS?