The Data Warehouse: a Transitional Bridge between Legacy and PeopleSoft 11:20 – 12:10 May 16, 2005 CUMREC 2005 Conference Keystone, Colorado.

The Data Warehouse: a The Data Warehouse: a Transitional Bridge Transitional Bridge

between Legacy and between Legacy and PeopleSoftPeopleSoft

11:20 – 12:1011:20 – 12:10

May 16, 2005May 16, 2005

CUMREC 2005 ConferenceCUMREC 2005 Conference

Keystone, ColoradoKeystone, Colorado

22

Art Brooks, Art Brooks, Director of Director of Applications Applications University of University of Missouri – RollaMissouri – Rolla

• 12 years in Registrar’s Office12 years in Registrar’s Office

• 26 years in Information 26 years in Information TechnologyTechnology

• Involved in data warehousing Involved in data warehousing since 1986 since 1986

• Instructor of Data Warehouse Instructor of Data Warehouse coursecourse

Your PresenterYour Presenter

33

• Founded 1871 as the Missouri School of

Mines and Metallurgy

• Smallest of 4 campus system.

• Enrollment around 5500.

• Predominantly engineering and science.

University of Missouri - University of Missouri - RollaRolla

44

University of Missouri has University of Missouri has implemented Finance and HR implemented Finance and HR across the enterprise.across the enterprise.

Admit/Recruit is in production at Admit/Recruit is in production at UMKC, UMR and UMSL.UMKC, UMR and UMSL.

Student Records is in production Student Records is in production only at UMR (since January, 2004)only at UMR (since January, 2004)

NO campus has Student Financial NO campus has Student Financial Aid implemented nor scheduled at Aid implemented nor scheduled at this time.this time.

University of Missouri and University of Missouri and PeopleSoftPeopleSoft

55

The Evolving Data Warehouse• UMR Started data warehouse adventure in UMR Started data warehouse adventure in 1986.1986.

• Over 18 years the data warehouse and Over 18 years the data warehouse and reporting has transitioned:reporting has transitioned:– from local mainframe SQL/DS to local client/server from local mainframe SQL/DS to local client/server

with Informixwith Informix– from Informix to Oraclefrom Informix to Oracle– from native SQL for report creation to InfoMakerfrom native SQL for report creation to InfoMaker– through four strategic methodological changesthrough four strategic methodological changes– through three version upgrades in the Advance through three version upgrades in the Advance

systemsystem– through implementation of PeopleSoftthrough implementation of PeopleSoft– through one PeopleSoft version upgrade.through one PeopleSoft version upgrade.

66

• Basic reporting perspective

• UMR’s unique approach

• The Registrar’s conversion

Presentation Topics

77

Basic Reporting Basic Reporting PerspectivePerspective

KEEP IT SIMPLE

88

The Players and their rolesThe Players and their roles

• Technical staffTechnical staff – address the data – address the data infrastructure, provide a user friendly infrastructure, provide a user friendly reporting software and provide reporting software and provide training as well as consulting services.training as well as consulting services.

• Functional officeFunctional office – identify and define – identify and define reporting needs. Produce new reporting needs. Produce new reports, handle production scheduling reports, handle production scheduling and submit reports.and submit reports.

99

• Is user oriented.Is user oriented.

• Is an entity separate from the Is an entity separate from the transactional system.transactional system.

• Must be insulated from Must be insulated from transactional system changes.transactional system changes.

• Must include a transitional bridge.Must include a transitional bridge.

• Can begin BEFORE a system Can begin BEFORE a system conversion.conversion.

ReportingReporting::

1010

Data Warehouse dividendsData Warehouse dividends

• Is an evolutionary phenomenon.Is an evolutionary phenomenon.• Insulates business processes (reporting side) from Insulates business processes (reporting side) from

changes in hardware or software. (and in some changes in hardware or software. (and in some cases process changes)cases process changes)

• Provides extended flexibility.Provides extended flexibility.• Produces a point in time reference for reporting.Produces a point in time reference for reporting.• Produces a single point for change.Produces a single point for change.• Can provide consistency across reports and time.Can provide consistency across reports and time.• Can eliminate some logic errors.Can eliminate some logic errors.• Can simplify the data.Can simplify the data.

1111

• Data is refreshedData is refreshed– NightlyNightly– WeeklyWeekly– SeasonalSeasonal– On-demandOn-demand

UMR Download ProcessUMR Download Process

UM core systems

UMRDW

UMR written download scripts

Campussystems

Reports Applications

1212

Staff for reporting and data Staff for reporting and data warehousewarehouse

• 4 fulltime staff4 fulltime staff • Schedule downloads Schedule downloads from core systemsfrom core systems

• Maintain download Maintain download scriptsscripts

• Local DW table supportLocal DW table support

• Create new tablesCreate new tables

• Write reportsWrite reports

• Define data to usersDefine data to users

• Advise and assist usersAdvise and assist users

1313

UMR’s Unique UMR’s Unique ApproachApproach

FOCUSED ON THE USERS

1414

The UMR ApproachThe UMR Approach

• FFree standingree standing

• AAmbiguouslymbiguously

• RRelatedelated

• EEntitiesntities • In contrast to such In contrast to such conventional conventional approaches as star approaches as star schemas, fact tables schemas, fact tables and dimension tables.and dimension tables.

1515

The Reporting FoundationThe Reporting Foundation

• ‘‘Event oriented, functional tables’:Event oriented, functional tables’:relational tables designed for a specific relational tables designed for a specific reporting need that draws data from reporting need that draws data from multiple data warehouse tables, multiple data warehouse tables, focusing on the needs of the user and focusing on the needs of the user and not the technical staff.not the technical staff. GoalGoal – reduce – reduce technical requirements to formatting technical requirements to formatting the output page. the output page.

SIMPLICITY!!SIMPLICITY!!

1616

‘‘Functional’ Table Concept was Functional’ Table Concept was developed to:developed to:

– 1. ‘Empower the users’1. ‘Empower the users’– 2. Simplify the data structure2. Simplify the data structure– 3. Reduce report development time3. Reduce report development time– 4. Reduce processing time for the 4. Reduce processing time for the

server (quicker response)server (quicker response)– 5. Improve programmer efficiency5. Improve programmer efficiency– 6. Provide another tool for 6. Provide another tool for

reportingreportingYOU can do this!YOU can do this!

1717

ConceptConcept

– SimplicitySimplicity– Zero table joinsZero table joins– Zero ‘Where’ statementsZero ‘Where’ statements– Event orientedEvent oriented– Report design centricReport design centric– User takes less than five minutes to User takes less than five minutes to

develop querydevelop query– User concentrates on report appearanceUser concentrates on report appearance– Data source is the data warehouseData source is the data warehouse

1818

Advantages of the Functional Table Advantages of the Functional Table StrategyStrategy

– 1. Simplicity in data presentation and 1. Simplicity in data presentation and development.development.

– 2. Reports run significantly faster.2. Reports run significantly faster.– 3. Changes in data standards much more 3. Changes in data standards much more

adaptable (One place to make changes in data adaptable (One place to make changes in data interpretation and NOT in every report.)interpretation and NOT in every report.)

– 4. Provides a transitional bridge from legacy to 4. Provides a transitional bridge from legacy to new systemnew system

– 5. Provides a greater potential to ‘empower the 5. Provides a greater potential to ‘empower the users’.users’.

– 6. Reporting accuracy improved and many 6. Reporting accuracy improved and many potential errors removed.potential errors removed.

– 7. Reporting consistency is greatly enhanced7. Reporting consistency is greatly enhanced

1919

PlusPlus– 8. Shorter learning curve (no requirements on 8. Shorter learning curve (no requirements on

the part of the report writer to learn the core the part of the report writer to learn the core system data structure or methodology)system data structure or methodology)

– 9. Allows for continuation of longitudinal 9. Allows for continuation of longitudinal studiesstudies

– 10. Estimated eleven fold reduction in code to 10. Estimated eleven fold reduction in code to create a reportcreate a report

– 11. Allows for the creation of hybrid tables and 11. Allows for the creation of hybrid tables and thereby creating a TRANSITIONAL BRIDGE to thereby creating a TRANSITIONAL BRIDGE to SPAN the REPORTING ABYSS.SPAN the REPORTING ABYSS.

2020

Comparison of Relational Comparison of Relational ApproachesApproaches

Fact HR table

Appt data

Ed. data

Benefit data

Address data

Dept data

Bio data

Fact and dimension structure with Star Schema

Functional HR table with same data

UMR Approach

2121

Report creation processReport creation process

User identifies report

requirements and format

Technical staff creates

appropriate functional

tables

Report created by user or

technical staff

Finished report

2222

Proven ApproachProven Approach• Concept developed in 1998Concept developed in 1998

• First put into use in 1999 with conversion First put into use in 1999 with conversion of University Advancement system from of University Advancement system from mainframe to client/servermainframe to client/server

• Over 500 new reports created for that Over 500 new reports created for that department using this techniquedepartment using this technique

• Utilized in 2000 to retain orientation Utilized in 2000 to retain orientation system when Admissions implemented system when Admissions implemented PeopleSoftPeopleSoft

• An integral part of reporting solution at An integral part of reporting solution at UMR prior to 2004 Registrar’s PeopleSoft UMR prior to 2004 Registrar’s PeopleSoft implementationimplementation

2323

Example report:Example report:User requests a count of all new User requests a count of all new freshmenfreshmen

University of Missouri – RollaUniversity of Missouri – Rolla

New Freshman Applicant CountNew Freshman Applicant Count

FS2005FS2005

MajorMajor CountCount

BiologyBiology 21 21

C ScC Sc 53 53

ISTIST 78 78

MISMIS 39 39

2424

Report Creation ComparisonsReport Creation Comparisons

select a.admajor, count(*)from admfresh a where a.adterm=’FS2001’and a.fraction=’A’and a.fractiondate<=sysdategroup by a.admajor”;

select f.acad_plan, count(*)from ps_pers_data_effdt a, ps_pers_dtef_sa_vw a1, ps_stdnt_career b,ps_adm_appl_data c, ps_adm_app_car_seq d,ps_adm_appl_prog e, ps_adm_appl_plan f,where a.emplid=a1.emplid andfrom ps_pers_data_effdt a_eda.effdt=a1.effdt and(a.effdt=(select max(a_ed.effdt)where a.emplid=a_ed.emplid anda_ed.effdt<=sysdate) anda.emplid=b.emplid and b.acad_career=c.acad_career and c.emplid=d.emplid and c.acad_career=d.acad_career and c.stdnt_car_nbr=d.stdnt_car_nbr and c.adm_appl_nbr=d.adm_appl_nbr and d.emplid=e.emplid and d.acad_career = e.acad_career and d.stdnt_car_nbr = e.stdnt_car_nbr and d.adm_appl_nbr = e.adm_appl_nbr and d.appl_prog_nbr = e.appl_prog_nbr and e.effdt=(select max(e_ed.effdt)from ps_adm_appl_prog e_edwhere e.emplid=e_ed.emplid ande.acad_career=e_ed.acad_career ande.stdnt_car_nbr=e_ed.stdnt_car_nbr ande.adm_appl_nbr=e_ed.adm_appl_nbr ande.appl_prog_nbr = e_ed.appl_prog_nbr and e_ed.effdt <=sysdate) andfrom ps_adm_appl_prog e_eswhere e.emplid=e_es.emplid and e.acad_career=e_es.acad_career ande.stdnt_car_nbr=e_es.stdnt_car_nbr and e.adm_appl_nbr=e_es.adm_appl_nbr ande.appl_prog_nbr=e_es.appl_prog_nbr and e.effdt=e_es.effdt ande.prog_status=e_es.prog_status) and e.admit_term=’FS2001’ ande.prog_status in (‘AC’, ‘AD’) and e.action_dt <= sysdate andc.admit_type=’FTC’ and d.acad_career=’UGRD’ ande.emplid = f.emplid ande.acad_career=f.acad_career and e.stdnt_car_nbr=f.stdnt_car_nbr and e.adm_appl_nbr=f.adm_appl_nbr and e.appl_prog_nbr = f.appl_prog_nbr and f.effdt=(select max(f_ed.effdt)from ps_adm_appl_plan f_ed where f.emplid=f_ed.emplid and f.acad_career=f_ed.acad_career andf.stdnt_car_nbr=f_ed.stdnt_car_nbr and f.adm_appl_nbr=f_ed.adm_appl_nbr andf.appl_prog_nbr=f_ed.appl_prog_nbr andf_ed.effdt<=e.effdt) andf.effseq=(select max(f_es.effseq)from ps_adm_appl_plan f_es where f.emplid=f_es.emplid andf.acad_career=f_es.acad_career and f.stdnt_car_nbr=f_es.stdnt_car_nbr and f.adm_appl_nbr=f_es.adm_appl_nbr and f.appl_prog_nbr=f_es.appl_prog_nbr and f.effdt=f_es.effdt))group by f.acad_plan;

1 query1 table

0 joins3 'where' statements

6 lines

1 query

7 distinct tables

12 process tables

6 joins

5 sub queries

50 'where' statements

70 lines

UMR Functional Table Query Direct PeopleSoft Query

Each of these queries Each of these queries countscounts

the number of the number of freshmenfreshmen

admitted for a admitted for a specifiedspecified

term. The results are term. The results are the same.the same.

2525

Report Prepared direct from PeopleSoftReport Prepared direct from PeopleSoft

PS table1

PS table2

PS table7

PS table3

PS table4

PS table5

PS table6

12 Processtables

Query

Report:Count number of

Freshmen forSpecified term

Sub Query1

Sub Query2

Sub Query3

Sub Query4

Sub Query5

12 Processtables

12 Processtables

12 Processtables

12 Processtables

12 Processtables

2626

Report prepared from UMR functional Report prepared from UMR functional tabletable

Report:Count number of

Freshmen forSpecified term

UMR functiona

l table

Query

2727

Hybrid tablesHybrid tables– After further experience and discussion it was After further experience and discussion it was

realized the functional tables could be realized the functional tables could be hybridized to satisfy specific reporting needs hybridized to satisfy specific reporting needs and to provide a transitional bridge to the and to provide a transitional bridge to the future.future.

– Definition – a hybrid functional table is one that Definition – a hybrid functional table is one that has data derived from disparate systems. has data derived from disparate systems. (normally legacy and PeopleSoft)(normally legacy and PeopleSoft)

– Hybrid tables can become transitional tables.Hybrid tables can become transitional tables.– With time, hybrid tables can become normal With time, hybrid tables can become normal

functional tables. (When the legacy data is no functional tables. (When the legacy data is no longer required, the columns cease to be filled longer required, the columns cease to be filled or are removed.)or are removed.)

2828

Hybrid table translationHybrid table translation

Stdnt No. Addr_type PS_Addr_type

12345678 L LOC

23456789 M Mail 203 State

34567890 B BUS 4578 Hampton Circle

Addr_line1

123 Bishop Ave

Stdnt No. Addr_type

12345678 L

23456789 M 203 State

34567890 B 4578 Hampton Circle

Addr_line1

123 Bishop Ave

Legacy

Converted PeopleSoft

Table name (address) is unchanged

2929

Legacy reports

New reports

OutputHybrid Tables

Hybrid table flexibilityHybrid table flexibility

Code changes kept to a minimum

3030

Evolution of a Hybrid TableEvolution of a Hybrid Table

StunoStuno GenderGender Vet_codeVet_codeCitizenshiCitizenshi

ppFin_Aid_InFin_Aid_In

tt Mar_StatMar_Stat Eth_OrgEth_Org DOBDOB

Studnt #Studnt # M/FM/F 11 JpnJpn Y/NY/N SS 66 MM/DD/YYYYMM/DD/YYYY

BIO TableBIO Table

-- PeopleSoft value-- PeopleSoft value

Original -- UMR Data Warehouse Table Name/Structure and Column Original -- UMR Data Warehouse Table Name/Structure and Column NamesNames

StunoStunoOld Old StunoStuno GenderGender

Vet_codVet_codee

PS_Mil_PS_Mil_StSt

CitizenshCitizenshipip

Fin_AidFin_Aid_Int_Int Mar_StatMar_Stat

PS_MarPS_Mar_St_St Eth_OrgEth_Org DOBDOB

Sp Sp InterestsInterests

EmpliEmplidd

Studnt Studnt ## MM 11 VetVet JpnJpn    SS SingleSingle 66    BandBand

     -- translated values PS to UMR DW-- translated values PS to UMR DW    -- new PeopleSoft data -- new PeopleSoft data itemitem

First Degree of First Degree of HybridizationHybridization

Second Degree of HybridizationSecond Degree of HybridizationBIO TableBIO Table

BIO TableBIO Table

StuStunono

Old Old StunoStuno

GenGenderder

PS_GPS_Genderender

Vet_cVet_codeode

PS_MilPS_Mil_St_St

CitizensCitizenshiphip

PS_CitzPS_Citznn

Mar_SMar_Stattat

PS_MaPS_Mar_Str_St

Eth_Eth_OrgOrg

PS_EtPS_Eth_Orgh_Org DOBDOB

Sp Sp InterestInterestss

EmplEmplidid

Studnt Studnt ## MM MaleMale 11 VetVet JpnJpn JapanJapan SS SingleSingle 66

CaucasCaucasianian    BandBand

Not needed or found at this Not needed or found at this levellevel

Third Degree of Third Degree of HybridizationHybridization

BIO TableBIO Table

StunoStuno GenderGenderPS_Mil_SPS_Mil_Stt PS_CitznPS_Citzn

PS_Mar_SPS_Mar_Stt PS_Eth_OrgPS_Eth_Org DOBDOB Sp InterestsSp Interests

AmbassadoAmbassadorr

Collection_ICollection_IDD

EmplidEmplid MaleMale VetVet JapanJapan SingleSingle CaucasianCaucasian YYYY/MM/DDYYYY/MM/DD BandBand YESYES 1234512345

* the BIO table draws its data from 11 PeopleSoft * the BIO table draws its data from 11 PeopleSoft tablestables

3131

Bridge tablesBridge tables

Code Abbrev Description

Legacy system tables

PeopleSoft bridge tables

001 Aerospace EngineeringAE

Code Abbrev Description

001 AE Aerospace Engineering

PS_abbr

Aero E

3232

Multi-year reportsMulti-year reports

ExistingReportsLegacy Tables

Bridge tables

Hybrid tables

NewReports Longitudina

l studies/ trend

analysis requests

PeopleSoft tables

UMR DW

3333

The Registrar’s The Registrar’s ConversionConversion

KEEP THE BUSINESS

PROCESSES FUNCTIONING

3434

Road Closed -- ConstructionRoad Closed -- Construction

• When transitioning from a legacy When transitioning from a legacy system to a new system, you system to a new system, you CANNOT shut reporting down.CANNOT shut reporting down.

• Tantamount to shutting down an Tantamount to shutting down an Interstate for road construction.Interstate for road construction.

• MUST find an alternate solution.MUST find an alternate solution.

3535

Registrar’s EnvironmentRegistrar’s Environment

• Legacy system in production since Legacy system in production since 19751975

• Mainframe basedMainframe based

• Four campus systemFour campus system

• Nearly all UMR campus reports Nearly all UMR campus reports utilizing local data warehouse since utilizing local data warehouse since late 1980s.late 1980s.

3636

Strategic StatementStrategic Statementfor Registrar’s Conversionfor Registrar’s Conversion

• ““If data can be If data can be convertedconverted from legacy to from legacy to PeopleSoft, then data can be PeopleSoft, then data can be translatedtranslated from PeopleSoft to legacy.” from PeopleSoft to legacy.” (We NEVER (We NEVER stated ALL of the data could be translated.)stated ALL of the data could be translated.)

• Terms:Terms:– Converted Converted – data in a legacy format, – data in a legacy format,

modified and loaded in PeopleSoft tablesmodified and loaded in PeopleSoft tables– TranslatedTranslated – data in a PeopleSoft format, – data in a PeopleSoft format,

modified and loaded in legacy defined tables.modified and loaded in legacy defined tables.

3737

The ChallengeThe Challenge• With modules being implemented over a six year With modules being implemented over a six year

period of time, applications and reports had to period of time, applications and reports had to continue to function when some data was in the continue to function when some data was in the mainframe in a legacy format and other data was mainframe in a legacy format and other data was in PeopleSoft in a client server/relational in PeopleSoft in a client server/relational environment. environment.

• This required applications and reports to draw This required applications and reports to draw data from tables that had diverse sources.data from tables that had diverse sources.

• Inventory indicated over 3,000 reports and 25 Inventory indicated over 3,000 reports and 25 applications built over a 10+ year period of time applications built over a 10+ year period of time would cease to function.would cease to function.

• No defined subset of reports required or in use.No defined subset of reports required or in use.

3838

Implementation tacticsImplementation tactics• Wanted to ‘BUY TIME’ to keep reports running until Wanted to ‘BUY TIME’ to keep reports running until

replacements could be written.replacements could be written.

• Focused on identifying PeopleSoft data to re-create Focused on identifying PeopleSoft data to re-create legacy tables for reports needed within first 2 weeks legacy tables for reports needed within first 2 weeks of the semester.of the semester.

• All staff dedicated to effort:All staff dedicated to effort:– Programming staff worked on bringing tables up, Programming staff worked on bringing tables up,

AB tested and modified needed existing reports. AB tested and modified needed existing reports.

• After 2 weeks focused on next set of most critical After 2 weeks focused on next set of most critical tables.tables.

• Found some data just could not be re-created. Found some data just could not be re-created. Developed a work around.Developed a work around.

3939Users did not see the difference.Users did not see the difference.

UMR DWUMR DW

ExistingReports &

Applications

Legacy Tables

Bridge tables

Hybrid tables

PeopleSoft System

NewReports

UMR Startup TacticUMR Startup Tactic

4040

ResultsResults• Translated PeopleSoft admissions data for 3 years Translated PeopleSoft admissions data for 3 years

without interrupting production.without interrupting production.

• Integrated PeopleSoft HR with student legacy for 2 Integrated PeopleSoft HR with student legacy for 2 years without interrupting production.years without interrupting production.

• Integrated legacy and PeopleSoft grant data for a fiscal Integrated legacy and PeopleSoft grant data for a fiscal year report.year report.

• Amended HR data without affecting applications.Amended HR data without affecting applications.

• Had 400 reports and all applications needing student Had 400 reports and all applications needing student data in production by the end of the 4data in production by the end of the 4thth week of classes. week of classes.

• Integrated admissions PeopleSoft data with legacy Integrated admissions PeopleSoft data with legacy student data for 18 months until student system went student data for 18 months until student system went live with PeopleSoftlive with PeopleSoft

4141

Questions?Questions?

4242

Art BrooksArt BrooksDirector of ApplicationsDirector of ApplicationsInformation TechnologyInformation TechnologyUniversity of Missouri – RollaUniversity of Missouri – RollaE-mail: E-mail: [email protected]@umr.edu

ContactContact