© 2012 Boise State University1 Data Warehouse User Engagement Readout Daniel Gold, Project Manager.

© 2012 Boise State University 1

Data Warehouse User Engagement Readout

Daniel Gold, Project Manager


• Executive Summary• Study Methodology and Participants• Tool Needs and How They Align to Pyramid• Common Reporting Tasks• Process Opportunities• What is Needed for Success• Summary and Next Steps

Agenda:


Executive SummaryGoal of User Engagement• Understand the key features and functions that users need from our ProClarity

replacement• Understand barriers that exist today that create complexity when obtaining or using

data• Understand what training and support will be needed as we roll out our ProClarity

replacement

Outcome• Based on features requested by users, Pyramid is a suitable replacement for ProClarity• Key reporting areas were identified that if improved would return immediate value to

end users • Hands-on training based on knowledge level (i.e. Novice, Power User, Expert), robust

self-help resources, BIRS point of contact and peer groups will be needed to support new tool adoption

• Process changes are needed to streamline report intake and prioritization• Process changes are needed to build trust in the data warehouse and improve usability


Methodology• Two user engagement sessions were

held– 12 participants on 7/15– 18 participants on 7/18

• Participants were broken into small groups and led through a series of activities including:

1. Brainstorm and categorization of data needs

2. Completing a “reporting story” form3. Extracting tool features and process

enhancements from their stories4. Prioritization of features and process

changes

30• Participants

50• Reporting stories

collected

43• Unique tool feature

needs identified

12• Unique process

changes requested


Participant PopulationAreas Represented:

– College of Arts and Sciences– Institutional Research– College of Engineering– eCampus– Extended Studies– College of Health– Registrar– College of Health Sciences– Provost– College of Education– College of Social Sciences and Public Affairs– Admissions– Honors College– University Financial Services– BSU Foundation– STEM– OIT

Roles Represented:– 5 directors– 3 associate directors– 3 business managers– 2 administrative assistants– 2 data analysts– 1 adjunct faculty coordinator– 1 advisor– 1 associate registrar– 1 business operations manager– 1 chair– 1 coordinator– 1 financial technician– 1 management assistant– 1 management systems coordinator– 1 program information coordinator– 1 project coordinator– 1 project manager– 1 senior business manager– 1 vice provostHR and Finance under-represented! Further

investigation to needs of these teams required.


Agenda: Executive Summary Study Methodology and Participants• Tool Needs and How They Align to Pyramid• Common Reporting Tasks• Process Opportunities• What is Needed for Success• Summary and Next Steps


Requested Tool FeaturesTop 10 Prioritized Features:1. Drill to Detail (that I need)2. Snapshots (compare points in time)3. Dashboards4. Build in Calculations 5. Easy export to Excel and PDF6. Trend over time 7. Ability to combine multiple reports 8. Sort/filter data in web tool9. Data “Table of Contents” and improved logical structure 10. Ability to add charts/graphs in web tool11. Support for custom reports (MyViews)12. Distribute reports to email audience from tool


Pyramid Can Address Our Tool NeedsRequested Feature Available in Pyramid Suite Demo links

Drill to detail (that I need) Basic Analytics ; Cascading Slicers ; Custom Parameters

Support for Snapshots BIRS Process Controlled

Dashboards Building Dashboards Trending over time Basic Analytics

Combine multiple reports Data Mash up with Power Pivot Built in calculations Built in and Custom Calculations Data “Table of Contents” BIRS Process Controlled

Sort/Filter data Basic AnalyticsAbility to add charts/graphs Multi/Combo Charts

Distribute reports via emailEasy export to excel/PDFSupport for custom views

http://www.pyramidanalytics.com/demo/movies/classic.aspx

http://www.pyramidanalytics.com/demo/movies/slicers.aspx

http://www.pyramidanalytics.com/demo/movies/params.aspx

http://www.pyramidanalytics.com/demo/movies/design.aspx


http://www.pyramidanalytics.com/demo/movies/powerpivot.aspx

http://www.pyramidanalytics.com/demo/movies/calcs.aspx


http://www.pyramidanalytics.com/demo/movies/charts.aspx


Agenda: Executive Summary Study Methodology and Participants Tool Needs and How They Align to Pyramid• Common Reporting Tasks• Process Opportunities• What is Needed for Success• Summary and Next Steps


Report Frequent Flyers• 4 report categories made up

nearly 2/3rds of all reporting stories written by participants.

• Short Term: Understanding the usage of reports in these categories and improving them could have a significant impact to user satisfaction and eliminate needless effort

• Long Term: Consider prioritizing the creation of dynamic dashboards in Pyramid for these common reporting tasks The “Core Four”


Recommendations:• Recognize the “core four” reporting needs and prioritize

enhancements for these areas• Form tiger teams for each area and investigate reporting

enhancements• Identify standard reporting tasks for the “core four” that will

be good candidates for dashboards when Pyramid is released

What’s a Tiger Team?“ A team of undomesticated and uninhibited technical specialists, selected for their experience, energy,

and imagination, and assigned to track down relentlessly every possible source of failure” - Program Management in Design and Development


Common Data Manipulation TasksPain Points:• Lack of complete data (HR,

Finance, shadow systems) in data warehouse leads to users having to combine DW reports with queries from other sources

• Comparing data for particular points in time (aka snapshots) requires running multiple reports

• Drill to detail often provides significantly more data than is needed and excess rows/columns must be deleted

• Lack of built in calculations and sort/filter options in web tool makes exporting data to excel mandatory

• Lack of native graph building capabilities makes exporting data to excel mandatory

User comment:All this would be better if… “we were able to gather point in time comparison and make the comparisons in a more efficient manner. Right now this takes 2 people approximate 6-8 hours every week to complete.”


Recommendations:• Utilize tiger teams to understand why reports frequently

need to be combined.• Make snapshot data available to users where it exists• Review report frequent flyers to determine how simple

changes in formatting can reduce manual manipulation


Use of Production Queries• 52% of participants rely

on queries against the production PeopleSoft system– Expected for data that is

not in the warehouse (HR, Finance, and Financial Aid)

– Expected for data that needs to be real-time

• 35% of participants rely on queries for student data that is in the warehouse– Unexpected since this

data is available

Potential Reasons for production queries:1. Historical point in time comparisons (snapshots)2. Validate data obtained through MyInsights or

ProClarity3. Familiarity with querying production over using DW

tools


Recommendations:• Utilize tiger teams to understand why production queries are

needed to support the “core four”– Is data missing in the warehouse?– What kind of snapshots are needed for comparison?– Are we missing key data elements like type/rank of faculty?

• Ensure that Pyramid is marketed well and encourage users to begin using the warehouse to meet reporting needs


Agenda: Executive Summary Study Methodology and Participants Tool Needs and How They Align to Pyramid Common Reporting Tasks• Process Opportunities• What is Needed for Success• Summary and Next Steps


Data IntegrityDuring group discussions two types of data integrity / validation issues emerged:1. Trust in the accuracy of the data

in the warehouse (checking for warehouse errors)

2. Trust that manual manipulation performed on data had not corrupted the report (checking for user errors)

User Comments:

“I think the most complex part of the reporting is validating the accuracy of the data. So much of what we do is this it takes the most time and is the most frustrating.”

“[M]anual manipulation in excel is time consuming and there is risk of error. Data validation.”

All of this would be better if… “[It] didn't take extensive manipulation each time. Confidence that the data was correct.”

“[N]umerous proclarity reports are referenced in order to create the summer report. Difficult to identify the criteria used to create the ProClarity report and difficult to validate the data. Time consuming to create report.”

1 in 4 participants cited data validation as a time consuming

and required step.


Recommendations• Look for ways to limit the need to combine multiple reports

in excel through enhancements to current reports• Build trust in data warehouse reports by consistently

validating data with data stewards and stating explicitly on reports when validation took place in each report


Data Definitions• Users reported difficulty

interpreting the data due to irregular naming conventions and inconsistent, unavailable, or out of date definitions.

• Knowing which fields should be

used in a report was often difficult when data from multiple sources had to be combined.

User Comment:All this would be better if… “You could get a clear definition for what "active" means (e.g. not discontinued on PS, or what?!)”

User Comment:“have you seen the terminology? there are five categories for each line. Things like STEM secondary non-STEM major”


Recommendations• Determine what information needs to be stored in each data

definition to aid users in making decisions when building reports

• Ensure that updating data definitions is built into the development teams “definition of done” any time a report is altered or created

• Improve data definition discoverability by creating links or embedding definitions in reports


Data Access and IntegrationAccess:• Access to data was not a wide spread issue, and most users who spoke about

access were concerned with limiting it (security).• A small number of users stated that they needed access to HR data to perform

their job functions

Integration:• Integrating HR, Finance and Student data into the data warehouse was a frequent

request• Many non-BIRS supported reporting systems are used on campus. Frequently

these are used for storing snapshot data for comparisons.


Recommendations• Leverage the Identity and Access Management Project to help

define role based data access authorization• Continue roadmap work to bring in HR and Finance data into

the warehouse• Investigate commonly used reporting systems and determine

if it would be reasonable to bring the data into the warehouse


Process EnhancementsTop 10 Prioritized Process Enhancements1. Data validation2. Data integration 3. Data definitions4. Data access5. Clear report structure6. Standardize reports for wider usage7. Make the data current (where is a student now)8. Migrate custom views to new tool9. Improve report request workflow10. Map tools to user needs


Recommendations• Continue with roadmap work to bring HR and Financial data into the

warehouse• Establish a holistic “definition of done” for reports that includes data

validation and data definitions for every report• Utilize Identity and Access Management project to help define roles for data

access• Leverage tiger teams to standardize reports for wider use• Develop a clear and logical hierarchy for reports• Revise and evangelize report request intake process with a simple workflow

that that takes into consideration requirements gathering and prioritization• Begin working to establish a “data doctor” program that includes support

from subject matter experts as well as BIRS• Begin developing a rollout strategy for Pyramid that will help map training

and tool sets to groups of users based on their reporting needs


Agenda: Executive Summary Study Methodology and Participants Tool Needs and How They Align to Pyramid Common Reporting Tasks Process Opportunities• What is Needed for Success• Summary and Next Steps


Success Requires…• Hands on training for users

– Levels of training (novice, power user, expert)

• Robust documentation and tutorials• An accessible reporting expert

– Data Dr. from BIRS– Peer groups

• Migrate existing ProClarity reports into new tool• Clearly define what tool should be used for what task• Early adopter program for new tool• More report building resources on the BIRS team


Agenda: Executive Summary Study Methodology and Participants Tool Needs and How They Align to Pyramid Common Reporting Tasks Process Opportunities What is Needed for Success• Summary and Next Steps


Tool Implementation

•Early adopter program for IR and key users

•BIRS developer training

Create Experts

•FAQ and how-to documents

•Tutorials•Tool documentation

Develop Self Help Tools •Classroom based

training•Training based on data tasks and complexity

Train Users

•Data Dr.•Peer groups•Ongoing training opportunities

Provide Ongoing Support

Install Pyramid

Migrate ProClarity Reports

Develop Report Hierarchy

Develop Core Reports and Dashboards

Early adopter rollout

End User Rollout

User Rollout Track

Development Track


Recommended Next Steps• Stay the Course for Roadmap Activities

– Continue to move forward with Pyramid purchase and implementation– Continue to integrate HR and Finance data into the warehouse

• Pause Report Development and Modernize Process– Define and evangelize project intake process– Refine the team “definition of done”– Respond to user engagement requests by generating specific

enhancement projects for “core four” report issues identified in sessions• Identify Goals and Participants for Tiger Teams

– Develop specific and measurable goals for tiger teams– Identify subject matter experts, power users, and developers to participate– Establish a timeline for tiger teams to provide actionable

recommendations (likely post-roadmap)


THANK YOU

Daniel Gold, Project ManagerEmail [email protected]

mailto:[email protected]


BACKUP


Breakdown of the Core FourEnrollment

• Out of State Students –Currently Enrolled

• Class Enrollment Report• Fall Enrollment by Plan• Enrollment Activity Report• Student Enrollment• Enrollment Profile• Credit and Enrollment for

Extended Studies• Summer Enrollment and

Credit Data

Student Success

• Subsequent Course Success• Graduation Rate and

Retention• # of Graduates• Count of Graduates• Cohort Graduation• Cohort Retention and

Graduation• Student Progress through

Curriculum• Retention and Graduation

Success Rates• Academic Progress Report• Honors Students Graduated

from Honors College

Admissions

• Funnel report• Admissions comparative

data over time• Admissions reporting• Daily snapshot of

applications and registration

• Student registrations

Course/Class Management

• Class management• Time utilization• Class availability and

utilization• Course statistics for BAS

students• Course schedule

The above items represent the report or report goal that participants identified in their reporting story forms. Duplicates intentionally included.


Tiger Teams

Simplify Enrollment Reporting

Student Records

SME

Report Developer

ETL Developer

2-3 key report users

Tiger teams work best when they are small hand-picked groups focused on a specific task (e.g. simplifying enrollment reporting).

The team should include not only subject matter experts and power users, but also report and back-end database developers to speed up the process of weighing out potential solutions. These “tigers” are easily identified by being the go-to people in their area when a problem arises.

The activity should be time-boxed and the output should be a list of enhancements the team can prioritize and implement with the goal of delivering on the specific task assigned.

© 2012 Boise State University1 Data Warehouse User Engagement Readout Daniel Gold, Project Manager.

Documents