UNIVERSITY OF EXETER BI/08/21 BI PROJECT BOARD, 12 June …€¦ ·

UNIVERSITY OF EXETER BI/08/21 BI PROJECT BOARD, 12 June 2008

SolstonePlus BI Needs Analysis and Recommendations for the University of Exeter Attached is a document produced by consultants from SolstonePlus, a leading independent provider of BI solutions and services. The executive summary recommends a number of areas within the University which would benefit from BI, and sets out a strategic roadmap for the project. In addition, Section 5 of the document highlights three areas to be considered for the first phase of the project and includes a recommended approach for implementation. The three suggested areas are Student Admissions, Research and the Top 10 Key Performance Indicators (KPI’s). After considering these areas the following conclusions are put forward for discussion:

• Student Admissions is a good area to focus on but has already been visited in part as a data warehouse during a previous pilot project. Further work could form part of phase 1 but lacks the impact to be a good candidate for the main focus.

• Research is also a high priority, but in view of other developments will be difficult to achieve for a year or more

• Many of the 10 Super KPIs are annual or periodic measures that are not suitable for constant monitoring, e.g. the KPI for Student Employment rates is measured 6 months after graduation so it is difficult to measure or influence the result in any meaningful way until we start to look at planning and forecasting tools.

Therefore the area proposed as the main focus for the starting phase of the project should be those KPIs for which BI will provide the best improvements in terms of accuracy, timeliness and accessibility to data whilst considering the balance between areas where BI can make an early difference and our confidence to deliver. These then need to be considered and sorted by both importance and potential business impact. In view of this the recommendation is to consider the following KPIs:

• Postgraduate Research Students. To improve retention and completion rates. • Undergraduate Progression/Achievement. To improve retention and completion rates.

University of Exeter BI Needs Analysis & Recommendations

Version 3.0


Version 3.0

Page 1

Page 1


Version 3.0

Page 2

Document Control

Change Record

Date Author Version Change Reference

08/05/08 Jane Winter & Paul Cannon

1.0 No Previous Document

08/05/08 Graham Spicer 1.2 Amended following review 09/05/08 Graham Spicer 2.0 Amended following review 20/05/08 Jane Winter &

Paul Cannon 2.1 Amended following UoE

review 21/05/08 Graham Spicer 3.0 Amended following review 27/5/08 Sue Milward 3.1 Final amendments

Reviewers

Name Position

Graham Spicer CEO Mick Bull Sales & Marketing Director

Distribution

Version and Copy No.

Name Location

V2.0 Copy 1 Deborah Welland University of Exeter V3.0 Copy 1 Deborah Welland University of Exeter

All enquiries in connection with this document or requests for further information should be addressed to: -

Graham Spicer SolStonePlus 48a Old Steine Brighton East Sussex BN1 1NH Tel: +44 (0)1273 206555 Fax. +44 (0)1273 279139 Mob: +44 (0)7775 653956 Email: [email protected]

http://www.SolStonePlus.co.uk


Version 3.0

Page 3

Table of Contents

1 Executive Summary ........................................................................................................4

2 Introduction ...................................................................................................................6

3 Generic Requirements .....................................................................................................7 3.1 Data Integrity .....................................................................................................7 3.2 Use of Excel........................................................................................................7 3.3 Diversity of Data Sources.....................................................................................8 3.4 Use of Bespoke Systems in Individual Schools .......................................................8 3.5 Rollout of Best Practice........................................................................................8 3.6 Consistency in Business Definitions.......................................................................8 3.7 KPI’s ..................................................................................................................9 3.8 Business Structures .............................................................................................9 3.9 Reporting Levels .................................................................................................9

4 Business Requirements .................................................................................................13 4.1 Research ..........................................................................................................13 4.2 Student Admissions and Registration ..................................................................14 4.3 Programme Information.....................................................................................15 4.4 Competitor Data ...............................................................................................15 4.5 Marketing .........................................................................................................15 4.6 KPI’s ................................................................................................................15 4.7 Planning...........................................................................................................16 4.8 Financial Information.........................................................................................16 4.9 Alumni .............................................................................................................17 4.10 INTO Exeter .....................................................................................................17 4.11 NSS Data..........................................................................................................17 4.12 Space and Corporate Social Responsibility ...........................................................17

5 Recommendations ........................................................................................................18 5.1 Approach..........................................................................................................18 5.2 The Selling Exercise ..........................................................................................19 5.3 Phase One........................................................................................................19 5.4 Later Phases.....................................................................................................20 5.5 Identifying Key Areas for Phase One...................................................................20 5.6 Confirming and Controlling the Business Definitions .............................................21 5.7 The Data Model ................................................................................................22 5.8 Dimensions and Hierarchies ...............................................................................23 5.9 Ensuring Data Availability...................................................................................23 5.10 Identifying Reporting Requirements....................................................................24

6 Conclusion ...................................................................................................................25


Version 3.0

Page 4

1 Executive Summary Three discussion groups were undertaken on the 28th of April 2008 at Exeter University as part of the University’s overall Business Intelligence (BI) strategy in order to gather and consolidate their current BI requirements. This document collates the details obtained from these discussions and highlights the issues faced by the University in gathering, analysing and reporting key business information and recommends an approach to how these issues could be overcome to develop a business intelligence strategy. The reporting requirements highlighted by those staff attending the meetings can be grouped into specific areas, this should not be regarded as a definitive list and is only a reflection of the information gathered on the day; but may be considered as the areas which should be addressed first by a BI system. They are as follows:

• Research • Student Admissions and Registration • Programme Information • Competitor Data • Marketing • KPI’s • Planning • Financial Information • Alumni • INTO data • NSS Data • Space and Corporate Social Responsibility

However in addition to simply considering areas for BI reporting, consideration was given to where the data would come from, how accurate and up to date it would be and what issues the University currently faces in gathering and using this information. A number of generic areas of data gathering and use were highlighted which seem to affect most aspects of the current reporting procedures, briefly these being:

• Data integrity – The University would benefit from a single consistent source of data for BI reporting, enabling individuals to report data with confidence and understanding.

• Use of Excel – The wide use of Excel for producing BI reports, resulting in users downloading data from disparate sources and combining information using their interpretation of business classifications. This is time-communing and potentially inaccurate.

• Diversity of data sources – This requires individuals to identify the most appropriate data sources and then combine them, with all the inherent problems of mapping and interpretation.

• Bespoke systems in individual schools - The Schools vary in their data collection and reporting methods, an analysis of best practice should be considered.

• Consistency in business classifications – The University would benefit from a centrally held list of definitions, for example what is a Full Time Student, and how are breaks in study reflected.

• KPI’s – The University has a clearly defined set of KPI’s and Super KPI’s but these are not easy to report as they require data from many sources, reporting is static with little opportunity for drilling or time-based analysis.

• Business structures – Changing structures such as cost centre allocation and departmental reorganisation lead to issues with historical reporting, for example a report of the number of students in a School for the last five years will not produce a meaningful comparison if that School was merged with another during that time.

• Reporting levels – The Business Intelligence system should address all the reporting types required, management reporting, analytical reporting, process reporting, formal reporting (i.e. government bodies).


Version 3.0

Page 5

Further understanding of these aspects and planning how they can be overcome are a critical part of the design of a BI reporting system. These aspects are discussed in more detail and options for addressing them covered in the main body of the report. In summary, perhaps the most important point to come out of the discussions is the understanding that BI reporting is not a new concept for the University. The data and reports required to underpin the decisions being made on the day-to-day and strategic running of the University are being produced, however there is no cohesive structure to the data, how it be being collected or how it is being used. The result being that everyone gets the data they want eventually, but too much effort is required to gather it, it is being collected from too many different places and is often not directly comparable to similar reports of data due to differences in interpretation and understanding of the data. A Business Intelligence system can help the University pull all of the data from many different source systems into one place. It can re-order the data; transform it so that the same scales and measurements are being used, make the data available in a timely fashion without a lot of manual effort and provide a single reporting interface for everyone to use. This ensures that everyone uses the same data from the same source, that it is understood and interpreted by everyone in the same way. In this document we explain the approach the University of Exeter needs to take to being building a BI system to deliver this.


Version 3.0

Page 6

2 Introduction This document has been produced as part of the University’s overall Business Intelligence (BI) strategy, to gather and consolidate their current requirements for BI. It also highlights the issues faced by the University in data gathering, analysis and reporting. It is based upon three discussion groups held on the 28th of April 2008 at the University. The participating groups broadly covered the following areas:

• Research. • Teaching and Learning. • Overall Strategy.

Prior to the meeting these groups were asked to consider BI requirements where either no reporting facility exists, reporting exists but requires significant manual intervention to produce the required output, or reporting exists but has issues related to data quality or timeliness. These three areas were considered with reference to the following factors:

• How many users would benefit from the proposed report? • What, if any, financial benefit could be obtained from creating the report? • Would the report help the decision making process? • Does data for the report exist in an accessible format? • Is the report for general publication, or will additional security be required? • How should the report be produced and distributed? • Is exception reporting or alerting required? • Will the output require filtering options? • Can time based analysis be applied to the report?

As a result the participants where able to bring along a number of specific examples of the type of reporting they are currently performing and the issues they are encountering. The findings of the Needs Analysis can be summarised into two main areas:

• The generic issues faced by members of the organisation in their delivery of BI data. This focuses on the common issues that arose over all three groups and refer to general BI considerations rather than specific reporting requirements.

• The specific reporting requirements identified by individual groups or departments. Section 4 of this document details the recommended approach to enable the University to move to the next stage of their BI strategy.


Version 3.0

Page 7

3 Generic Requirements At present within the University there is no capability for BI global reporting within a single standard environment. There are a number of different technologies in use for delivering reporting functionality to the various Schools and departments within the University, one of the aims of the proposed BI project is to standardise these to ensure everyone understands and uses the same tools, data sources and business metrics. The result of this is that many different reporting systems and methods are used across the University, particularly within individual Schools. Whilst much of the data is being extracted from the same underlying data sources, interpretation of results can be varied and the individual effort involved in producing BI is probably being replicated across different Schools and departments. This section of the document outlines the challenges facing individuals and teams in their pursuit of reliable and sustainable BI data.

3.1 Data Integrity

One of the major challenges at the University, as in many organisations, is having a single consistent, up to date source of data for BI reporting. Without this individuals are unable to report data with confidence and understanding to both internal parties and, perhaps more importantly, external bodies, such as the Higher Education Statistics Agency and the Higher Education Funding Council for England, where the reputation of the University may be under scrutiny Common practice at the moment involves taking data dumps from operational systems such as RAG, in the case of Research, which are then loaded into Excel or other packages and manipulated to produce the required output, such as financial planning forecasts. This practice can lead to mistakes in data analysis and misinterpretation of key business metrics; it is uncontrolled and reliant on an individual for publication. Using a BI system will ensure that data is mapped, loaded and calculated correctly by a small group of people with specialist knowledge. Users will then no longer need to manipulate data before reporting but just use the BI tools available. In addition to the downloading of data from operational systems for individual use, there is also the practice of individual data collection and recording within Schools, this again can lead to issues of data integrity. An example highlighted was difference in the reporting of financial data, where the data in an individual School varied from that held centrally largely due of the timeliness of record keeping, as individual Schools tend to update records more quickly than those held in the central repository.

3.2 Use of Excel

Much of the BI information currently produced by the University is done through Excel. Users are frequently extracting data from a variety of sources and exporting it to Excel to perform their analysis. In addition to the obvious problems of data integrity referred to in section 2.1 it also takes a great deal of time and effort for users to do this, furthermore re-publishing of information can be cumbersome as macros or lookups need to be rebuilt to cope with changes in business rules, such as new groupings. Representatives from marketing referred to an individual instance of where four data sources were required to produce information on student admissions, because of differences in the data sources and missing values, a total effort of almost three man days were needed to produce the final output.

http://www.hefce.ac.uk/


Version 3.0

Page 8

3.3 Diversity of Data Sources

One of the other issues facing the University is the number and diversity of their data sources, this leads to individuals downloading data, either by themselves or on request by IT staff, from a number of operational systems in order to produce meaningful BI output. These data sources, in reality, have different reporting standards and business classifications, which require individuals to create their own meaningful mappings and interpretations. Some of those data sources used currently which should be considered for standardised BI reporting are listed below:

• SITS (Student Record System, mostly accessed via Discoverer) • RAG (Research Admin and Grants System, Access database - due for replacement) • Aptos (Finance System) • X-Change (Holds information on Business links with external bodies - due for replacement) • InfoEd (Manages Research Costing) • MidlandHR (HR and Payroll) • Raisers Edge (Fund raising system used by External Relations) • Scientia (Timetabling software)

In addition to the internal operational systems the University utilises data from external sources such as HESA and UCAS.

3.4 Use of Bespoke Systems in Individual Schools

The requirement for up to date, accurate data within an individual School has driven the ad-hoc development of bespoke systems within them. The Schools vary significantly in their size and complexity and this is reflected in the data collection and reporting systems they use. As an example in the research area, some Schools have a Web based facility for data collection; this means data is current as changes are reflected immediately. Unfortunately the central source of funding held by Finance tends to be published monthly and is a snapshot of a particular moment in time; this causes issues of data integrity as the two sources rarely converge.

3.5 Rollout of Best Practice

This wasn’t discussed at the meetings but is a general observation based on the previous point. It was widely agreed that individual Schools vary significantly in their data collection methods and BI and operational reporting, consideration should be given to the rollout of best practice from an individual School to others in the organisation. It is certainly worth identifying particular areas for further analysis; for example the practice of collecting and reporting research funding via the web used by some schools. It will allow for consistency in approach and should make the centralisation of BI reporting less onerous.

3.6 Consistency in Business Definitions

Business definitions are the keys to understanding data and business performance. These are the standard interpretations any business places on key pieces of information. An example would be Full-Time Student and what is meant by this. Typical questions arise such as, which students are covered by this? Does it include students who take a gap year but are full time either side of the gap? Does it include Research students? What about students who take a long period off for illness? Depending on the individual and the purpose of the report the exact definitions of Full-Time Student will probably change leading to confusion and misinterpretation when different reports are compared. This is compounded when users know the data doesn’t fit their interpretation and try to make adjustments to it, using calculations applied to the final result rather than the base data.


Version 3.0

Page 9

The solution is to define exactly what each definition means – having multiple versions if needed, but with different names, so that each version can be easily identified. The rules and algorithms that are used to derive each business definition should be agreed and controlled by a small group of people with specialist knowledge. Centralisation of BI reporting should ensure that anomalies are removed and agreed definitions more widely understood. The whole University then needs to sign up to these and agree to their use. This will ensure users are able to correctly interpret the data in reports.

3.7 KPI’s

The University has a set of agreed KPI’s; these can be broken down into two types: ‘Super KPI’s’ (10) and ‘Supporting KPI’s’ (12). It was clear at the meetings that these are widely supported, work well and form the basis of BI reporting. Whilst these are accepted throughout the University as good performance measures, they are not easily generated or reported. Many of the KPI’s discussed during the meetings can only be reported using data accessed from multiple sources. This data is then consolidated by individuals or departments (with all the inherent problems of mapping, data interpretation and manual effort associated with this practice). In addition reporting is static with no opportunity for drilling to lower levels for detailed analysis or time based comparisons. A standard BI reporting tool would enable instant access to KPI’s across the organisation with the opportunity for individuals to drill own to their level of interest.

3.8 Business Structures

Operational systems tend to hold a single definition of business structures and are generally poor at correctly representing the structure of an organisation as of a month or a year ago. This is because they are constantly evolving in line with the needs of the business, but this can cause a problem if the output requires reference to one or many historical definitions from the past. This can be seen in areas such as cost centre allocation and departmental reorganisation. The number of individual Schools has decreased over recent years as the Schools have been reorganised and merged together. A report of the total number of students in a School for the last five years will not produce a meaningful comparison when that School was merged with another three years ago. In the above example there would be two records for this School, how it is now and how it was three years ago any BI reporting tool must address how to switch between the two in order to provide a historical context. By storing both definitions a user could choose to report on ‘actual’ numbers or ‘adjusted’ numbers – adjusted so that data three years ago is represented as if the School structure back then was as it is now – hence providing an accurate comparison over time.

3.9 Reporting Levels

When considering a new BI reporting system, it is necessary to highlight what type of reporting is currently undertaken. This falls into four basic areas:

• Management Reporting • Analytical Reporting • Process Reporting • Formal Reporting


Version 3.0

Page 10

Management Reporting Management Reporting tends to be around fixed high-level views of the data, usually in the form of Key Performance Indicators & Business Metrics. These views need to be updated frequently to provide up-to-date measures of performance. Some level of drilling into lower level detail reports is required to allow a quick analysis of what is happening. This level of reporting needs to be very simple, allowing managers to see the state of play, with anomalies and performances issues being highlighted to ensure a fast response, e.g. dashboards. Analytical Reporting Analytic users within the business units require the ability to investigate data, looking for underlying information. For example the complex analysis of financial and non-financial data, i.e. costs vs. staffing levels viewed over time. These users need tools that allow them to quickly drill into data, pull data from disparate sources, and display data in different views. The tools need to be simple to use, but complex enough to enable the users to accomplish their tasks. These users generally need the most training to ensure they can get the most out of the tools. Process Reporting This can be classified as ‘standard’ reporting. In terms of complexity, these reports are usually quite simple but they often require the lowest level of data and need to be up-to-the-minute, and hence often need to be run against a production application database rather than a data-warehouse. Formal Reporting These are the ‘legal’ documents that underpin the business. At the University these would tend to be reports to government bodies which require exacting standards in pre-agreed formats. The data they contain must be 100% accurate. These ‘reports’ may also be stored for future retrieval both for business and audit purposes. These requirements fall into a pattern that is identical to almost every other business organisation: A Reporting Hierarchy, usually known as a Reporting Pyramid:


Version 3.0

Formal Reporting

Underpinning the reporting pyramid are the actual databases. Production databases for ‘live’ reporting and data warehouses for the remaining data imported and structured ready for reporting. At the lower level of the pyramid are the bulk of the users – producing reports that support the day to day business activities and requiring basic reporting tools. These usually have the largest number of reports and require the most data to populate them. The mid level users tend to look at a higher level of data, usually summarised, with advanced tools to help them. There will be fewer mid-level users than lower level users and the quantity of data they view will be less. At the top are the management users. These only look at small quantities of highly summarised data, usually in the form of key performance indicators. There are only a small number of these users with simple view-at-a-glance reports. Both Management and mid-level users have the ability to drill into data in the levels beneath them but only on demand when they specifically need to, lower level data would not automatically be presented to them. The data in the reporting pyramid starts at the lower level in the databases and is populated upwards with aggregations and summarisations being performed as it goes. The most important point is that the source of the data for all three levels is the same therefore everyone sees the same data (“one version of the truth”), however the higher up the pyramid the user, the more summarised the data. The reporting tools used also change in each level. Lower level users use basic tabular reporting tools. Mid-level users require advanced tools to provide the analytic capabilities they need. Management users tend to need specialised reports and dashboards that display the data in summary ‘at-a-glance’ views.

Page 11

The pyramid structure can be flexible. There are often mini-levels within the three basic levels. A user near the top of the lower level may need all of the lower level reports, but also some of the analytic capabilities of the mid-level and a manager may want to drill into a KPI and perform their own analysis on the underlying data rather than wait for an analyst to do it.


Version 3.0

Page 12

Any individual user should therefore have the ability to access all of the reporting tools they need, not just one or the other. The pyramid may also represent security levels. Summarised data that provides overviews of the organisation may be sensitive. So as data is summarised it may also have the necessary security applied to control access.


Version 3.0

Page 13

4 Business Requirements This section covers the specific reporting issues raised at the three meetings held on the 28th of April 2008. The requirements listed below should not be considered definitive; they are representative of the opinions of those staff that were able to attend one of the three sessions. They are grouped into business/departmental areas for reporting purposes but in reality many are interrelated. They are not necessarily listed in order of importance. The summary below lists the main areas that have been considered so far.

1. Research 2. Student Admissions and Registration 3. Programme Information 4. Competitor Data 5. Marketing 6. KPI’s 7. Planning 8. Financial Information 9. Alumni 10. INTO data 11. NSS Data 12. Space and Corporate Social Responsibility

It is possible that other business areas may need to be considered for BI reporting outside of those covered at the meetings. It should also be emphasised that this is not necessarily a full account of the current situation within the areas covered by the report, only a representation the information gathered on the 28th of April.

4.1 Research

The Research area was highlighted in the fact finding exercise as one of particular concern. This area faces many of the same challenges as other parts of the University but was widely considered to be the most likely to benefit from a standardised approach to BI reporting. Some of the particular issues highlighted were as follows:

• The availability of robust data for the calculation of ‘Super’ KPI’s as identified centrally by the organisation.

• Difficulties in combining data from different research applications; currently the Je-S system is used to prepare electronic research grant proposals for AHRC, BBSRC, EPSRC, ESRC, NERC or STFC. This covers only about a third of all University funding applications.

• Issues with CRM data currently held in X-Change, the data is incomplete and out of date. However the system is in the process of being replaced, the new contact system will also contain the data currently held in RAG.

• Issues with accessing data via data dumps from diverse operational systems such as RAG, SITS, MidlandHR, Aptos which require manual mapping within products like Excel.

• Difficulties with producing standard returns for external agencies such as HESA and HEFCE, and the related issue of ongoing changes in reporting requirements, particularly in the case of the Research Assessment Exercise (RAE).

• Differences in the recording and reporting practices of individual Schools and Campuses and the difficulty with matching this to centrally held data.

• Problems with the accurate forecasting of project funding. • Missing information in areas such as failed funding applications and internal and external

collaborations between academics. • The ad-hoc purchase and use of externally available competitor data, which is driven by an

individual School, but may be of interest to a wider audience.


Version 3.0

Page 14

• Mapping issues, for example in the case of UoAs and HESA cost centres which need to map into the department and school hierarchy for RAE submissions.

• Business classifications, for example the definition of what makes up research income i.e. which categories should be included.

The reporting requirements and key business definitions that were discussed are listed below:

• The centralisation of data collected at individual School level e.g. up-to-date financial records collected via a web interface at School level.

• A Single easily accessible source of research funding information, by funding type (i.e. internal, external, self).

• Information on both internal and external collaborations, such that individuals and groups with specific expertise can be identified.

• Number of post-graduate research (PGR) students per staff FTE. • Research income per staff FTE. • Earned income expressed as a percentage of all income. • RAE metrics by units of assessment (UoAs). • Post Graduate Research (PGR) completion rates, percentage of completion within 7 years,

percentage completed within ‘n’ years. • Success level of funding applications, including reasons for funding refusal i.e. lack of money,

poor submission or re-submission invited. • Use of HEFCE Research Activity Survey (RAS) information, linking and understanding staff/student

data. • Wider use of HESA supplied competitor data to compare Exeter with other Univerisities

4.2 Student Admissions and Registration

Particular issues that were highlighted by individuals gathering and reporting data in this area were:

• The diversity of data sources, for example users needed four different data sources to produce one BI report and because of incomplete data and issues with mapping and interpretation, this process took the equivalent of 3 man days to complete.

• Slowly changing dimensions and issues with historical reporting; because of changing dimension definitions, i.e. the structure of departments, historical reporting becomes difficult. BI reporting must accommodate this and make data comparable.

• Issues with data structures, these often need to be modified before reporting, for example a student could be allocated to two departments but actually only reported against one. Specific business rules may need to be designed to address these types of issues.

• The definition of specific business metrics, for example how a FTE is derived, these should be standardised and held centrally to avoid individuals generating their own interpretations.

• Unreliable data in SITS caused by different timeframes e.g. current data versus a snapshot. Some of the required metrics identified were as follows:

• Student numbers and admissions. • Student feeder institutions. • Student country of origin. • The number of applications compared to actual registrations. • Filtering students to identify special requirements i.e. in the case of a disability.

This data would need to be accessed over a number of years to allow time-series analysis.


Version 3.0

Page 15

4.3 Programme Information

Particular issues that were highlighted by individuals gathering and reporting data at this level were:

• The degree of manual effort needed to produce data at this level. • Issues with comparing data especially in a historical context. • Programme data held in SITS which is not easily accessible for use.

The business metrics are largely the same as those for Student Admissions and would also require time based analysis:

• Student numbers and admissions for an individual programme. • Student profile data; users would like to drill from a programme through to an individual student. • The analysis of programme information by student profile; for example to be able to identify a

programme that is particularly popular with students from the USA. • The number of applications compared to actual registrations.

4.4 Competitor Data

This area of analysis requires data acquired from external sources such as HESA and UCAS. Particular issues that were highlighted by individuals gathering and reporting data were:

• Issues with missing or incomplete data. • When data is acquired it is not always communicated to other schools. • Do not make the best use of external data. • Potential mapping issues caused by changes in definitions.

Key business metrics discussed were:

• The comparison of course take-up between Universities. • The comparison of offers made to students.

4.5 Marketing

The key business definitions highlighted by Marketing staff were:

• The analysis of effectiveness of Post Graduate Marketing campaigns overseas. This could involve the comparison of marketing spend against registration.

• The analysis of trends in Post Graduate applicant declines; to help focus marketing initiatives.

4.6 KPI’s

As previously stated the University currently has a set of agreed KPI’s, these can be broken down into two types: ‘Super KPI’s’ (10) and ‘Supporting KPI’s’ (12). These are a key consideration when analysing BI reporting requirements, they should be drillable and related, for example, by making comparisons between entry standards and employment prospects. A number of individual KPI’s and super KPI’s were discussed at the meeting and these are listed below, however it should be noted that this is not a definitive list:

• Earned income, expressed as a percentage of all income. • Percentage of students who complete. • Percentage of students who continue to study. • Percentage of students who are employed six months after completion. • Student to staff ratio.


Version 3.0

Page 16

• Research income per staff FTE. • Number of PGR (post graduate research) students per staff FTE. • Under-graduate entry qualifications (UCAS sourced data). • Graduate employment. • RAE outcomes. • NSS information. • Percentage of non-UK students.

4.7 Planning

The planning process uses ‘actual’ reports to predict future demand. This area faces similar challenges to other areas of the business:

• The ‘actual’ reports are time-consuming to create and analyse, particularly when mapping through to individual Schools.

• There can be issues with the source data in SITS particularly when a student moves between programmes or interrupts their study.

Key business definitions discussed were:

• Student information such as number of FTEs, or basic student characteristics. • Module level information that can map through to individual Schools.

4.8 Financial Information

A number of the issues identified in this area have already been covered, they are as follows:

• Issues of mapping financial data against non-financial data for the reporting of KPI’s and other business metrics.

• Difficulty of costing programmes effectively due to the different workload models adopted by each School.

• Differences in the recording and reporting of financial data by individual Schools or Campuses and the difficulty with matching this to centrally held data.

• Definitions of groupings and business rules i.e. FTE. • Drilling to lower level data for example at individual School level.

Key business metrics discussed were:

• Research income per staff FTE. • Earned income. • Programme costing information delivered in a standardised format.

One key area for consideration that was not highlighted during the BI needs analysis workgroups but has since been raised was BI as related to the Income Distribution Model (IDM). This model looks at the income and costs associated with individual Schools. There are two key metrics for consideration:

• Taking census data from SITS and using it to calculate T grant allocations and to provide student FTE at School and module level for income projections.

• Taking actual income data from Aptos and matching it to census data to allocate the income by School and module.


Version 3.0

Page 17

4.9 Alumni

Raiser's Edge is the University's fundraising and Alumni contact management system. It manages all of the day-to-day details and interactions with Alumni, volunteers and donors. Donors are becoming an increasingly important income stream for the University, identifying potential Alumni through BI by linking disparate information such as salary and destination data would represent a significant enhancement over current practices.

4.10 INTO Exeter

INTO Exeter prepares international students for undergraduate or post-graduate study at the University. Data from INTO could be made available for BI reporting, suggested areas for consideration are:

• A breakdown of the applications that are made to INTO using student data for example, age, nationality etc.

• A breakdown of conversion rates from INTO, this may not need another feed from INTO as the original application data could be mapped against that held in the University SITS database.

4.11 NSS Data

NSS (National Student Survey) data available from HEFCE is an external data source that is currently in a downloadable Excel format. The data is made available to individual Schools but often requires reformatting before use. The survey covers levels of student satisfaction in areas such as: teaching, assessment and feedback, academic support, organisation and management, resources, personal development and overall satisfaction.

4.12 Space and Corporate Social Responsibility

Particular issues that were highlighted by individuals gathering and reporting data for this area were:

• Whilst some data for the analysis is available through the space database and staffing information other data may need to come from sources that are not currently available.

• Targets and performance in the context of corporate social responsibility are likely to become more important as they rise up the social agenda. This could potentially mean a wide external audience for these measures.

Key business metrics identified were as follows:

• How space is used in the context of occupancy rates/staff and students. • How students use space within the City. • The university’s carbon footprint • Other aspects of Corporate Social Responsibility.


Version 3.0

Page 18

5 Recommendations Given the issues and requirements listed above, SolStonePlus suggests that the following recommendations should be given careful consideration when planning the implementation of a BI system at the University.

5.1 Approach

Probably the most important piece of advice that can be given is not to try to achieve everything in one go. To develop a BI reporting system capable of supporting all BI reporting within the University would be a very large and complex task and would take far too long to implement – with the inevitable consequence that many of the original requirements will have changed before the development is complete. It is far more sensible to take a phased approach to building a BI system, tackling a small number of requirements within each phase. This way some BI reporting can be accomplished in a relatively short space of time and the system will grow and adapt to further requirements over time. It is unlikely that a wide ranging BI reporting system covering the whole University will ever be truly “complete” as requirements will constantly change in order to meet the demands of an ever changing sector. One of the most important aspects of the BI system is the design, fundamentally how will all of the disparate data fit together? Obviously all of the University’s requirements for accuracy, timeliness, ease of access etc… must be met, but the design must allow for constant change – initially with new data being added with each development phase and ultimately with changes to the data due to changes in the University’s operations. There is no standard or unique method of designing BI systems to meet all of an organisations needs; every BI system is different because the data sources are different, as are the definitions of the data and how the data is to be used. SolStonePlus’s advice would always be to start small – tackle a small number of specific areas, ideally areas that are common across the University, such as staff, student and course details. Make sure all data items are fully defined and documents, define a list of naming standards for data objects along with common data types and display formats. Identify data keys as early as possible and define what those keys are and how new keys must map to them. If data comes with obscure keys consider implementing new keys within the BI system (known as surrogate keys). Once the core of the design has been defined, new data items can be added, but again in small pieces, ensuring that duplicate data items are removed (or merged if appropriate), that all keys match or are transformed and that the new data items confirm to the same naming, data type and display format definitions. Also consider removing data items that are pre-calculated by the source systems – it is often beneficial to allow the BI system to calculate them – less data needs to be loaded and far more control can be exercised over the calculation, ensuring it fits with the business definitions and matches similar calculations in other areas of the system. An important aspect of BI reporting systems is that they can also provide non-BI reporting. With the data in a an easy to use, accurate, up to date format, there is every reason to maximise its use by allowing for the production of standard operational reports, such as letter production or time table printing. It does, however, need to be controlled in the early phases of development otherwise a tendency to concentrate on the operational reports takes over.


Version 3.0

Page 19

Once the first phase is complete and later phases are being worked on, there is the potential issue of the need to make many enhancements to the data model simultaneously – especially if users start making individual requests to add extra data into their own reports. It is important therefore to separate out the design and control of the data model from the report building process. A separate team should have sole responsibility for the data model – only they can add in new data fields, amend existing data fields or even delete or replace data fields. That way it can be guaranteed that the changes are consistent with the data model design and don’t replicate or conflict with data that already exists. It also ensures that the data model continues to meet the requirements of the business metric definitions. This point is very important to the success and growth of the BI system. If large numbers of users are able to add data into the model then the control of the data model will be lost and it will become unwieldy, data will become difficult to locate or understand and confidence in it will be lost.

5.2 The Selling Exercise

The University will only benefit from a BI Reporting system if the staff (and students/external personnel where applicable) make full and proper use of the system. Users will only use it if they have the confidence that it can produce the reports they require and continues to do so in a way that surpasses current systems and processes. The system must evolve. It is important that users are encouraged to suggest improvements that make their working lives easier and also assist departments with greater integration. A BI system isn’t simply a list of reports the users can run – it must provide the ability for users to create their own reports and to enable analysis of data in different ways, often ways that have not previously been considered before due to their complexity, or even the non-availability of data. Users need to be encouraged to make full use of the systems functionality and capabilities and to be discouraged from automatically downloading data to Excel for analysis because they are more comfortable with that. In short, users need to be “sold” the BI system. Users need to be exposed to the system at all stages of its development in order to appreciate the benefits it brings. The phased approach to development is ideally suited to the implementation of BI systems. It is possible to bring users onboard and using the BI system at different stages during the development and deployment which enables maximum exposure and for internal selling to other users in other departments and Schools. Users will hopefully become more enthusiastic and helpful in assisting with the definition of their personal and departmental requirements.

5.3 Phase One

The tasks listed below must be part of the initial phase of any BI development:

• Design of the core data model - to accommodate the data for phase one and be flexible to permit expansion in later phases. This includes establishing the team responsible for building and maintaining the data model.

• Identify key data sources and analyse data quality, identifying discrepancies and missing values. Set rules to handle missing or invalid datay.

• Inclusion in the model of key data items which apply across-the-board in most, if not all, areas of the University. This includes ID codes, Name and common attributes of the key identifiers of data such as Student, Staff, Departments, Schools and Programmes. Examples of common attributes would, for Students, include elements such as Nationality, Gender, Full/Part time flags and HESA codes, Even if their use is limited in the first phase suite of reports, including them now will make for easier development in later phases.

• Understanding the data requirements for the next development phases so that their implications can be considered early on. This helps with ensuring that the data model does have the necessary flexibility built in.


Version 3.0

Page 20

• Setup of the core meta-data repository. In addition to supplying the data to the reports being built in the first phase, this will set the standard for the inclusion of more data in later phases.

• Identifying a suite of key reports that the phase will deliver. • Defining the Business Definitions for the data included in this phase, such as all of the required

definitions for FTEs. • The realisation of a business benefit. Simply replicating a number of reports isn’t enough, users

need to look at those reports and feel that they are better than those they have at present – faster, more accurate, and easier to use.

• Capture users’ attention and imagination. This will enable users to visualise how later phases may benefit them. To this end, spending some extra time on the presentation is important. Using the Universities colours and logos, using colour highlighting, perhaps defining a standard screen layout.

The time scale for Phase one is important. The message needs to be sent out that a BI system is being developed, but it needs to deliver benefits before its impact is lost. A sensible time period for a first phase is 3 months. This is often quite a tight timescale but that helps focus on the important aspects and prevents scope-creep – one of the biggest problems with BI development, the “whilst I’m working on this bit, why don’t I add in this extra bit as well?” scenario. It is very tempting to do that with BI, but needs to be avoided during the early phases of development.

5.4 Later Phases

Later phases of development should be planned during the development of phase one, so that their development can start as soon as phase one is complete and a rolling programme of development and deployment can be instigated. Whilst phase one generally tries to achieve part of the requirements in a few key business areas, later phases should be aimed at individual business areas, so for instance phase 2 might be to complete the BI reporting requirements in Research, and phase 3 might be targeted at Alumni reporting. Provided the University has the necessary resources there is no reason why after the first phase is complete, later phases could not be developed in parallel, i.e. phase 2 and 3 could be developed alongside each other. The key will be the involvement of the team tasked with building and maintaining the data model. Once data has been added to the model, different teams of people can populate the meta-data repositories and build the reports themselves.

5.5 Identifying Key Areas for Phase One

Identifying the key areas for phase one is obviously critical to how well the BI reporting system will be received by users within the University. The reports delivered must:

• Have immediate benefit to some users within the University. • Provide a level of data that management are not currently receiving either timely, accurately or

easily. • Allow drilling to view the data underlying any management level statistics

Also, if possible, it is a good idea to include one or more elements of reporting that currently suffer from existing issues with respect to integrity or performance. This enables the project to demonstrate “quick-wins”. Following the discussions held on the 28th April 2008 and considering the requirements mentioned both on the day and subsequently, SolStonePlus recommend that reporting from the following three areas be considered for phase one:


Version 3.0

Page 21

Student Admission and Registrations Being part of the core business of the University, Student Admissions and Registrations will be an area most, if not all, users will have some knowledge of and hence will be able to see and understand the benefit that BI reporting is bringing in this area, even if not directly relevant to themselves. It also includes many of the core data items used across the University, such as Student, Staff, Department and programme details. Some work has already been carried out in this area in a prototyping phase. While this may need expanding to include more data, it is an initial step where some work has already been undertaken and this should be considered for inclusion. It is also an area where there are major reporting issues at present, mainly because data has to be collated from a number of sources and manually built into a report that often takes days to generate, so an immediate business benefit should be achieved by its inclusion. Research This is a particular area which seems to suffer quite heavily from issues around matching data from multiple databases, data completeness and the manual effort required to produce reports. There is therefore also the question of reliability and accuracy of this data. Research is one of the main areas in which the University receives funding, so this would provide a financial angle to BI reporting and it also feeds some of the ‘super’ KPI’s. ‘Super’ KPIs The ‘Super’ KPIs obviously play a major role in how well everybody understands the University’s performance. Displaying these in a dashboard or perhaps a regular emailed newsletter style report would clearly demonstrate “typical” BI high-level Reporting. Combining this with drilling to the underlying data will help get across the message that the KPI’s (and BI reporting in general) isn’t just about single numbers, but about the data supporting them. Two KPI’s particularly highlighted by attendees were:

• Research income per staff FTE. • Number of PGR (post graduate research) students per staff FTE.

These KPI’s seemed to have the greatest significance to the audience and the overlap with the research area would indicate that they should be a high priority for phase 1 These three areas are very large and it may well be considered too much to achieve in a short three month implementation project. It is therefore important to address the key areas – those areas where there are specific and regular reporting issues, whatever the cause. Whether or not the University decides to adopt the above three areas, or choose different areas from the business requirements in section 3, a more focused requirements gathering exercise is required to identify the actual individual requirements and enable the areas to be included in phase one to be short-listed. Remembering always that any additional requirements in these areas will not be forgotten – merely put off until a later phase.

5.6 Confirming and Controlling the Business Definitions

One of the important steps in phase one is the creation of the team that will control the Business Definitions in the system – carefully defining those terms that are used generally within the University environment, but that might actually mean something different to users from different departments.


Version 3.0

Page 22

The likely outcome of this is that multiple definitions for some data items will still be needed, but they must be uniquely identifiable, with specific names. This will result in the publication of a “Glossary” or “Dictionary” alongside the BI system that can be used for lookup by users. It may also mean that common terms such as “Full Time Equivalents” are not directly used, being replaced with “Standard Full Time Equivalents” for all members of staff with alternatives such as “Research Full Time Equivalents” for research related staff only.

5.7 The Data Model

The data model is the key to any BI reporting system and will probably account for the majority of the development effort. All of the data required by the reports needs to be identified, all lookups between different databases resolved, any data issues cleared up and any additional calculated data built in. (these are just a few of the steps). Like the Business Definitions above, a team of people needs to be established as the owners of the data model and only these people must be permitted to amend the data in the model. This is the only way of ensuring a smooth, accurate and consistent data model across all business areas and all databases. A decision needs to be made as to whether to use a data warehouse, or possibly multiple data warehouses. A data warehouse is by no means compulsory in a BI reporting system, but more often than not there is at least one as it allows data lookups, conflicts and missing data issues to be resolved before the data is made available to the BI reporting tools. It also minimises the performance impact on the source systems. This decision should be based on a number of factors:

• Performance. If additional reporting would adversely affect the source systems then the data would need to be extracted out into a separate database.

• Complexity. If the data in the source systems is very complex, or there is little knowledge the University about systems provided by third parties, extracting the data into a separate database may be the best method of gaining control of its use for reporting.

• Integrity and cleanliness issues. If there are known issues with a source system using an separate database to clean and improve the data may be the simplest option

• Matching data. If data from two systems needs to be merged for reporting, this may be easier to do in a separate database rather than attempting to match the data when reports are required.

• Cost and work involved. Taking data into a separate database can be a costly and time consuming task, so the decision to do so should not be taken lightly.

In the case of the University of Exeter, SolStonePlus strongly recommends a data warehouse for the Student Admissions & Registrations and Research areas because of the sheer number and complexity of source databases. Once the individual business requirements for phase one have been defined the next step is to identify the actual data required – the source systems, what mappings and lookups are required and any missing data or data integrity issues that need to be resolved. A development team can then immediately start working on this model. There are a few specific design aspects which SolStonePlus recommend should be included in this model to ensure maximum flexibility. Without going into exact definitions of these here they are mentioned so as they can be borne in mind then the data model is designed.

• Slowly Changing Dimensions (Type 1 and Type 2) - to record changes in dimensions (such as university organisation) over time.


Version 3.0

Page 23

• Surrogate Keys for Dimension members - partly for slowly changing dimensions, but also to divorce data in the data model from the source systems to minimise the impact of future changes to the source systems on the data model.

• Use of a data modelling tool to ensure the process used to map the data is fully documented in a way any developer would be able to understand quickly and easily.

• Pre-seeding the dimensions and hierarchies with ‘Null’ values to allow a range of different options for handling missing data.

The design of the data model also includes the design of the meta-data repository. A meta-data repository is a “library” detailing all of the data available to the BI reporting tools and includes a lot of information about how the data it to be used, how it is displayed and how data from different sources is linked together. It is this repository which holds much of the intelligence which enables BI reporting to use, calculate and drill through the data which provides a much richer set of reporting functionality than that available in normal reporting systems. Just like a database, there are many ways of designing a repository and careful thought needs to be put into how this is done. In addition the repository needs to have any and all additional calculated data items included to minimise the development effort for the users. The data model also needs to consider security aspects of the data. Security can be built into either the underlying data sources (or data warehouse), or the meta-data repository or in the actual reporting tools that the users see, or of course a combination of these areas. However security is to be implemented, it must be considered early on in the design of the data model to ensure nothing in the design prevents security or impairs performance.

5.8 Dimensions and Hierarchies

One of the most important concepts within Business Intelligence is the “Dimension”. A dimension is defined as “something about which we have information”. For instance, a programme - A programme by itself is not a useful piece of information, but the number of students on a programme would be. So programme can be considered a dimension with the number of students a piece of data “dimensioned” by programme. Most BI Reporting tools are capable of understanding dimensions and will therefore do much of the “development” work automatically, thereby dramatically reducing the amount of actual work required to create a report. Dimensions also have hierarchies – that is clearly identifiable groupings to which an item belongs – for instance a course belongs to a School, which may in turn belong to a Faculty. These hierarchies enable the fast aggregation of data and allow for drilling into lower level data by users. By understanding the course-school-faculty hierarchy a BI reporting system can automatically take the number of students on each course and provide totals by Schools and Faculties. This issue cannot be understated – clearly defining the dimensions in a data model can reduce the development time of reports by up to 80% and it is this aspect which plays a major role in making BI systems easy enough for users to create their own reports. An important part of the phase one data model design is therefore to identify the dimensions within the data and to define the hierarchies and other attributes which will make reporting a lot simpler later on, both in the first and later phases.

5.9 Ensuring Data Availability

Before finalising the requirements for the first phase, confirmation is required that the source systems are available during the development time period of this phase.


Version 3.0

Page 24

If other developments or outages are planned for the source systems during, or especially close to the completion date, this may affect the delivery of phase one from both an availability point of view and possible changes to the data and data structures in those systems.

5.10 Identifying Reporting Requirements

The last area that needs careful consideration before a BI system can be developed is how the users want to view the data, and most BI reporting tools provide a very wide range of options. If this is not planned in advance for phase one there will be no clear vision of what phase one is to deliver and the resulting deliverable will fall short of meeting both requirements and expectations. It is important to consider the method of delivery; there are quite a few, but the main options being:

• Dashboard • Ad-Hoc Reports • Printed • Emails • Excel/PowerPoint • Web

Once a “destination” for each report has been decided, then it’s a question of the style of the report – should it be a simple table of data, should there be some extra formatting – colours, highlighting, logic not to show unimportant data, should the data be shown in a graph – bar, line, pie etc… is more than one type of output required (some users want a table in an email, others want a graph in a dashboard). Just like every other aspect of the BI system, none of this needs to be set in stone – every report can be changed later on, ideally the users should be able to do it themselves online rather than submitting a change request to the IT department. It is important, however, to get a starting point – a suite of known reports with a specific output style as an initial delivery objective.


Version 3.0

Page 25

6 Conclusion In conclusion to SolStonePlus’ recommendations to the University of Exeter it needs to be stated that there is a lot of work needed to address the issues with high-level reporting and the data in the existing systems, but that there is also much benefit that will be gained by doing so. Whilst some scepticism was evident in the three group discussions, the overriding impression was that most of the attendees agreed that a BI system would greatly help overcome many of the issues they all recognised existed. The scepticism was mainly focused on the approach to resolving issues in specific areas and without the time to investigate to the level of detail needed, SolStonePlus were unable to address every area other than explain the concepts behind Business Intelligence and how data needs to be organised, matched, cleansed and presented to the users in a consistent format. SolStonePlus recommend that the University take a phased approach to building a BI system, tackling a small number of requirements within each phase. This way some BI reporting can be accomplished in a relatively short space of time and the system will grow and adapt to further requirements over time. The initial phase should concentrate on delivering high profile achievable goals within a fixed timeframe; this will avoid the danger of scope-creep. In our experience a sensible timeframe would be in the region of 3 months. This is challenging but will help focus development on the most important areas of the system. Some consideration should also be given at this stage to the type of reporting required in phase one. Management reporting would have the highest profile and process reporting should probably be avoided until later phases. Once the business requirements for phase one have been defined the actual data required must be identified. This will involve confirming the source systems, identifying and resolving data integrity issues, agreeing mappings and lookups and recording business definitions. It is recommended that the business definitions are agreed and controlled by a small team with specialist knowledge; the definitions should be uniquely identifiable and be widely circulated around the University in the form of a published document. One of the most important aspects of the BI system is the design. How all of the disparate data sources fit together? Will a data warehouse or multiple data warehouses be required? It is important to separate out the design and control of the data model and meta-data repository from the report building process, a separate team should be identified and only these members would be permitted to amend the model, thus ensuring consistency throughout the organisation. The initial model will need to be future proofed to ensure that the data elements intended for later phases are easily incorporated. Later phases of development should be planned during the development of phase one, this will ensure that the design incorporates the elements required for later phases and that they can begin as soon as phase one is complete. Whilst phase one will concentrate on a few key areas later phases should be aimed at whole business areas such as research or KPI’s. The final point for consideration is how the reports will be delivered to the users (i.e. dashboards, email, alerting) and in what style? Most BI reporting tools offer a wide range of options, this should be planned in advance of phase one to avoid deliverables falling short of expectations. SolStonePlus make these recommendations based on our experience and the recognised best practise procedures within the BI industry. However it is ultimately the University’s decision how to proceed with the building of the BI system. It is possible that timescales or practicalities mean that some of the recommendations for phase one may need to be postponed, however we would of course strongly recommend that these be reconsidered during later phases to ensure maximum flexibility and maintainability of the system into the future.

UNIVERSITY OF EXETER BI/08/21 BI PROJECT BOARD, 12 June …€¦ ·

Documents