Recommendations on data gathering and - eQuATIC · Recommendations on data gathering and ... Support for the enrollment proces 1999: Start DTO ... System is extensively tested by

Recommendations on

data gathering and

quality of data

Goele Bossaert & Pieter Vos

December 8th 2016, Brussels.

DHO: Database for HigherEducation

Central date depository for higher education in Flanders

Service oriented architecture

Universities and university colleges connect using webservices

Defined goals:1. Support the implementation of (new) legislation (Financing,

Language Policy, Mobility, …)2. Integration with third parties (study grants, child support, …)3. Source for policy information4. Support for the enrollment proces

1999: Start DTO2008: Start DHO2016: Start DHO 2.0

DHO: Database for HigherEducation

Integration with several other parties:

Finance module

Study grants

Child support

Online portal for students

Central diploma depository (LED)

Data for operational use (e.g. DUO)

Use of Datawarehouse Policy Reporting for government and HE institutions (e.g. evaluation flexibilisation HE)

Reporting to interest groups: e.g. parliamentary questions, students,

Generating official education statistics: e.g. yearbook, Dataloep, Education at a glance, Flemish Education in Figures, …

Scientific research

French

5 ECTS

Politics

6 ECTS

Studyprogramme

Statistics

7 ECTS

ProgramsCoursesResultsDiplomas…

Web services DHO

ETL

ADS

ETL

Sounding board with experts of universities and university colleges (student administration, IT, …)

Preliminary consult of and discussion with the field about requirementsAgreement on definitionsOperationalisation of new policiesBilateral contact with software suppliers

Implementation of business rulesEnsure that inserts and updates of objects in database are logicalGive feedback to institutions about transactions (error handling)

Institutions can consult reference data through a ‘taxonomyservice’

Ensuring Data Quality DHO

Testing on several levels

System is extensively tested by both AHOVOKS andinstitutionsInstitutions have access to external testing environment forintegration tests To get access to production, institutions must pass anobligatory test ‘Blauwe wimpel test’Institutions need to give proof they can meet therequirements

Ensuring Data Quality DHO

Ensuring Data Quality DHO

MonitoringWeekly reportsObserving the timeliness, correctness and completeness of the registrations in DHO6 topics linked to deadlines + ad hoc monitoring

If necessary: Escalation

Monitoring Monitoring period Deadlines

Enrolments for programmes01/09 – 31/10: weekly

After 31/10: monthly check31/10

Enrolments for courses01/11 – 01/01: weekly

After 31/12: monthly check31/12

Reorientation 15/03 15/03

ResultsFebruary: first check

01/07 – 15/12: weekly

15/10

15/12 for courses followed abroad

Diplomas 01/07 – 15/12: weekly 15/12

Diploma bonus 01/07 – 15/12: weekly 15/12

E.g. Monitoring timeliness of the registrations of enrolments for programmes

0

10.000

20.000

30.000

40.000

50.000

60.000

70.000

80.000

90.000

100.000

20

21

22

23

24

25

26

Jul -

27

28

29

30

31

Au

g -

31 33

34

35

Sep

-3

63

73

83

9O

kt -

40

41

42

43

No

v -

44

45

46

47

48

Dec

- 4

95

05

15

25

3Ja

n -

1 2 3 4Fe

b -

5 6 7 8M

aa -

9 10

11

12

13

14

2016

Aantal gebeurtenissen Aantal registraties

Mean number of

days between event

and registration

2012 - 2013 3,1

2013 - 2014 3,1

2014 - 2015 3,6

2015 - 2016 3,6

Ensuring Data Quality DHO

E.g. Monitoring completeness of the registrations of enrolments for programmes – comparing 2016-2017 vs. 2015-2016

Goal: 31/10 min. 80% of enrolments of last year

Ensuring Data Quality DHO

<30%

>80%

2016-20172015-2016

total

2015-2016

at 31/10% total % 31/10 2016-2017

2015-2016

total

2015-2016

at 31/10% total % 31/10

HE Institution 1 9.421 10.137 10.240 92,9% 92,0% 10.661 10.135 10.240 105,2% 104,1% 1.240

HE Institution 2 12.055 13.300 12.993 90,6% 92,8% 13.111 13.301 12.993 98,6% 100,9% 1.056

HE Institution 3 2.154 5.724 5.686 37,6% 37,9% 5.123 5.724 5.686 89,5% 90,1% 2.969

HE Institution 4 658 694 0,0% 0,0% 201 658 694 30,5% 29,0% 201

HE Institution 5 1.026 14.481 14.264 7,1% 7,2% 7.465 14.481 14.264 51,6% 52,3% 6.439

HE Institution 6 7.492 8.131 8.093 92,1% 92,6% 7.945 8.125 8.093 97,8% 98,2% 453

HE Institution ….. … … … … … … … … … … …

Total 235.193 281.347 275.804 83,6% 85,3% 258.944 281.347 275.804 92,0% 93,9% 23.751

1/10/2016 Difference

2016-2017

last week

24/09/2016Number of

enrolments

E.g. Monitoring completeness of the registrations of results

Goal: Number of active courses without result = 0

Ensuring Data Quality DHO

Number of

active

courses

Number of

results

% active courses

with registrated

results

Number of

active courses

without result

Number of

active

courses

Number of

results

% active courses

with registrated

results

Number of

active courses

without result

HE Institution 1 98.680 98.680 100,0% 0 98.680 98.680 100,0% 0 0

HE Institution 2 60.850 60.778 99,9% 72 60.850 60.847 100,0% 3 -3

HE Institution 3 361.670 361.629 100,0% 41 361.658 361.645 100,0% 13 -25

HE Institution 4 41.113 40.733 99,1% 380 41.131 41.115 100,0% 16 2

HE Institution 5 164.078 163.885 99,9% 193 164.078 164.065 100,0% 13 -13

HE Institution 6 102.754 102.648 99,9% 106 102.754 102.644 99,9% 110 -110

HE Institution ….. … … … … … … … … …

Total 2.492.730 2.491.379 99,9% 1.351 2.492.558 2.492.003 100,0% 555 624

24/09/2016 1/10/2016

Difference

2016-2017

last week

E.g. Monitoring correctness of registration diploma bonus

Goal: no diplomas left with ‘Verification Not OK’

Ensuring Data Quality DHO

Verification

OK

Verification

Not OK

Difference

motivated

Diploma

bonus n.a.

Verification

OK

Verification

Not OK

Difference

motivated

Diploma

bonus n.a.

HE Institution 1 1.696 1 194 1.696 1 194

HE Institution 2 2.635 78 2.635 78

HE Institution 3 928 1 3 176 928 4 176

HE Institution 4 66 3 78 66 3 78

HE Institution 5 1.298 2 201 1.298 2 201

HE Institution 6 962 45 74 964 39 74

HE Institution ….. … … … … … … … …

Total 42.945 179 10 24.875 43.555 3 75 24.875

Number of

diplomas

12/11/2016 19/11/2016

Validation by HE Institutions

Weekly, automatically renewed reports

Goal: Affirmation correctness and completeness registrationsBaseline for correct finance calculation

Procedure:1. Institution monitors, compares, controls and revises the

registrations using the weekly reports2. If correct and complete: Institution prints out and signs3. Institution informs AHOVOKS4. Institution sends signed documents to AHOVOKS

Deadline: 15/12After the deadline, the data are considered as definite and cannot be changed anymore

Ensuring Data Quality DHO

Mijn Onderwijs - portal for HE institutionsPersonal website with secured login (e-id)Data and information about and for own institution

Monitoring reportsValidation reportsDataloep: interactive, self-service reporting tool

Ensuring Data Quality DHO

Securing Privacy

HE institutions DHO

Reporting statistics

FinanceThird parties

Securing Privacy

HE institutions DHO

Reporting statistics

FinanceThird parties

Privacy authorization

Privacy authorization: all parties must receive authorization from the official privacy committees to exchange personal data

Personal data are all data referring to the participation of a person in higher education

All required data need be motivated on a legal base

All parties must meet security requirements

Securing Privacy

HE institutions DHO

Reporting statistics

FinanceThird parties

Security measures

User application of AHOVOKS is secured by ACM/IDM (Access control and identification management): users need their eID of token

Users need to be authorized by administrator

Connection between institutions and AHOVOKS is secured by certificates (digital handshake and SSL)

Securing Privacy

HE institutions DHO

Reporting statistics

FinanceThird parties

Third parties

Study grants and child support: need to be authorized too to receive or consult data

Student portal: student needs to log in with eID or token to get access to personal data

Securing Privacy

HE institutions DHO

Reporting Statistics

FinanceThird parties

Statistics<5 and >95% rule: No privacy-sensitive data about recognizable students (e.g. study efficiency)

Scientific researchRequest Vlaamse Toezichtcommissie / Privacy CommissionIndividual data but with a anonymized identification number

Mijn Onderwijs – portal for HE institutionsAdmission granted by administrator of HE institutionSecured login via e-id

Impact analysis (e.g. study progress policy)

Evaluation of policy (e.g. financing of the higher education)

Monitoring (e.g. students in a STEM program)

Using data in policy