Quality Procedures in European Higher Education Findings of the 2008 Survey Achim Hopbach Managing Director German Accreditation Council.

Quality Procedures in European Higher Education

Findings of the 2008 Survey

Achim Hopbach

Managing Director German Accreditation Council

Background

“QPP 1” in 2003: Major progress towards convergence despite

varying national priorities

Major developments since then: Quality Assurance in the European Higher

Education Area (ESG) in 2005 Common understanding of quality assurance

Framework for Qualifications of the European Higher Education Area (FQEHEA) in 2005

Common understanding of learning outcomes

Goals of the survey

to update the 2003 survey to assess the agencies’ opinion on their

compliance with Part 3 of the ESG to survey agencies’ external review

plans to survey agencies’ attitudes towards

the European Quality Assurance Register for Higher Education (EQAR)

Method Development of online questionnaire 2nd half 2007 Responses by 31st January 2008 1st analyses: Division of 71 questions into seven

sections (by May 08) Final analysis, draft report (by August 08)Organisation Project run by HETAC and QAA Steering group Project group (9 members responsible for different

chapters)

Responses

51 responses from 30 EHEA countries (2002: 36 from 23)

29 full member agencies (out of 34)

7 candidate member agencies (out of 11)

11 Associates 5 Affiliates

Findings: Agencies

Scope, remit and domain of agencies Agencies predominantly cover both

universities and other types of HEI (28 in 24 countries), and all fields of learning

35: (whole) country 7: part of a country (6: Spain, Belgium) 4: international agencies by definition

42: All HEI subject to compulsory external QA

Scope, remit and domain of agencies Agencies predominantly cover both universities

and other types of HEI (28 in 24 countries), and all fields of learning

35: (whole) country 7: part of a country (6: Spain, Belgium) 4: international agencies by definition

42: All HEI subject to compulsory external QAOne QA framework in national HEI systems

Status of agencies

44 out of 49 agencies based on law or recognised by public authority

only one responsible agency per country (except for federal systems of Spain and Germany)

QA as main task which is carried out regularly (42 out of 46)

Status of agencies

44 out of 49 agencies based on law or recognised by public authority

only one responsible agency per country (except for federal systems of Spain and Germany)

QA as main task which is carried out regularly (42 out of 46)

Legally based agency with sole responsi-bility as widespread European pattern

Function of agenciesFunctions Response

s %(very) important

Quality assurance 100 95.3

Quality enhancement 97.6 93.0

External QA of programmes 95.5 84.1

Collecting/Diseminating information on quality

95.5 70.5

External QA of institutions 90.9 77.3

Recognition/licensing of institutions 66.7 57.1

Development/Maintenance of discipline standards

66.7 41.9

Development/Maintenance of NQF 62.8 41.9

Deciding on the funding of iunstitutions 50.0 26.2

Recognition of national diplomas 47.6 42.9

Funding

Funding

Government funding is the main source of funding for agencies

Source Average %

No. of agencies

Government 70.63 41

HEIs (fixed subscriptions only) 37.35 15

Fees (for specific activities) 32.85 28

Grants (other than government) 2.25 12

Other 11.72 18

Findings: Quality Procedures

Activities

Most frequent types of QA:Evaluation, accreditation, audit

65% programme approach40% institutional approach

Either regular or never (very seldom occasionally)

90% are not confined to only one approach

Reference frame (1)

What defines the reference frame for the external quality procdure?

% of agencies

Legal regulation 74.5

Stated goals of the institutions 57.4

Guidelines of good practice 57.4

National Qualifications Frameworks 55.3

European (Bologna) Qualifications Framework 44.7

Standards defined by professional organisations

34.0

Subject benchmarks 27.7

Reference frame (2)

Notion of criteria and standards as a common feature in European quality assurance (2003 survey: emerging feature)

What specific criteria and standards are used for your agency‘s external quality procedure?

% of agencies

Agency‘s own published criteria and standards 87.2

European Standards and Guidelines 83.0

National criteria and standards 66.0

OECD/UNESCO Guidelines on Quality Provision in Cross-boarder HE

14.9

Areas addressed in the quality proceduresThe five highest impacting items programme level procedures: qualifications of staff; curriculum/syllabus; facilities and resources; internal QA procedures; mission/goals.

institutional level procedures: internal QA procedures; management, organisation; mission/goals; qualifications of staff; facilities and resources.

Areas addressed in the quality proceduresThe five highest impacting items programme level procedures: qualifications of staff; curriculum/syllabus; facilities and resources; internal QA procedures; mission/goals.

institutional level procedures: internal QA procedures; management, organisation; mission/goals; qualifications of staff; facilities and resources.

Isn’t one item missing?

Areas addressed in the quality proceduresThe five highest impacting items programme level procedures: qualifications of staff; curriculum/syllabus; facilities and resources; internal QA procedures; mission/goals.

institutional level procedures: internal QA procedures; management, organisation; mission/goals; qualifications of staff; facilities and resources.

Addressing learning outcomes in (external) QA is only at the beginning.

Consequences of QA

External QA leads to formal consequences (approval, funding) in more than 75% of cases.e.g. approval in 90% of accreditationsapprox. 80% of evaluations and audits

All the common procedures are applied for these purposes

Consequences of QA

External QA leads to formal consequences (approval, funding) in more than 75% of cases.e.g. approval in 90% of accreditationsapprox. 80% of evaluations and audits

All the common procedures are applied for these purposesConsequences of external QA are taken independently of QA-type

Stakeholder involvement:Responsibilities66.7% External stakeholders may influence

neither the conclusions nor the recommendations in reports

78.7% External stakeholders have no share in responsibility for the operations

Others: Influence mainly in way of participation in agency’s body, panels, etc.

Stakeholder involvement:Responsibilities66.7% External stakeholders may influence neither

the conclusions nor the recommendations in reports

78.7% External stakeholders have no share in responsibility for the operations

Others: Influence mainly in way of participation in agency’s body, panels, etc.

Stakeholder involvement is a common feature; it does not necessarily compromise independence of the agencies

Stakeholder involvement:Specification of processes and criteria• In more than half of all respondent agencies

“government”, “quality assurance agencies” and “student representatives” are involved; “industry and labour market” (36%) and “professional organisations" (32%). In 4 cases also government.

• The final decision on the specification of processes and criteria is likely to be made by the relevant agency (79%). In two cases it is the government alone.

Panels: responsibility

WHO PERFORMS THE FOLLOWING FUNCTIONS IN THE EXTERNAL QUALITY PROCEDURES

Panel

No. of respondents

Agency

No. of respondents

Choice of basic methodology applied 12 44Preparation of the guidelines for the self-evaluation

4 42

Preparation of the external quality procedure concept

11 42

Contact with the institution 11 41Planning of the site visit 23 36Preparation of the guidelines for the site-visit

16 38

Writing of the report 41 17

Panels: membershipWho are members of the external expert panel?

% of agencies

(2003)

National experts representing area of focus

76.1

International experts 73.9

Professional practitioners 63.0

Students 56.5 22.0

National experts representing institutions

32.6

Employers 45.7 36.0

Staff members of the agency 26.2 40.0

Other 26.1

Professional organisations 19.6

Graduates 15.2

Panels: training

Duration: 1.5 days on average

Characteristics and circumstances of briefing/training provided to panel members

% of agencies

Training/briefing is compulsory for all panel members

75.0

Training/briefing is compulsory for all panel chairs

50.0

Training/briefing provided is matched to the experience of each panel members

42.5

Training outcomes are assessed by the agency 27.5

Training is requiered before a person may be nominated to a panel

25.0

Training is provided using distance learning 22.5

Other 17.5

Findings: Future developments

Future developments

Expected developments in the QA sector by January 2010

Response%

Revision of external quality procedures 57.4

Establishment of National Qualifications Framework

48.9

Introduction of different methodologies for external quality procedures

46.8

Commencement of new higher education legislation

31.9

Reorgamnisation or merging of the agency 31.9

36 out of 48 agencies changed their approach recently or are about to do so; 27: significant changes (8 referring to ESG, 3 switching to another approach, 16 adding another approach)

Future developments

Expected developments in the QA sector by January 2010

Response%

Revision of external quality procedures 57.4

Establishment of National Qualifications Framework

48.9

Introduction of different methodologies for external quality procedures

46.8

Commencement of new higher education legislation

31.9

Reorgamnisation or merging of the agency 31.9

36 out of 48 agencies changed their approach recently or are about to do so; 27: significant changes (8 referring to ESG, 3 switching to another approach, 16 adding another approach)

QA in flux

The register

78% planning to apply66% think it’s a useful tool

Expected benefits Response%

International recognition

85

Credibility 82

International recognition

59

Accountability 56

National recognition 26

Concluding remarks

National QA systems and role of agencies legally based and well developed

Methodology well developed (but 4-stage-model not applied everywhere, due to national context)

ESG as major reference frameConvergence continues but national context matters! Still: QA in flux (1)

Developing external QA in a changing world:Student centred learning, LO, stakeholder involvement, accountability to society

Demands on QA get diverse Purpose of QA gets diverse

QA in flux (2)

Final question:Is there a gap between direction/stage of development of QA methods and function/ purpose of QA?

Is the purpose of external QA compromised by different usage of results?What does it mean for the type and design of methods applied?

Thank you for your attention!

Achim Hopbachhopbach@akkreditierungsrat.de

www.akkreditierungsrat.de