Accreditation Authority for Universities of Applied Sciences ENQA seminar: Current trends in the European QA and the situation in Southern Europe How to.

Accreditation Authority for Universities of Applied Sciences

ENQA seminar: Current trends in the European QA

and the situation in Southern Europe

How to prepare for an external review?

Kurt Sohm Managing director FH Council (FHR)


Outline

1. What is the agency under review?• History, Framework, FH Council, EQA• Educational mandate, accreditation decision

2. How to prepare for the review?• Context• Key features• Terms of reference and protocol for the review• Composition expert panel• Management SE-process• SE-report• Schedule of the review• Lessons learned

5 slides

16 slides

= 21 slides

~ 30 min


FH sector at a glance: short history

Short history and is still developing• the FH Studies Act became effective on 1 October 1993

- the whole sector has been completely new developed since 1993

• building up of the sector: not by transforming existing educational institutions but by accrediting new programmes

Actual state of affairs• the first 10 programmes started 1994/95• meanwhile: 20 institutions offering 243 programmes, 30.000

students, 9.000 annual intake, about 23.000 graduates• at present 94% of the programmes are offered in the BaMa-

System - the BaMa-reorganisation is nearly completed (amendment of FH

Studies Act 2002)


FH sector at a glance: Framework conditions

FH Studies Act (lean law with 21 sections) is based on principles of „New Public Management“• Deregulation at the state level and regulation by private sector

under state supervision- the state no longer centrally controls and regulates the Higher

Education sector as it previously did - decentralisation of decision-making process in order to foster

independence, responsibility and flexibility of the institutions- FH institutions were given greater autonomy to organize themselves

• Providers are – with one exception – privately organised- legal person under private law, e.g. companies with limited liability,

associations or public foundations (e.g. contracts with lecturers are also concluded under private law)

• Public funding (concept of study place management)• Accreditation by a public authority (FH Council)


FH sector at a glance: FH Council

Public authority responsible for EQA• independent decision-making body > guaranteed by law • members are not bound by any ministerial directives

Members appointed by Federal Minister of Science and Research• four members being appointed on recommendation of the

Advisory Board for Economic and Social Affairs • for three-year terms, singular re-appointment for a second,

consecutive term is possible FH Council comprises 16 members

• with academic and professional qualifications- current president: Leopold März, former rector Vienna University of

Natural Resources and Applied Life Sciences


FH sector at a glance: External QA

Close link between initial accreditation, evaluation and re-accreditation• accreditation refers to programs > valid for max. of 5 years

- formal and independent decision, indicating that a program offered meets certain standards

• decision on initial accreditation is carried out by the members of the FH Council themselves

- no expertise in the FH Council written expert opinions are asked for

• decision on re-accreditation is based on a previously conducted evaluation

- each re-accreditation requires a new application and the submission of an evaluation report

• Evaluation: internal, external, follow-up, publication- institutional and programme-related evaluation

• Evaluation doesn’t state an own methodological concept but it serves to fulfil the task of accreditation


FH sector at a glance: Educational mandate

Practice-oriented education at a higher education level• focused on the employability of graduates• curricula are to be designed in such a way that

- the graduates will stand a reasonable chance of finding a job that matches their qualifications on higher education level

- interrelation between vocational fields of activity, qualification profile and curriculum plays a crucial role

Accreditation decision• programmes are reviewed against the fulfilment of the

educational mandate- Does the submitted concept fulfil its educational mandate in a reliable

and transparent way?- Has the [field-specific] implementation of the educational mandate

been demonstrated in a logical, conclusive and valid way?


Review FH Council: context

FH Council (public authority) is under the supervision of the Federal Ministry of Science and Research• supervision is limited to the observance of laws and regulations• founding member of ENQA (March 2000, Brussels)

Bergen ministerial meeting – May 2005• Adoption of the European Standards and Guidelines for Quality

Assurance in EHEA (ESG)- ESG are identical with the ENQA criteria for Full membership- condition of Full membership: all Full members have to undergo an

external review at least once every five years- if not: an agency will cease to be a member of ENQA

Member of the European Consortium for Accreditation (ECA)• Code of Good Practice (agreed upon 2004)• agreement of all ECA members to be reviewed by end of 2007



Organisation of review• five-yearly reviews are normally initiated and coordinated by

national authorities- principle of subsidiarity which underpins the ESG and as part of the

national quality assurance arrangements

• limited resources of ENQA to coordinate reviews- exception: no suitable or willing national body- agencies wishing ot engage ENQA must be able to justify why a

national review is impossible- ENQA reserves the right to decline such a request

• the management of the review must be completely independent of the agency under review

• all parts of the process must be transparent and easily open to examination by the ENQA board



Remit of the review• Two types of nationally coordinated reviews

- „sole purpose review“ (type A): only to fulfil the periodic review requirement of ENQA membership

- „multiple purpose review“ (type B): review has a number of purposes, one of which is to fulfil the periodic review requirement of ENQA membership

• Clarification well in advance before the review starts• Determination in the Terms of reference and protocol for the

review (ToR)


Review FH Council: key features

Review was coordinated by the Federal Ministry of Science and Research (national authority)• contact details were communicated to ENQA

Considered national and international requirements (type B review) Panel predominately consisted of international experts Close cooperation and consultation with ENQA

• review process should meet the requirements of ENQA board• ENQA was kept informed of progress throughout the review

- notification to ENQA at an early stage that the review will be conducted in autumn 2007

- acceptance of ToR by ENQA- to preserve the integrity of the review the selection process of panel members

was carried out by the Ministry in consultation with ENQA

Site visit: interviews with HE-Institutions, Association of UAS, Students, Ministry, Council members, Staff members, Members expert panels, business/industry (about 60 persons)



Purpose and aim, questions to be addressed• In which way and to what extent does the FH Council fulfil the

tasks stipulated by the Fachhochschule Studies Act in the area of external quality assurance?

• In which way and to what extent does the FH Council thereby fulfil the criteria for the ENQA membership and thus the European Standards and Guidelines?

• In which way and to what extent does the FH Council comply with the ECA Code of Good Practice?

Involvement of stakeholders• the final SE-report was submitted to the relevant stakeholders by

the Ministry (institutions and students) for statement• the statements were communicated to the review panel

- all relevant documents were made accessible on a restricted part of our website



Basic attitude towards the self-evaluation (SE)• organised as a project with a clearly defined schedule• adhered to the principles of self-criticism, objectiveness and

openness• structured within the seven criteria of ENQA membership

- description of the actual situation- identification of strengths and weaknesses- proposals for improvement

• SE-report is to present in an understandable manner in which way the FH Council fulfils

- its tasks of external quality assurance as stipulated by the Fachhochschule Studies Act

- the criteria for the ENQA membership and thus the European Standards and Guidelines as well as the requirements of the ECA Code of Good Practice


Review FH Council: terms of reference (ToR)

Drafted well before the process started in consultation between Ministry, FH Council and Conference of FH institutions

Table of contents• Summary • Background and Context

- Legal Basis of the FH Council- National Context - International Context

• Purpose and Aim of the Review • Steps of the the Review Procedure

- Nomination and appointment members of the expert panel - Self-evaluation of the FH Council - External Review by an expert panel- Drawing up the Evaluation Report

• Follow-up Procedure and Publication of the Report • Schedule of the Review


Review FH Council: composition expert panel

Compostion of expert panel• one national expert well familiar with the Austrian higher

education system- nominated and appointed directly by the ministry

• two international experts from organisations that are responsible for external quality assurance

- nominated by ENQA and appointed by the ministry

• one international expert from a higher-education institution- nominated by ENQA in consultation with EURASHE

• one international expert with students’ experience- nominated by ENQA in consultation with ESU

• one assistant (not to be in a state of dependence to the Council)- nominated by the chair of the panel which has to come from an ENQA

member agency

Ministry prepares the panel in an appropriate manner


Review FH Council: members expert panel

Members• Jon Haakstad, Chair, Norwegian National Quality Assurance

Agency for Higher Education (NOKUT)• Mark Fredericks, Nederlands-Vlaamse Accreditatie Organisation

(NVAO)• Manfred Prisching, national expert, University of Graz, Institute of

Sociology• Bert Hoogewijs, Rector, University College Ghent• Vanja Ivošević, student member, ESU, Croatia• Agnes Leinweber, secretary, German Accreditation Council

Language• Review was made in German (and English)• international experts with good oral and written command of

German language


Review FH Council: management SE-process

responsibility: managing director (K. Sohm)• drafting the descriptive parts of the SE-report• ensuring a broad involvement of staff members and Council

members especially in terms of discussing the strengths/weaknesses analysis and areas for improvement

• observance of time schedule• chairing the meetings with staff and Council members• revising the draft report after the meetings• preparing the workshops and meetings with the self-evaluation

working group of the FH Council• consultation with the ministry• final wording of descriptive parts, strenths/weaknesses, areas for

improvement


Review FH Council: SE-report

Table of contentsI. Preliminary remarks

- Aims + principles, composition expert panel, description SE-process

II. Introduction- Demonstration HE sector in Austria and national EQA system;

general description FH sector; important key features FH sector

III. The FH Council and the 7 ENQA criteria for membership- Activities; Official status; Resources; Mission statement;

Independence; External QA criteria and processes; Accountability procedures- Integration contents of ESG, ENQA criteria, CGP ECA under the headings

of the 7 ENQA criteria for membership- Description tasks FH Council according to FH Studies Act, analysis of

strengths and weaknesses, areas for improvement

IV. Appendix: index of abbreviations, annexes, additional documents at the site-visit, statements of stakeholders- report: 80 pages, appendix: 200 pages


Review FH Council: SE-report (example independence)

ENQA Regulations 4.74.7 A Full member should be independent to the extent both that it has autonomous responsibility for its operations and that the conclusions and recommendations made in its reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders. The member will need to demonstrate its independence through measures, such as:

-its operational independence from higher education institutions and governments is guaranteed in official documentation (e.g. instruments of governance or legislative acts); -the definition and operation of its procedures and methods, the nomination and appointment of external experts and the determination of the outcomes of its quality assurance processes are undertaken autonomously and independently from governments, higher education institutions, and organs of political influence; -while relevant stakeholders in higher education, particularly students/learners, are consulted in the course of quality assurance processes, the final outcomes of the quality assurance processes remain the responsibility of the member.

European Standards and Guidelines 3.63.6 Independence: Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders.ECA, Code of Good Practice 33. Must be sufficiently independent from government, from higher education institutions as well as from business, industry and professional associations.


Review FH Council: Panel draft report

Site visit: 16 – 19 September 2007 Expert panel draft report

• submitted to the Ministry and forwarded to the FH Council „only“ for factual verification on 10 October 2007

• deadline for statement: 24 October 2007 Statement FH Council

• 10 pages• not only factual verification • but also clarification on misunderstandings related to important

contents of the draft report• request for further justification of essential findings

Final report: expected on mid November 2007


Early March 2006Decision after consultation between the Ministry and the FH Council to conduct the review in autumn 2007

Mid March 2006 Notification to ENQA and request for nomination of panel members

End J une 2006Letter ENQA to the Ministry to confirm notification and request of the Ministry (outcome Board meeting)Drafting terms of reference and protocol for the review (ToR)Consultation ToR between Ministry, FH Council and Conference of FH InstitutionsConstitution of a working group of FH Council membersTranslation of ToR into EnglishForwarding ToR to ENQAConfirmation ToR by ENQATraining for staff members on project managementDrafting descriptive parts of the SE reportContinuation drafting descriptive parts of the SE reportNomination and appointment of panel membersInformation composition of panel members to the FH Council for statement10 meetings with the academic staff members: discussing descriptive parts, strengths, weaknesses, areas for improvement (each with 2 to 3 hours)2 consultation meetings with the MinistryOne-day workshop with the SE-group of the FH Council on external QA and whole QA arrangementPresentation and discussion of the results within the Plenary Council meetingWritten information by the Ministry to the stakeholders about the reviewSubmission of the draft SE report to the FH Council working groupHalf-day meeting with the FH Council working group

End J une 2007 Adoption of the SE report by the FH Council

Schedule of the ReviewPhase Date

J une - September 2006

October - December 2006

Pre

para

tio

n p

hase

December 2006

J annuary - May 2007

SE p

hase


Ende J une 2007 Submission of the final SE report to the MinistryForwarding the SE report to the stakeholders for statementStatements of the stakeholders

Mid - End August 2007 Submission of the SE report and the stakeholder statements to the expert panelSite visit by the expert panelInterviews with institutions, students, Conference of the Institutions, ministry, panel members of QA procedures, FH Council, staff members

10 October 2007 Submission of draft review report to FH Council for statement24 October 2007 Submission statement FH Council to the Ministry

Mid November 2007 Submission final report to FH Council

Discussion of report and implementation plan FH Council and Ministry

Publication of report and implementation plan

Submission of report to ENQA

Schedule of the Review (cont.)

Next

ste

ps

November - December 2007

16 - 19 September 2007

Phase Date

J une - August 2007

Sit

e v

isit


Lessons learned

division of labour between project management SE-process and responsibility as regards content

national expert who is familiar with the national context self evaluation was very fruitful and valuable

• discussion of strengths, weaknesses and areas for improvement• strong incentive to systematically reflect on the situation• we placed a lot of weight in the production of the report

it´s all about self-critical reflexivity broad involvement of staff and Council members („no one-

person-show“) gain of credibility by the stakeholders international experts: provide very valuable insights for the

review and help to establish its credibility


Thank you very much for your attentionhttp://www.fhr.ac.at/

Document„Guidelines for national reviews of ENQA member agencies”

http://www.enqa.eu/

Accreditation Authority for Universities of Applied Sciences ENQA seminar: Current trends in the European QA and the situation in Southern Europe How to.

Documents

accreditation authority

initial accreditation

possible fh council

task of accreditation

institutions fh institutions

fh studies act

director fh council

amendment of fh studies