Kutseõppeasutuste kvaliteedikonverents 2009 20. november 2009 Sokos Viru hotell (Viru väljak 4) Hea praktika avarast maailmast (inglise keeles) Peer-review.

Kutseõppeasutuste kvaliteedikonverents 2009

20. november 2009 Sokos Viru hotell (Viru väljak 4)

Hea praktika avarast maailmast (inglise keeles) 

Peer-review kui üks võimalus kvaliteedialasteks parendusteks

Ms. Leena Koski, Counsellor of Education

Content

• How has Peer Review procedure for VET been developed?

• What is Peer Review? How to use Peer Review?• Who are the Peers?• How can Peer Review be organised?’• Scenarios for transnational European Peer Reviews• An example: Peer Reviews in Finland

Leena Koski

Leonardo projects on European Peer Reviews

Leonardo project ”Peer Review in Initial VET ”1.10.2004−30.9.2007

Leonardo project ”Peer Review Extended”. Implementing the CQAF through Peer Review – “Mutual Review and Assessment of VET Providers and Extension of the European Peer Review Network” 1.1.2007–31.12.2007

Leonardo project ”Peer Review Extended II”1.11.2007–30.10.2009

www.oeibf.at

www.peer-review-education.net

Partners in Peer Review projectsLeonardo project “Peer Review in Initial VET”22 (25) Partner institutions from 11 European countries

1. AT, DE, DK, FI, HU, IT, NL, PT, RO, UK, CH; 2. 13 (15) Providers of initial VET

15 Pilot Peer Reviews completed, Pool of Peers (92), evaluation, adaptation of Manual 2007

Leonardo project ”Peer Review Extended”9 partner institutions from 6 European countries

4 VET Providers (AT, DE, ES, HU), 2 Coordinating Partners (ES, HU), 2 Developing Partners (FI, IT)

4 Pilot Peer Reviews completed

Leonardo project ”Peer Review Extended II”14 partner institutions from 9 countries, 3 „new“ countries (CZ, SI,

TR)

6 Pilot Peer Reviews

What is Peer Review?

• External evaluation • Carried out by "peers", i.e. "persons of equal

standing", colleagues(sometimes called “critical friends”)

• Follows a self-evaluation/self-assessment • Includes a site visit of the peers• Prevalent in evaluation of HE institutions today• Overall aim is that Peer Review procedure supports

VET providers and schools in their effort to improvethe quality of their provision.

Products of Leonardo projects on European Peer Reviews

• European Peer Review Manual for Initial VET

•Toolbox (templates)

• Face-to-face training

• Online training

• Transfer Strategies for partner countries

• European Peer Review Manual for Continuing VET

• Peer Review procedure for guidance and counselling (centres)

Overview characteristics of Peer Review

• The focus of European Peer Reviews is on "Core business" learning and teaching.

• Voluntary: decision by VET Provider, ownership of findings lies with VET Provider

• On national level, a European Peer Review can also be conducted using the relevant national quality framework.

• Respect Quality assurance procedures used by the VET provider

• Qualitative: qualitative evaluation methods, use of existing quantitative data

Who is a peer?A Peer is a person, • who is an equal of or is on equal standing with the

person(s) whose performance is reviewed• who works in a similar environment (and/or in a similar

institution)• who is external (i.e. from a different institution) and

independent (has no personal/institutional "stakes" in the evaluation process)

and• has specific professional expertise and knowledge in the field

(shares values, professional competence and attitudes, language…)

• thus bringing to some extent “inside” knowledge of the object of review into the process and combining it with the external view of somebody coming from a different organisation (“external insider”).

Composition of Peer Teams in European transnational Peer Reviews

Number of Peers (4 Peers)

Occupational background Required competences

2 "Real" Peers*(2 current teachers)

Professionals from other VET providers (teachers, counsellors, managers, quality experts, etc.)

Knowledge of Quality Areas reviewed Experience in teaching & learning Experience in QA and QD procedures

1 "Stakeholder" Peer ** Representative from other stakeholder groups (other educational levels, companies, social partners, etc.)

Knowledge of Quality Areas reviewedExperience in QA and QD procedures

1 Evaluation Expert* Professional evaluator/quality assessor (e.g. from research institute/university, independent auditing/accrediting body, also from VET Provider)

Expertise in evaluation, moderation and communicationKnowledge of VET system

Additional: 1 Gender Mainstreaming Expert*; 1 transnational Peer*** required; ** recommended, *** required for transnational European Peer Reviews

© öibf, Gutknecht-Gmeiner 2008

Phases of a European Peer Review

© öibf, Gutknecht-Gmeiner 2008

European Quality Areas in IVET

QA 1: CurriculaQA 2: Learning and teachingQA 3: AssessmentQA 4: Learning results and outcomesQA 5: Social environment and accessibilityQA 6: Management and administrationQA 7: Institutional ethos and strategic planning

QA 8: Infrastructure and financial resources

QA 9: Staff allocation, recruitment and development

QA 10: Working conditions of staffQA 11: External relations and internationalisationQA 12: Social participation & interactionsQA 13: Gender mainstreamingQA 14: Quality management and evaluation

At least one of the Core Quality Areas (QA 1-4; key processes).

Which quality areas should VET providers choice for Peer Reviews?

• Which quality areas might have problems?

• Which quality areas are most critical for the success?

• In which quality areas are there developments?

• Which quality areas show a good performance?

• Which quality area have not been evalauted

shortly?

• It is many times appropriate to start with some of core quality areas?

• No more than 2 - 3 quality areas at same time.

School BSize of Peer team4-6

School ASize of Peer team

4-6 visit

School CSize of Peer team4-6

visit

visit

School ASize of Peer team4-6

Scool BSize of Peer team4-6visit

visit

How can you organise Peer Reviews?How can you organise Peer Reviews?

Verkosto (”Tridem”)

School ASize of theTeam 4-6

Expert(Peer leader)

Peer Review Network

Peer A

Peer C

Peer B

Pool of Peer Review experts ¨

EQARF: Framework for self-assessment

Methodology

Indicator 1, 2 Define goals and objectives

Indicator

3, 7, 8 Coherent with goals

Indicator 4, 5, 6 Collection and processing data; discuss results

Indicator 9, 10Review

Purpose And plan

Implementation

Assessment and evaluation

Feedback andprocedures for

Change

© öibf, Gutknecht-Gmeiner 2009

Contribution of Peer Review to the European Quality Assurance Reference Framework

Peer Review process

(Analysis of Self-Report; Peer Visit)

Follow-up(Consequences of review report and

follow-up activities)

Peer Review Report (strengths & improvement

areas)

Self-assessment of VET providers(Self-Report)

Peer ReviewMethodology

Quality Criteria implemented in the European Peer Review Dissemination and practical use of quality criteria enhanced through Peer ReviewPeer Review as anew methodologyof external evaluation (Further) development and use of descriptive indicators

Benefits of Peer Reviews

• Provide a systematic approach to external evaluation following the quality assurance and improvement cycle

• Is a new instrument for external evaluation and assessment of VET providers

• Combines internal and external evaluation• Focus on core business of the school• Is complementary to other quality assurance models

like EFQM• Ascertain the quality of VET provision• Present the strenghts and showcase good practices• Detect blind spots and weaknesses• Promote cooperation between teachers and networking

Leena Koski

Benefits of Peer Reviews

• Is an attractive and motivating methodology which promotes bottom-up cooperation between VET providers and nurtures the development of a professional quality-improvement culture within the VET sectorEuropean Peer Review:

• Promotes exchange and networking between VET providers within and across Member States

• Can led to further cooperation between VET providers including the establishment of transnational mobility schemes.

Leena Koski

• Scenario 1: Grass-roots model• relies on grass-roots activities by VET providers with no European

coordination and support

• Scenario 2: Decentralised coordination model• encompass a coordinating body on the European level which in the

“decentralised coordination model” would primarily provide technical and administrative support

• Scenario 3: Transnational cooperative model• active and stringent cooperation between the different actors is

foreseen with a coordination body that also provides expert support for both the national lead bodies and the VET providers involved.

• operative tasks and responsibilities by and large remain with the national bodies

(

Leena Koski

Scenarios for transnational European Peer Reviews

• The Thematic Group unanimously and strongly recommends the implementation of scenario 3

• ENQA-VET board has decided to include all three scenarios and the SWOT analysis of them.

• Forecast for transnational European Peer Reviews 2010-2013

(Source:Gutknecht-Gmeiner 2009 (figures as of March 20, 2009)

Leena Koski

Scenarios for transnational European Peer Reviews

Year 2010 2011 2012 2013 Total

No. of Peer Reviews 43 56 70 78 247

• National adaptations and implementation• EQARF - Devise a national approach for

implementing EQARF • QANRPs will make more systematic

cooperation in the future to support the use of Peer Reviews at national level too.

Leena Koski

Scenarios for transnational European Peer Reviews

An example: Peer Reviews

in Finland year 2009

Leonardo projects and other activities year 2009Leonardo project: Peer Review Extended II, coordinator Öibf, AustriaProject number- LLP-LdV/TOI/2007/AT/0011

1.11.2007 – 31.10.2009Results:• Face-to-face Peer Training, CVET too• Peer Review procedure and criteria for CVET for Finland• Transfer Strategy for Finland.  Peer Review as a part of ENQA-VET work programme/Thematic group

National development activities: Peer Reviews as a method of QA. Result agreement between Ministry of Education and FNBE includes year 2009 the national development and support activities to promote the use of Peer Reviews.

Quality Management Recommendation for VET is directed to IVET and CVET

Leena Koski

Quality Management Recommendation for VET

VET providers:• collect and analyse evaluation results and other feedback information;• make use of results obtained in other evaluations, such as the results of

national evaluations;• have an external evaluation of their operations carried out at least once

every three years;• develop their operations by means of internal audits;• co-operate with other education and training providers by

carrying out benchmarking and peer reviews, for example.

Leena Koski

•.

VET providers make diverse use of various evaluation methods and data in order to develop their operations.

PEER REVIEW IN VET IN FINLAND YEAR 2009

PEER REVIEW IN VET IN FINLAND YEAR 2009

At Europeanlevel

ENQA VET

• The structure for European Peer Revies 2008-2009

Leonardo project

- Peer Review Extended II- Peer Review for Continuing VET- Face to face training 1.11.2007 - 1.11.2009

The Manual for EuropeanPeer Reviews

• Student Assessment

Application of the criteriaand indicators for PeerReviews at the nationallevel- quality areas and criteria and indicators based on Finnish Education System and Quality recommendation for VET - 14/16 quality areas -

Activities- responsible partner for piloting face to face training- pilot face to face training- to take part in application of the Peer Review Procedure and the Manual suitable for VET

Peer Review as a tool in skilldemonstrations for VET

- ESR project- to pilot Peer Review procedure and develop criteria for skill demonstrations

In Finland

guide for national approach

Qualityarea

2007 Leena Koski/rtLeena Koski

Preliminary results of a Survey on QA to the Finnish VET Providers

• FNBE has made a survey on QA year 2004 and spring 2009.

• FNBE has sent the on-line questionnaire to 158 VET providers.

• 110 answered the questionnaire (69 %).• 27 respondents (27 %) has made systematic

quality work over 10 years, 28 respondents (27 %) 6 -10 years, 19 respondents (19 %) 3-5 years and 28 respondents (27 %) less than 3 years.

Quality models/methods which have been used and/or are in use?

Leena Koski

An example: National Peer Reviews in CVET in Finland

• Competence-based qualifications are independent of the method of acquiring the required skills prior studies or formal training.

Three levels of qualifications: * Vocational qualification * Further vocational qualification and * Specialist vocational qualification.

Individualisation is embedded in all the phases of getting aqualification and vocational skills:1. Individualisation of the applaying phase2. Individualisation during the making of a competence test3. Individualisation in completing the vocational skills

Leena Koski

National Peer Reviews in CVET in Finland

• Preparation with VET providers national criteria for CVET is in final stage in Finland

• The procedure for Transnational European PeerReviews can be used in Finland for both IVET and CVET (only some slight modification needed).

• Challenge: To find common criteria for transnational EuropeanPeer Reviews for CVET

Quality areas:1. Organization of Competence-based Qualifications2. Individualisation at the application stage 3. Acquisition of the required vocational competences4. Individualisation during competence-based qualification

Adaption to CVET• Terminology/definitions and concepts must be

adapted.• Quality Areas have to be modified.• “Real Peers” should also have as “required competence”,

experience in learning and teaching adults, as adults. • CVET is organised different than lVET, the processes of

accreditation of prior learning are more important in the continuing VET.

National Peer Reviews in CVET in Finland5. Assessment of candidates´ competences6. Effectiveness of Competence-based Qualifications7. Diversity and accessibility of the operating environment8. Infrastructure and financial resources9. Leadership, management and administration10. Strategic planning11. Human resources planning, management and development12. Staff well-being at work13. Co-operation with stakeholders and internationalization14. Social interaction and participation15. Gender mainstreaming16. Quality management and evaluation

Leena Koski

Leonardo project ”Peer Review Impact”

Leonardo project: coordinator FNBE1.10.2009 – 30.09.2011Results:• Report on Impact of Peer Reviews• Peer Review Impact Guidelines• Self-assessment guide for Peer Reviews used on VET

provider or school level• Seminar/Workshop for VET provides how to use

Impact Guidelines

More information: [email protected]

Leena Koski