Kutseõppeasutuste kvaliteedikonverents 2009 20. november 2009 Sokos Viru hotell (Viru väljak 4) Hea praktika avarast maailmast (inglise keeles) Peer-review.
Post on 29-Dec-2015
214 Views
Preview:
Transcript
Kutseõppeasutuste kvaliteedikonverents 2009
20. november 2009 Sokos Viru hotell (Viru väljak 4)
Hea praktika avarast maailmast (inglise keeles)
Peer-review kui üks võimalus kvaliteedialasteks parendusteks
Ms. Leena Koski, Counsellor of Education
Content
• How has Peer Review procedure for VET been developed?
• What is Peer Review? How to use Peer Review?• Who are the Peers?• How can Peer Review be organised?’• Scenarios for transnational European Peer Reviews• An example: Peer Reviews in Finland
Leena Koski
Leonardo projects on European Peer Reviews
Leonardo project ”Peer Review in Initial VET ”1.10.2004−30.9.2007
Leonardo project ”Peer Review Extended”. Implementing the CQAF through Peer Review – “Mutual Review and Assessment of VET Providers and Extension of the European Peer Review Network” 1.1.2007–31.12.2007
Leonardo project ”Peer Review Extended II”1.11.2007–30.10.2009
www.oeibf.at
www.peer-review-education.net
Partners in Peer Review projectsLeonardo project “Peer Review in Initial VET”22 (25) Partner institutions from 11 European countries
1. AT, DE, DK, FI, HU, IT, NL, PT, RO, UK, CH; 2. 13 (15) Providers of initial VET
15 Pilot Peer Reviews completed, Pool of Peers (92), evaluation, adaptation of Manual 2007
Leonardo project ”Peer Review Extended”9 partner institutions from 6 European countries
4 VET Providers (AT, DE, ES, HU), 2 Coordinating Partners (ES, HU), 2 Developing Partners (FI, IT)
4 Pilot Peer Reviews completed
Leonardo project ”Peer Review Extended II”14 partner institutions from 9 countries, 3 „new“ countries (CZ, SI,
TR)
6 Pilot Peer Reviews
What is Peer Review?
• External evaluation • Carried out by "peers", i.e. "persons of equal
standing", colleagues(sometimes called “critical friends”)
• Follows a self-evaluation/self-assessment • Includes a site visit of the peers• Prevalent in evaluation of HE institutions today• Overall aim is that Peer Review procedure supports
VET providers and schools in their effort to improvethe quality of their provision.
Products of Leonardo projects on European Peer Reviews
• European Peer Review Manual for Initial VET
•Toolbox (templates)
• Face-to-face training
• Online training
• Transfer Strategies for partner countries
• European Peer Review Manual for Continuing VET
• Peer Review procedure for guidance and counselling (centres)
Overview characteristics of Peer Review
• The focus of European Peer Reviews is on "Core business" learning and teaching.
• Voluntary: decision by VET Provider, ownership of findings lies with VET Provider
• On national level, a European Peer Review can also be conducted using the relevant national quality framework.
• Respect Quality assurance procedures used by the VET provider
• Qualitative: qualitative evaluation methods, use of existing quantitative data
Who is a peer?A Peer is a person, • who is an equal of or is on equal standing with the
person(s) whose performance is reviewed• who works in a similar environment (and/or in a similar
institution)• who is external (i.e. from a different institution) and
independent (has no personal/institutional "stakes" in the evaluation process)
and• has specific professional expertise and knowledge in the field
(shares values, professional competence and attitudes, language…)
• thus bringing to some extent “inside” knowledge of the object of review into the process and combining it with the external view of somebody coming from a different organisation (“external insider”).
Composition of Peer Teams in European transnational Peer Reviews
Number of Peers (4 Peers)
Occupational background Required competences
2 "Real" Peers*(2 current teachers)
Professionals from other VET providers (teachers, counsellors, managers, quality experts, etc.)
Knowledge of Quality Areas reviewed Experience in teaching & learning Experience in QA and QD procedures
1 "Stakeholder" Peer ** Representative from other stakeholder groups (other educational levels, companies, social partners, etc.)
Knowledge of Quality Areas reviewedExperience in QA and QD procedures
1 Evaluation Expert* Professional evaluator/quality assessor (e.g. from research institute/university, independent auditing/accrediting body, also from VET Provider)
Expertise in evaluation, moderation and communicationKnowledge of VET system
Additional: 1 Gender Mainstreaming Expert*; 1 transnational Peer*** required; ** recommended, *** required for transnational European Peer Reviews
© öibf, Gutknecht-Gmeiner 2008
© öibf, Gutknecht-Gmeiner 2008
European Quality Areas in IVET
QA 1: CurriculaQA 2: Learning and teachingQA 3: AssessmentQA 4: Learning results and outcomesQA 5: Social environment and accessibilityQA 6: Management and administrationQA 7: Institutional ethos and strategic planning
QA 8: Infrastructure and financial resources
QA 9: Staff allocation, recruitment and development
QA 10: Working conditions of staffQA 11: External relations and internationalisationQA 12: Social participation & interactionsQA 13: Gender mainstreamingQA 14: Quality management and evaluation
At least one of the Core Quality Areas (QA 1-4; key processes).
Which quality areas should VET providers choice for Peer Reviews?
• Which quality areas might have problems?
• Which quality areas are most critical for the success?
• In which quality areas are there developments?
• Which quality areas show a good performance?
• Which quality area have not been evalauted
shortly?
• It is many times appropriate to start with some of core quality areas?
• No more than 2 - 3 quality areas at same time.
School BSize of Peer team4-6
School ASize of Peer team
4-6 visit
School CSize of Peer team4-6
visit
visit
School ASize of Peer team4-6
Scool BSize of Peer team4-6visit
visit
How can you organise Peer Reviews?How can you organise Peer Reviews?
Verkosto (”Tridem”)
School ASize of theTeam 4-6
Expert(Peer leader)
Peer Review Network
Peer A
Peer C
Peer B
Pool of Peer Review experts ¨
EQARF: Framework for self-assessment
Methodology
Indicator 1, 2 Define goals and objectives
Indicator
3, 7, 8 Coherent with goals
Indicator 4, 5, 6 Collection and processing data; discuss results
Indicator 9, 10Review
Purpose And plan
Implementation
Assessment and evaluation
Feedback andprocedures for
Change
© öibf, Gutknecht-Gmeiner 2009
Contribution of Peer Review to the European Quality Assurance Reference Framework
Peer Review process
(Analysis of Self-Report; Peer Visit)
Follow-up(Consequences of review report and
follow-up activities)
Peer Review Report (strengths & improvement
areas)
Self-assessment of VET providers(Self-Report)
Peer ReviewMethodology
Quality Criteria implemented in the European Peer Review Dissemination and practical use of quality criteria enhanced through Peer ReviewPeer Review as anew methodologyof external evaluation (Further) development and use of descriptive indicators
Benefits of Peer Reviews
• Provide a systematic approach to external evaluation following the quality assurance and improvement cycle
• Is a new instrument for external evaluation and assessment of VET providers
• Combines internal and external evaluation• Focus on core business of the school• Is complementary to other quality assurance models
like EFQM• Ascertain the quality of VET provision• Present the strenghts and showcase good practices• Detect blind spots and weaknesses• Promote cooperation between teachers and networking
Leena Koski
Benefits of Peer Reviews
• Is an attractive and motivating methodology which promotes bottom-up cooperation between VET providers and nurtures the development of a professional quality-improvement culture within the VET sectorEuropean Peer Review:
• Promotes exchange and networking between VET providers within and across Member States
• Can led to further cooperation between VET providers including the establishment of transnational mobility schemes.
Leena Koski
• Scenario 1: Grass-roots model• relies on grass-roots activities by VET providers with no European
coordination and support
• Scenario 2: Decentralised coordination model• encompass a coordinating body on the European level which in the
“decentralised coordination model” would primarily provide technical and administrative support
• Scenario 3: Transnational cooperative model• active and stringent cooperation between the different actors is
foreseen with a coordination body that also provides expert support for both the national lead bodies and the VET providers involved.
• operative tasks and responsibilities by and large remain with the national bodies
(
Leena Koski
Scenarios for transnational European Peer Reviews
• The Thematic Group unanimously and strongly recommends the implementation of scenario 3
• ENQA-VET board has decided to include all three scenarios and the SWOT analysis of them.
• Forecast for transnational European Peer Reviews 2010-2013
(Source:Gutknecht-Gmeiner 2009 (figures as of March 20, 2009)
Leena Koski
Scenarios for transnational European Peer Reviews
Year 2010 2011 2012 2013 Total
No. of Peer Reviews 43 56 70 78 247
• National adaptations and implementation• EQARF - Devise a national approach for
implementing EQARF • QANRPs will make more systematic
cooperation in the future to support the use of Peer Reviews at national level too.
Leena Koski
Scenarios for transnational European Peer Reviews
Leonardo projects and other activities year 2009Leonardo project: Peer Review Extended II, coordinator Öibf, AustriaProject number- LLP-LdV/TOI/2007/AT/0011
1.11.2007 – 31.10.2009Results:• Face-to-face Peer Training, CVET too• Peer Review procedure and criteria for CVET for Finland• Transfer Strategy for Finland. Peer Review as a part of ENQA-VET work programme/Thematic group
National development activities: Peer Reviews as a method of QA. Result agreement between Ministry of Education and FNBE includes year 2009 the national development and support activities to promote the use of Peer Reviews.
Quality Management Recommendation for VET is directed to IVET and CVET
Leena Koski
Quality Management Recommendation for VET
VET providers:• collect and analyse evaluation results and other feedback information;• make use of results obtained in other evaluations, such as the results of
national evaluations;• have an external evaluation of their operations carried out at least once
every three years;• develop their operations by means of internal audits;• co-operate with other education and training providers by
carrying out benchmarking and peer reviews, for example.
Leena Koski
•.
VET providers make diverse use of various evaluation methods and data in order to develop their operations.
PEER REVIEW IN VET IN FINLAND YEAR 2009
PEER REVIEW IN VET IN FINLAND YEAR 2009
At Europeanlevel
ENQA VET
• The structure for European Peer Revies 2008-2009
Leonardo project
- Peer Review Extended II- Peer Review for Continuing VET- Face to face training 1.11.2007 - 1.11.2009
The Manual for EuropeanPeer Reviews
• Student Assessment
Application of the criteriaand indicators for PeerReviews at the nationallevel- quality areas and criteria and indicators based on Finnish Education System and Quality recommendation for VET - 14/16 quality areas -
Activities- responsible partner for piloting face to face training- pilot face to face training- to take part in application of the Peer Review Procedure and the Manual suitable for VET
Peer Review as a tool in skilldemonstrations for VET
- ESR project- to pilot Peer Review procedure and develop criteria for skill demonstrations
In Finland
guide for national approach
Qualityarea
2007 Leena Koski/rtLeena Koski
Preliminary results of a Survey on QA to the Finnish VET Providers
• FNBE has made a survey on QA year 2004 and spring 2009.
• FNBE has sent the on-line questionnaire to 158 VET providers.
• 110 answered the questionnaire (69 %).• 27 respondents (27 %) has made systematic
quality work over 10 years, 28 respondents (27 %) 6 -10 years, 19 respondents (19 %) 3-5 years and 28 respondents (27 %) less than 3 years.
An example: National Peer Reviews in CVET in Finland
• Competence-based qualifications are independent of the method of acquiring the required skills prior studies or formal training.
Three levels of qualifications: * Vocational qualification * Further vocational qualification and * Specialist vocational qualification.
Individualisation is embedded in all the phases of getting aqualification and vocational skills:1. Individualisation of the applaying phase2. Individualisation during the making of a competence test3. Individualisation in completing the vocational skills
Leena Koski
National Peer Reviews in CVET in Finland
• Preparation with VET providers national criteria for CVET is in final stage in Finland
• The procedure for Transnational European PeerReviews can be used in Finland for both IVET and CVET (only some slight modification needed).
• Challenge: To find common criteria for transnational EuropeanPeer Reviews for CVET
Quality areas:1. Organization of Competence-based Qualifications2. Individualisation at the application stage 3. Acquisition of the required vocational competences4. Individualisation during competence-based qualification
Adaption to CVET• Terminology/definitions and concepts must be
adapted.• Quality Areas have to be modified.• “Real Peers” should also have as “required competence”,
experience in learning and teaching adults, as adults. • CVET is organised different than lVET, the processes of
accreditation of prior learning are more important in the continuing VET.
National Peer Reviews in CVET in Finland5. Assessment of candidates´ competences6. Effectiveness of Competence-based Qualifications7. Diversity and accessibility of the operating environment8. Infrastructure and financial resources9. Leadership, management and administration10. Strategic planning11. Human resources planning, management and development12. Staff well-being at work13. Co-operation with stakeholders and internationalization14. Social interaction and participation15. Gender mainstreaming16. Quality management and evaluation
Leena Koski
Leonardo project ”Peer Review Impact”
Leonardo project: coordinator FNBE1.10.2009 – 30.09.2011Results:• Report on Impact of Peer Reviews• Peer Review Impact Guidelines• Self-assessment guide for Peer Reviews used on VET
provider or school level• Seminar/Workshop for VET provides how to use
Impact Guidelines
More information: leena.koski@oph.fi
Leena Koski
top related