ETF Policy Briefing: Vocational School Self-Assessment Turning Schools into Learning Organisations

phot

o: E

TF/A

. Jon

gsm

a

ISSUE 10

This policy briefing builds on theexperience of the European TrainingFoundation’s (ETF) Community ofPractice in Quality and QualityAssurance . Between 2009 and 2011 anumber of activities were implementedin the ETF regional mutual learningprogramme. The community ofpractice was used for knowledge-sharing across participating countries .The main objective was to create anetwork for policymakers andvocational education and training (VET)experts, and to enable the exchange ofinformation and experience, debate andmutual learning on quality assurance inVET. To facilitate the work, the ETFapplied the key characteristics of policylearning while the ‘learning platform’focused on quality in VET.

1

2

3

Through four face-to-face peer learningmeetings which took place in Italy(May 2009), Turkey (November 2009),Hungary (September 2010) andMontenegro (September 2011), aworkshop was organised to brainstormand discuss the most recentdevelopments in VET, such as policiesfor quality including the role of keyindicators, self-assessment invocational schools and links withexternal evaluation. A number of peersessions were held in whichcommunity members could reflect on,critique and summarise major issuesand draw conclusions. These findingswill be useful for policymakersconcerned with quality in VET, and whoare considering vocational schoolself-assessment as a policy choice forthe transformation of schools in theircountry

What is a vocational schoolself-assessment? Cedefop (2011)provides the following definition:

‘Any process or methodologycarried out by a VET provider underits own responsibility, to evaluateits performance or position inrelation to two dimensions: aninternal dimension (“micro level”)that covers services, internal staff,beneficiaries or clients, policyand/or internal organisation,development plan, etc.; and anexternal dimension (“macro level”)that covers analysis of theeducational offer of this institutioncompared to others: relationshipwith the territorial system of actors(local decision-makers, unions, localgovernments, type of labour marketand needs of VET, informationnetwork, type of populationsinterested in a learning offer andevolution of the needs, mainresults of work at national andEuropean levels in the VET sector).’

This conception of vocational schoolself-assessment provides anopportunity to evaluate a range offactors that influence the performanceof the school. Many factors areinternal, while others are external,such as VET policies, developments inthe labour market and theattractiveness of the vocational profilesoffered, as well as the availability ofjob placements – all of which have animpact on quality.

Vocational school self-assessment is aprocess that starts with a set ofcriteria against which the school canmeasure its performance. In more

practical terms, self-assessmentfollows a similar logic in all the schoolsin which it is implemented. It is aprocess of looking at specific areasand collecting evidence about theschool’s performance in these fields,measuring how quality has beenachieved. It looks at three areas:

What are the challenges andopportunities for the school?

How well is the school doing?

How can the school be improved?

In other words, a vocational schoolself-assessment is concerned withbuilding the staff’s awareness of thechallenges and opportunities confronted,and identifying possibilities for changeto address these challenges andenhance performance

�

�

�

THE PROCESS IN THREE STEPS

1

2

3

This community includes representatives from education ministries, VET agencies and vocational schools, as well as education inspectors and social partners.A ‘community of practice’ is a learning forum where motivated and skilled participants share their experiences of a specific practice and then work together to improve

that practice. In doing so they mutually develop new procedures, models and tools that they will go on to share with the wider community (Wenger & Snyder, 2001).For more information on the ETF’s use of communities of practice, see Nielsen (2011).Albania, Bosnia and Herzegovina, Croatia, the former Yugoslav Republic of Macedonia, Kosovo

, Montenegro, Serbia and Turkey.(so-called without prejudice to position on status, and in line with

UNSCR 1244 and the ICJ Opinion on the Kosovo declaration of independence – hereinafter ‘Kosovo’)

JANUARY 2012

VOCATIONAL SCHOOL SELF-ASSESSMENTTURNING SCHOOLS INTO LEARNING ORGANISATIONS

THE FOCUS OF QUALITY

IMPROVEMENT POLICIES

The nature and extent of the challengesin turning a vocational school into alearning organisation are also dependenton the that a particular policy, orset of policies, sets out to achieve.Experience with policy implementationsuggests that there are some educationgoals which are more difficult to realisethan others, and which are likely to meetwith more resistance from the school-level actors who have to put them intopractice. Therefore, it is important toconsider the diverse ways in which policygoals present implementation challenges(Sultana, 2008).There are a number ofsimilar developments throughout thecountries of the region which, to asignificant extent, integrate vocationalschool self-assessment into theireducation and training policies.

goals

In the former Yugoslav Republic ofMacedonia, the government strategy foreducation identifies three priority areasfor policy action: decentralisation,improving the quality of education andpromoting social inclusion and cohesion.Since the school year 2008/09 vocationalschool self-assessment as an approachto quality has been compulsory by law.At the heart of the process are sevenpriority areas: teaching plans andprogrammes; student achievements;teaching and learning processes; supportto students; school climate; schoolresources; and issues of leadership andmanagement. A number of quantitativeand qualitative indicators have beendeveloped for measuring quality. Basedon the self-assessment process, thevocational school prepares a report,which is the starting point for acomprehensive evaluation of the school.

In Montenegro, recent changes ineducation legislation favour vocationalschool self-assessment as a tool forquality development, linking it withexternal evaluation. According to theGeneral Law on Education, which cameinto force in August 2010, maintainingand improving the quality of vocationalschool work should be done throughself-assessment: each year anassessment should be conducted inseveral areas of education and trainingservices, and every two years a

complete and comprehensive self-assessment should be carried out. Thereare three institutions in Montenegro incharge of quality in education: the Bureaufor Educational Services, the ExaminationCentre and the VET Centre. The VETCentre, through its EvaluationDepartment, is in charge of the externalevaluation of vocational schools.According to its methodology for qualityevaluation, VET Centre advisors assesseach school’s achievement of standardsin the key areas and prepare a report,describing the situation in the school andproviding recommendations. On thisbasis, the school prepares a plan forquality improvement over a period of fouryears. The report is submitted to theMinistry of Education and Science andEducational Inspection.

In Croatia, quality is one of the mainissues in education and training policy.As a principle it is enshrined in bothstrategic and legal documentation.While the Act on Primary and SecondaryEducation regulates the externalevaluation and self-assessment ofprimary and secondary schools, the VETAct, adopted in February 2009, regulatesthe external evaluation andself-assessment of vocational schools.Hence, all vocational institutions areobliged to conduct a self-assessment,followed by an external evaluation (VETAct, Art. 11). The key institutionresponsible for the development of

quality in VET is the Agency for VET andAdult Education.

In Albania, the piloting of vocationalschool self-assessment, led by theNational Agency for VET, started in 2010.Bosnia and Herzegovina is in the processof strengthening its educationinstitutions, and various reforms aretargeting quality in schools. Kosovo hasinitiated VET quality improvementreforms, with a focus on measuringschool performance based on a set ofindicators. Quality improvement invocational schools though theintroduction of a number of qualitystandards is an important policy inSerbia.

In Turkey, total quality management(TQM) is used in vocational schools. TheMinistry of National Education initiatedits implementation back in November1999, through the adoption of theso-called Total Quality ManagementImplementation Directive (MoNE, 1999).TQM is not compulsory for all schools.However, in order to motivate schools tobe involved in quality developmentprocesses, the ministry introduced anaward for quality in education, based oncriteria published in the Manual of Awardfor Quality in Education. TQMimplementation at school level startswith vocational school self-assessment .Self-assessment reveals the ‘strengths’and ‘improvable aspects’ of the school.

4

5

4

5

According to the main principle of TQM, quality does not depend on the control of the product after its production, but rather on continuous improvement, which isbuilt into different organisational processes. In the case of vocational schools these processes will refer mainly to teaching and learning, and communication withstakeholders.In the case of TQM in vocational schools in Turkey, the so-called Excellence Model developed by the European Foundation for Quality Management (EFQM) has been

applied as a basis for the continuous improvement of those areas of concern that emerge from the self-assessment process.

phot

o: P

urdu

e

ISSUE 10

After several years of experience withTQM applications based onself-assessment activities in vocationalschools, the results were evaluated.According to the study, those teacherswho were involved in self-assessmentteams and TQM processes gave positiveviews about their effects on improvingthe feeling of belonging to the team andthe emphasis shifted from personalachievement to school achievement(moving from ‘me’ to ‘us’). The studyconcludes that the teachers who activelyparticipated in TQM andself-assessment in vocational schoolswere more open to ‘making an attempt tobe a part of the solution rather than theproblem’, and made more optimisticevaluations when compared to those whodid not take part in TQM. It is believedthat the main reason for the success ofTQM-based vocational schoolself-assessment is the possibility itaffords to move quickly to practicalimplementation.

Vocational school self-assessment is alsoa policy option for improving the school asan organisation. While its primary featureis the fact that it is based on dialogueamong staff, it also incorporates a reviewprocess. This helps the school to analysethe challenges it faces. It is a process forimproving the quality of VET in theschools, and it has the potential to turnthe school into a learning organisation.Nevertheless, no development is possiblewithout the appropriate policy framework.

Today vocational schools in differentcountries are invited to undertake self-assessment as part of national policies forVET quality development. At the EU level,the use of self-assessment is furtherencouraged through the European QualityAssurance Reference Framework(EQARF) . The EQARF is based on theidea that self-assessment is linked to thedevelopment of quality both at the VET-system level and at the school level. Thusit provides a systemic approach to quality,incorporating and interrelating therelevant levels and actors

6

6 The European Commission’s Directorate-Generalfor Education and Culture has established theEuropean Quality Assurance in Vocational Educationand Training (EQAVET) Network and, since May 2010,the EQARF for VET has been adopted as a basis forcooperation between Member States. The EQARFdescribes the various elements in a quality modeland raises a number of key questions to beconsidered by the major stakeholders.

OPPORTUNITIES AND CHALLENGES

The ETF community of practice visited a number of vocational schools in Turkey,Hungary and Montenegro, which had introduced self-assessment. A common featurewas the emergence of new development projects and initiatives as a result of teamsof teachers under strong leadership working together. Vocational schoolself-assessment puts a firm emphasis on the change that comes from within a school,rather than through external projects or regulations. This means that a school sets itsown development goals, identifies areas for improvement, implements necessaryactions and reflects on how problems and challenges can be solved, while, at thesame time the progress towards achieving development goals becomes measurable.Discussions with management and teachers in the schools confirmed thatself-assessment helped to raise awareness of both individual and collectiveresponsibility. The innovative approach and creativity of the vocational school as alearning organisation becomes a crucial parameter for survival, promoting adaptabilityin the face of a demanding future, and creating the conditions for growth.

The starting point for teams working on self-assessment in vocational schools is theidentification of the challenges and opportunities that the school faces. In other words,self-assessment needs to take into account a broad set of data that goes beyond theexamination results of students and administrative information. Team decisions needto be made on relevant areas for improvement. It is essential to agree on developmentobjectives, including how to measure the achievement of the objectives. The teammust then launch development projects in the school.

Visits to the vocational schools also confirmed that self-assessment is quite ademanding exercise. The major challenge is how to introduce the process into schools.No manual for vocational school self-assessment can help if the foundations have notbeen correctly laid. Schools often start out with self-assessment questionnaires ordifferent perception surveys for students and teachers. Another challenge is thereadiness of teachers to take part. This is perceived as a new responsibility, and, in theearly days, both teachers and leaders faced a number of questions that neededanswering.

It is essential that a common understanding is created among all the actors regardingself-assessment: from the start there needs to be clarity about the expectations andthe process should be integrated into the school development plan. There are anumber of issues to be considered, such as: How to start and when? Is it an exercisefor the whole school or only for a specially appointed team? How much time will ittake? What resources are needed?

JANUARY 2012

phot

o: ©

Sim

one

D. M

cCou

rtie

/W

orld

Ban

k

For information on our activities,job and tendering opportunities,please visit our website,

HOW TO CONTACT US

www.etf.europa.eu

Prepared by Margareta Nikolovska, ETF

European Training Foundation, 2012Reproduction is authorised, except forcommercial purposes, provided the sourceis acknowledged

BIBLIOGRAPHY

For other enquiries, please contact:ETF Communication Department

http://www.facebook.com/etfeuropa@etfeuropa

E [email protected] +39 011 630 2222F +39 011 630 2200

ISSUE 10

Cedefop (European Centre for the Development of VocationalTraining), , PublicationsOffice of the European Union, Luxembourg, 2011.

EQARF (European Quality Assurance Reference Framework), seehttp://ec.europa.eu/education/lifelong-learning-policy/doc1134_en.htm

ETF (European Training Foundation), ‘Country reports – Albania,Bosnia and Herzegovina, Croatia, former Yugoslav Republic ofMacedonia, Kosovo, Montenegro and Turkey’, ETF community ofpractice country team, Turin, May 2011 (unpublished).

Fullan, M., , Teachers CollegePress, New York, 2001.

Honig, M.I., ‘Where’s the “up” in bottom-up reform?’,, Vol. 18(4), 2004.

Honig, M.I., ‘Complexity and policy implementation: challenges andopportunities for the field’,

, State University ofNew York Press, Albany, 2006.

Hopkins, D., , RoutledgeFalmer, London,2001.

MoNE (Ministry of National Education), ‘Total Quality ManagementImplementation Directive’, , No 2506,November 1999.

Nielsen, S. (ed.), European Training Foundation,

, Publications Office of the European Union,Luxembourg, 2011.

Nikolovska, M., ‘How to achieve educational change in ETF partnercountries: between dreams and reality’,

, Office for OfficialPublications of the European Communities, 2007.

Okay, ., ‘Effects of total quality management applications on thecommunication between the stakeholders in vocational andtechnical secondary education’, Faculty of Technical Education,Pamukkale University, Denizli, 2010.

Oldroyd, D. and Nielsen, S. (eds), European Training Foundation,,

Publications Office of the European Union, Luxembourg, 2010.Senge, P.M., Cambron-McCabe, N., Lucas, T., Smith, B., Dutton, J.

and Kleiner, A.,,

Doubleday, New York, 2000.Sultana, R.G., European Training Foundation,

, Officefor Official Publications of the European Communities, 2008.

Wenger, E. and Snyder, W., ‘Communities of practice: theorganizational frontier’,

, Harvard Business School Press, Boston, 2001

Glossary: quality in education and training

The new meaning of educational change

EducationalPolicy

New directions in education policyimplementation: confronting complexity

School improvement for real

Journal of Communiqués

Learning from‘LEARN’: horizontal learning in a community of practice in South

Eastern Europe

ETF yearbook 2007 –Quality in vocational education and training: modern vocationaltraining policies and learning processes

Portraits of innovative vocational schools in South Eastern Europe

Schools that learn: a fifth discipline fieldbook foreducators, parents, and everyone who cares about education

The challenge of policyimplementation: a comparative analysis of vocational schoolreforms in Albania, Kosovo and Turkey – Peer learning 2007

Harvard Business Review on OrganizationalLearning

Ş

TOWARDS VOCATIONAL

SCHOOLS AS LEARNING

ORGANISATIONS

For VET policymakers, one of the keypriorities is to create the right conditions forthe development of quality in vocationalschools. International literature suggeststhat one of the most important factors forsuccess is finding the right balancebetween bottom-up school improvementand top-down policy steering. In a top-downapproach the main drivers are government-level ministries and institutions, as well aslegislation and decrees.The bottom-upapproach regards vocational schools as thefocal point for policy implementation.Beyond achieving quality in individualschools, for policymakers the biggestchallenge is ensuring that schools ingeneral achieve quality in their operations.

During three years of mutual learning in theETF community of practice, emphasis wasplaced on self-assessment to supportquality, with the aim of eventually turningschools into learning organisations.Furthermore, through self-assessment,vocational schools are seeking ways ofinvolving their various stakeholders, notonly in planning and implementation, butalso in the review process.

Potentially, this approach leads to schoolsthat could offer diversified and multi-learning services to students and

communities as a result of the assessment.Results are verifiable: improved studentachievements, better school leadership,improved accountability and greatersatisfaction among employers. However,‘neither nor strategies willimpact upon the progress of students,

the strategy itself impactson the internal conditions or

change capacity of the school. [… If]continual improvement is to be takenseriously, then the focus […] needs to beon the school’s capacity for development.“Real” school improvement strategiestherefore need to be context-specific, bothin terms of the learning needs of studentsand the organisational conditions of theschool’ (Hopkins, 2001, p. 160, emphasis inoriginal).

From the point of view of usefulness in VETpolicy making, self-assessment can help toensure comparability between schools andcan provide important information foridentifying weaknesses and assessingwhich schools are benefiting from specificpolicy actions. A common approach toschool quality means that it is easier toidentify, understand and find ways ofaddressing differences.

Self-assessment in itself is not sufficient,and some kind of external monitoring ofschools’ performance is necessary. A policyframework for quality in schools combiningself-assessment and external evaluationwith review and improvement processes,

supported by quantitative and qualitativeanalyses, can reinforce the benefits forindividual schools. Potentially, this canprovide a good balance between top-downsteering and bottom-up implementation.

In recent years the idea of the school as a‘learning organisation’ has been on theagendas of various EU school developmentprojects.This concept appears to promoteprinciples that will be in demand in thefuture. It aims to develop organisations thatcontinuously change and develop and areable to systematise and evaluate theirexperience, making learning an ongoingprocess. In a team-based organisation thereis a direct connection between the ‘learningteam’ and the ‘learning organisation’.A team can assemble, elaborate on andassess pedagogical experience in a moresubtle and complex way than is possible forthe individual teacher (Nielsen, 2011).Vocational school self-assessment could behelpful in this respect

external internal

unless at thesame time

JANUARY 2012

phot

o: F

otol

ia