Journal of Philippine Higher Education Quality Assurance

ISSN-1655-8545

Journal ofPhilippine Higher Education Quality Assurance

Volume 1 Number 1 January 2003

A Publication of the ACCREDITING AGENCY OF CHARTERED COLLEGES

AND UNIVERSITIES OF THE PHILIPPINES, INC. (AACCUP)

Editorial Board: Editor-in-Chief: Roberto N. Padua, Ph. D. Mindanao Polytechnic State College Associate Editors: Visayas: Angel C. Alcala, Ph. D. Silliman University Mindanao: Linda Burton, Ph. D. Xavier University Managing Editor: Manuel T. Corpus, Ph. D. Accrediting Agency of Chartered Colleges And Universities in the Philippines, Inc. Editorial Policies:

1. The Journal of Philippine Higher Education Quality Assurance is an annual publication of the Accrediting Agency for Chartered Colleges and Universities in the Philippines (AACCUP). It accepts articles and essays on the theory, practice and applications of Quality Assurance techniques.

2. All articles submitted for publication are subjected to an external refereeing system. The

observations and recommendations of the external referees will be the basis for the acceptance and rejection of an article.

3. The standard journal format is adopted. Authors are requested to submit three (3) copies

of the article to:

Accrediting Agency of Chartered Colleges and Universities in the Philippines, Inc.

812 Future Point Plaza I 112 Panay Avenue, South Triangle,

Quezon City

4. Only accepted articles will be acknowledged by the Editorial Board

Subscription Rate: PhP150.00 per issue

ISSN-1655-8545

Journal of Philippine Higher Education Quality Assurance (Vol. 1, No. 1, 2003)

Author Article Page M. T. Corpus Historical Perspective of the

Philippine Quality Assurance System

1

R. N. Padua International Higher Education Quality Assurance

8

B. Taylor Defining the Location of Responsibility for Institutional Quality Assurance

16

D. Mole and H. K. Wong Balancing Autonomy and Accountability in Higher Education: Quality Audit at City University of Hong Kong

31

M. T. Corpus Redesigning the Philippine Quality Assurance System

45

W. Indhapanya The Thai Standards for Quality Assurance

55

A. Stella and A. Gnanam Unit of Assessment for Accreditation

64

R. N. Padua A Quality-Based Normative Financing for State Higher Education Institutions in the Philippines

71

A. I. Vroeijenstijn Towards a Quality Model for Higher Education

78

This

accountabilia rivalry omechanismsterm used he Keywords a

1.0 Introduction

As is the case in mostcountries, the Philippines isexperiencing the phenomenon ofmass higher education with theconcomitant rise of universities andcolleges (public and private)offering a greater diversity ofprograms, and with varyingcapacity to deliver teaching andlearning services. Unfortunately, asseveral studies on Philippineeducation have revealed, theexpansion of educationalopportunities is inversely matchedby a deteriorating quality ofeducation in the country. This

1

situation has led to an increasing interest in the assessment of the academic program offerings, the manpower and financial capability of educational institutions, and the efficiency of their delivery systems - through a system of accreditation.

Accreditation is seen as a

system of evaluation based on the standards of an accrediting agency. It is a means of assuring and improving the quality of education. Its focus is the assessment of programs by external accrediting bodies using peer reviewers.

Unlike the grant of recognition

Historical Perspectives of the Philippine Quality Assurance System

Manuel T. Corpus Executive Director

Accrediting Agency of Chartered Colleges and Universities in the Philippines (AACCUP)

ABSTRACT

paper addresses the universal issue of autonomy versus/cumty in quality, or put in another form, the issue of quality assurance;r collaboration between the government and non-government/institutions. The Philippine experience in quality assurance (there is “accreditation”) provides the specific country case for study.

nd Phrases: quality assurance, accountability, autonomy.

to operate a school or offer acurricular program which is doneby government, accreditation isprivate and voluntary usingstandards above the minimumrequirements set by government.The private and voluntary nature ofaccreditation is zealously protectedby the accrediting agencies. Forindeed, accreditation originated asa purely private initiative. 2.0 The Rise of Accrediting

Agencies

Accreditation officiallystarted 46 years ago in 1957 withthe establishment of the firstaccrediting agency, the PhilippineAccrediting Association ofSchools, Colleges and Universities.The general condition of theeducational system then must haveinfluenced the birth of an agencydedicated to the promotion ofquality in the education sector. Itwas a time characterized bystructural reorganization and therapid growth of privately-ownededucational institutions many ofwhich were uncomplimentarilybranded as “diploma mills”. Two (2) other accreditingagencies followed after almost two(2) decades. The PhilippineAssociation of Colleges andUniversities-Commission on

2

Accreditation (PACU-COA)emerged in 1973 followed closelyby the Association of ChristianSchools and Colleges-AccreditingAgency (ACSC-AA) in 1976. These three (3) agencies(PAASCU, PACU-COA andACSC-AA) organized themselvesinto the Federation of AccreditingAgencies in the Philippines(FAAP) in 1977.

At a much later time(1987), accreditation got initiatedin the public sector, among stateuniversities and colleges.Parenthetically, this period wasalso characterized by the alarming“proliferation” of new stateuniversities and colleges withquestionable quality of theiracademic capability.

The Philippine Associationof State Universities and Colleges(PASUC) soon sponsored thecreation of an independent body,the Accrediting Agency ofChartered Colleges andUniversities in the Philippines(AACCUP), which was formallyorganized in 1989 although it hadseen its organizational stage two(2) years earlier. In 1995 AACCUPbecame the fourth member of theFAAP.

institutfour (4accredipracticcater clienteCatholthe ProCOA, AACCuniversexposuavailabserviceschoolsexpect accrediUnforthigher and prof theoffered

accredistartedvoluntaof the that thregulatinternato imprNo gobeen cThe praccredithis da

Higher educationions can choose any of the) accrediting agencies for thetation of their programs. Ine, however, the agenciesmostly to their respectivele, i.e., PAASCU, for theic schools; the ACSC-AA,testant schools; the PACU-the non-sectarian; and theUP, the state colleges andities. Given the longre to accreditation and theility of accreditations practically to all types of in the country, we would

a record number ofted programs to date.unately, only 15% of theeducation institutions, publicivate, have taken advantage program of accreditation by the accrediting bodies.

It must be noted thattation in the Philippines as a purely private andry undertaking. The framersaccreditation program stresse concept thrives on self-ion that focuses on thelly-driven private initiativeove the quality of education.vernment participation hadonceived to be necessary.

ivate and voluntary nature oftation has been preserved toy. The accrediting agencies

3

enjoy autonomy with very minimalaccountability to the government. 3.0 Recent Developments and

Government Participation

Against the backdrop of aprivate and voluntary accreditationare developments leading to thenecessity of governmentinvolvement in a supposedly pureprivate concern.

While claiming that

accreditation is a privateundertaking, the accreditingagencies through their federationconsidered it necessary to gainofficial recognition and supportfrom government which they haveeffectively sought and gained. Thisdevelopment diluted the hithertoprivate nature of accreditation, andsignaled the beginning ofgovernment involvement albeit thegovernment contributed more in theform of authority to the FAAP, andbenefits to institutions withaccredited programs, than what itdemanded as the accountabilityfrom the accrediting agencies. In1979, two (2) years after itsorganization, the FAAP wasgranted official recognition by theMinistry of Education, Culture andSports (MECS) as “the body thatwill coordinate with MECS onpolicies, programs, standards and

procedures on accreditation…”

In 1984, FAAP wasrecognized as the official“certifying” agency for theprograms accredited by theaccrediting bodies. This in effectstripped the accrediting agencies oftheir natural role to certify thestatus of programs that they havebeen authorized to accredit. Atbest, the certification by FAAP wasa redundant exercise taking over orduplicating the certificationfunction from the accountableaccrediting agencies.

The FAAP was further

empowered in 1987 when theMinistry of Education, Culture andSports (MECS) recognized it as theagency that would certify, pursuantto its standards, the accreditedstatus of schools and programswhich desire to avail themselves ofbenefits (mostly in the form oflimited administrative andacademic deregulation) extendedby the government to accreditedprograms. This power wasextended to FAAP notwithstandingthe fact that it is not an accreditingagency.

Meanwhile, the Higher

Education Act of 1994 detachedhigher education from theDepartment of Education, Culture

4

and Sports, and created theCommission on Higher Education(CHED) clothing it with authorityincluding the power to monitor andevaluate the performance ofprograms and institutions forappropriate incentives or sanctions,e.g., the withdrawal ofaccreditation. The law specificallyrequired the CHED to provideincentives for accredited programs.

The Commission on

Higher Education, the newgovernment agency responsible forhigher education, re-affirmedgovernment recognition of theFAAP in 1995 with the authority tocertify the accredited status ofprograms granted by theaccrediting agencies in accordancewith its own standards, althoughthis time, it required that thestandards should be “accepted byCHED.”

Accreditation was

accorded a well-deserved attentionin year 2000 when the PresidentialCommission on EducationalReform (PCER), a body created torecommend budget-feasibleprograms which included among itsonly nine (9) agenda for reform, aproposal affecting qualityassurance. The PCER, whilerespecting the autonomy of the four(4) accrediting agencies,

recommended the adoption ofcommon standards. Moreimportantly, it suggested that theCHED take a more active role inthe oversight of the accreditingsystem. Very specifically, itrecommended that CHED beresponsible in certifying theaccreditation status granted by theaccrediting agencies, thus,proposing to withdraw thisauthority from FAAP.

4.0 Scheme for Autonomy Cum

Accountability

Accreditation originated 46years ago as a purely privateinitiative designed to promote self-improvement of private institutions.It was a form of self-regulation. Asconceived, there was no thought ofgovernment participation.

This private “culture” has,

however, been overtaken by eventsthat justified the need for thegovernment. The accreditingagencies themselves actuallysought the recognition and supportof government.

The recent law on higher

education mandated theCommission (CHED) it created tooversee educational programs toassure their quality, even as itspecifically required the provision

5

of incentives for voluntaryaccreditation. The latest body tostudy Philippine education, thePCER, recommended a more activerole of government.

The accrediting agencies

would like to hold on to theautonomy they have traditionallyenjoyed, but with the entry ofgovernment into the picture of theaccreditation system armed withstick and carrot, there is a need fora scheme of defining the roles ofthe accrediting agencies vis-à-visthe government.

The scheme to provide a

system of autonomy withaccountability needs a cleardefinition of the roles of the keyplayers in accreditation. Theaccrediting agencies will: 1) adopta program of accreditation serviceswith their own standards, processesand protocols; 2) conductassessment of programs; 3) grantand certify accreditation status; and4) be accountable for the quality oftheir delivery systems as well astheir finances. The government’srole will be to: 1) give officialrecognition to accrediting agenciesafter qualifying based on itsstandards; 2) monitor the operationof accrediting agencies; 3) grantincentives to accredited programsand institutions; 4) provide

financial subsidy to recognizedaccrediting agencies; and 5) useaccreditation reports in theoversight of education, and inmaking decisions where quality is acritical consideration.

As an alternative scheme,

this role of government may bedelegated to an independent bodyclothed with authority to accreditaccrediting agencies.

The issue of how to

preserve autonomy of theaccrediting agencies, while at thesame time be able to define theiraccountability is a live issue in thePhilippines. Schemes for autonomycum accountability are beingworked out with the assistance oflocal and foreign experts, andprobably learning from foreignmodels. Nevertheless, it is stillunresolved which actuallymotivated the writing of this paperin search of alternative answers. References: Harman, G. (1996). “QualityAssurance for Higher Education”Developing and Managing QualityAssurance for Higher EducationSystems and Institution in Asia andPacific. Bangkok: UNESCO-PROAP. HKCAA (1995). Institutional

6

Review Report: Open LearningInstitute of Hong Kong. HongKong: Hong Kong Council forAcademic Accreditation. Indhapanya, W. (2000). “Standardfor Quality Assurance: A Case ofThai Higher Education.”International Conference onQuality Assurance in HigherEducation in Bangkok, Thailand. Kogan, M., I. Moses, and E. El-Khawas (1995). Staffing HigherEducation: Meeting NewChallenges. London: JessicaKingsley. Lee, H.C. (2000). “QualityAssurance in Higher Education:Standards and Mechanisms inKorea.” International Conferenceon Quality Assurance in HigherEducation in Bangkok, Thailand,November 2000. Tavakol, M. (1999). “HigherEducation Status in Iran.” Researchand Planning in Higher Education,6(4), 1-26.” Vroeijentijin, A.I. (1994).“External Quality Assessment,Servant of Two Masters? TheNetherlands Perspectives.” in A.Craft (ed.). Quality Assurance inHigher Education: Proceedings ofan International Conference.

London: The Falmer Press. Webb, C. (2000). “Mechanisms for Quality Assurance.” International

7

Conference on Quality in HigherEducation in Bangkok, Thailand,November 2000.

International Higher Education Quality Assurance Practices: Situating the Philippine System

Roberto N. Padua

Former CHED Commissioner Vice-President for Research and International Affairs

Mindanao Polytechnic State College

ABSTRACT

The higher education quality assurance practices of selected countries are reviewed and compared with the Philippine system. The purpose of this review is to determine the feasibility of adopting some of the best practices in quality assurance in higher education worldwide in the Philippine context. Keywords and Phrases: quality assurance, internal audit, units of assessment

1.0 Introduction

The Philippine highereducation system is one of the mostextensive in the world. It iscomposed of one hundred and ten(110) public chartered colleges anduniversities and one thousand twohundred eighty (1,280) privatecolleges and universities that serve acollege-age population of a littlemore than two million students. Theprivate higher education institutionsare classified as either sectarian(religious) or non-sectarian, thelatter often belonging to thecategory of family-ownedcorporations. Higher education is,therefore, mainly dominated by theprivate schools which account formore than eighty percent (80%) of

8

the entire sector. The higher education system

operates in a capitalist, free-economy environment and as such,is subject to the factors that influencethe country’s market economy.The private schools competeamong themselves and with thestate-funded colleges anduniversities for enrollment as wellas for government subsidy. Ideally,the presence of a large number ofeconomic players in highereducation would have been sufficientto ensure that individual colleges anduniversities vie for quality. Theinterplay of market forces wouldhave driven the quality of productsand services in higher education to astable and acceptable market

equilibrium. Such is not the case in the

Philippine setting. The interactionof cultural patterns and stateintervention in higher education hasposed a tremendous challenge inquality assurance. The typicalFilipino family values the collegediploma so dearly, regardless ofwhere this diploma is obtained, tothe point that cheap, substandardschools are deriving handsomeeconomic returns by takingadvantage of the situation. Thiscultural reality is a factor thatcannot be ignored when installing adependable quality assurancesystem in the country.

The other factor that poses

a serious threat to quality assuranceis the perceived competition of thestate with the private sectorthrough the former’s annualsubsidy to state higher educationinstitutions. For FY 2002, forinstance, the subsidy given to statecolleges and universities amountedto a staggering PhP 12.6B, withmost of it going to the Universityof the Philippines System. Theinequitable distribution of this hugegovernment resources to the 110state colleges and universitiescoupled with the fact that 80% ofthis defray the cost of salaries andwages, predictably results in less

9

than acceptable quality outputsamong state-funded schools. Withthis perception, the private schoolsare reluctant to invest on qualitywhen even the state does not spendproperly for quality education.

While the Philippine higher

education system grapples withquality, embarking on programaccreditation as a means to achievequality, the rest of the worldcontinuously improve theirrespective practices to attain qualityeducation. The issue of qualityassurance in higher education is nota new one. It began as a concept inindustries as quality controlgradually evolving into morerefined methodologies on processcontrol and finally into qualityassurance. In the Philippines, theadoption of quality assurance beganin 1957 when accreditation ofhigher education programs wasinitiated by an independent body.Accreditation of programs wasconceived of as a voluntarysubmission of institutions to thetenets and principles of quality. 2.0 Review of Quality

Assurance Practices InSelected Countries

A perusal of the current

practices of national accreditingagencies of different countries

shows a great deal of variations. They vary in structure, function and nature. Some of them are established and maintained by the State, while others are independent and private agencies and/or institutional consortia. Accreditation is undertaken on a two-point scale (accredited or non-accredited) or may be carried out on an elaborate continuum of scale and characterized by levels. Many of the accreditation systems are confined to assessments of teaching and learning, research activities or both. There also appears to be variations in the methodologies adopted by the different higher education systems in the world. There are reasons for these diversity in accreditation practices and, therefore, one cannot identify a best practice. Nevertheless, for a given context and a reference, it is possible to identify a workable model that will be suitable for the national context.

Units of Assessment Some countries use the

“program” as a unit of measure while others use the “institution” as a unit of assessing quality. In Mauritius, for instance, where there is only one university, the programs offer the most reasonable unit of assessing quality. However, in

10

India with 245 universities and 11,000 colleges and more than 8 million students, the “program” becomes an unreasonable unit of measure. In a country like the United Kingdom with fewer large institutions, the program was made the unit of measurement. The size of the country’s educational system often dictates the choice of units of assessment. The normal assessment cycle of 4 to 5 years precludes the possibility of using the “program” as a unit of assessment when the system involves thousands of colleges and universities. Program assessment, for instance, is possible in countries like Hongkong and New Zealand where there are very few universities and colleges to be covered, but can be impractical in countries like the United States of America, Russia and India.

In Korea, a new type of

University Accreditation has been started since 1994 executed by the Korean Council for University Education (KCUE). The unit of assessment is the University or the institution itself covering six (6) areas: education, research, social service, faculty and instruction, facilities and equipment, and finance and management. Once accredited, the effect holds for a seven (7) year cycle and is widely publicized. Accreditation results

are used in various ways: for supporting universities financially such as subsidy, scholarships or research, for providing universities with more autonomy and for charting the universities’ growth and development.

In Iran, the unit of

measurement is the institution. The country’s higher education system is roughly classified into two: medical university systems (MUS) and non-medical university system (NMUS). For the former case, institutional accreditation is the same as program accreditation. It is only in the latter case where the full difference between institutional and program accreditation can be observed. Higher education in Iran is under tight state control.

In the Philippines, despite

the huge number of colleges and universities and the extensiveness of its higher education system, the current practice remains that of “program” level accreditation. Since 1957 to date, only a little over 21% of all higher education programs have had the benefit of an accreditation visit in the private sector and only 65% in the public sector.

Nature and Structure

11

Accreditation in some countries is done on voluntary basis while in others, the same is prescribed by the State. Iran, and of late, Thailand, have State-prescribed accreditation while countries like the Philippines, United States and UK have voluntary accreditation. Notably, when accrediting bodies begin to demand for government recognition and government support, the accreditation process will, sooner or later, lose its voluntary character.

In almost all countries,

quality assurance mechanisms provide for Internal Quality Audit (IQA) by external references. Countries that have long history in higher education, like the United States and the United Kingdom, have strong IQA components. Self-regulation through internal quality control mechanisms are imposed by the universities unto themselves.

In the methodologies

adopted by countries like Thailand, the Philippines, Indonesia and Malaysia, strong emphasis is placed on independent external quality assessors. Despite the extensiveness of the Philippine higher education system, for instance, a major difficulty is encountered in finding the

independent external experts whowill audit the quality of theprogram offerings of institutions.The system’s current practice oftraining a pool of accreditors fromtheir own ranks is a barrier toobtaining an internationalrecognition of supposedly quality-certified programs.

In the United Kingdom, the

institutional QA methodologyemploys academic audits,institution-wide self-analysis, staffappraisal, teaching observations,external examination andmonitoring of outcomes. Theinstitutional QA is followed by anational QA where the QualityAssurance Agency conductsinstitutional review and externalaudits of subjects. The recent moveof UK higher education system toestablish a Quality AssuranceAgency (QAA) was in response toa felt need for an independentexternal quality team not affiliatedwith any of the universities in thecountry.

The ultimate mission of

quality assurance is: “to promotepublic confidence that the qualityof provision and standards ofawards in higher education arebeing safeguarded and enhanced”(Webb, 2000). Public confidence isenhanced and promoted if the

12

clients know that the assessment ofquality is made objectively. For thisreason, accreditation systems indifferent countries are constantlybeing improved in the direction ofensuring external, unbiased and fairassessment. Accreditation, as aprocess, would fail to generatepublic confidence if the pool ofaccreditors or experts came fromthe same ranks of universities to beevaluated.

The concept of internal

quality improvement (IQ) andaccountability (A) are clearly statedin the quality assurance documentsof Iran. Internal qualityimprovement refers to the set ofactivities done at the institutionallevel to ensure that qualitystandards are being observed.Accountability refers to theresponsibility of the institution toaccount for the governmentresource spent on it and this is donethrough an External QualityAssurance (EQA) mechanism.

Function

Accreditation results are

used in various ways in thecountries reviewed. Results areused, generally, in three (3)distinct ways: a) as a basis forgovernment subsidy,

scholarships and grants, b) as a basis for informing the public about the quality of education offered by the colleges and universities, and c) as a basis for further improving the educational services of the university.

In the United Kingdom, all

three functions of accreditation are observed, but perhaps, with great emphasis on the first. Since the quality assurance system of that country is well in place, reliance on accreditation results is high.

In Iran, public subsidy to

higher education institutions is slowly being linked to accreditation. The country has just started with its internal quality improvement for the Medical University System and will soon embark on its External Quality Assurance. Thus, no link has yet been established to connect quality with public subsidy. QA results are therefore utilized mainly for self-improvement of the universities.

Thailand’s system of

voluntary accreditation in the past has precluded the use of accreditation in determining state subsidy to colleges and universities. With the transfer of

13

accreditation function from the Ministry of University Affairs to the Ministry of Education, it is expected that accreditation results will be used more extensively by the higher education system of Thailand.

The Korean Council for

University Education (KCUE) utilizes accreditation results in all three ways. Efforts are currently being directed towards refining the assessment procedures and mechanisms to make public subsidy more responsive to quality improvements.

In the Philippines,

accreditation results are generally used for the grant of more autonomy to colleges and universities. Public subsidy is not linked with accreditation results but to other quality indicators. The Philippine Commission on Higher Education recently published a list of universities granted with “autonomous status” in recognition of their accreditation status.

Countries that have used

well established quality assurance systems, generally, utilize accreditation results as basis for public subsidy. Corollary to this, countries which put lesser reliance

AreComp

1. Unit ofAsses

2. Size oeducasystemcollegeuniver

3. NatureStructu

4. Functi

on their quality assurance systems use accreditation results in ways that donot involve government funding. 3.0 Summary

as of arison

United Kingdom Iran Korea India Thailand Philippines

sment

Program / Discipline

Institutional Institutional Institutional Program Program

f higher tion (no. of s and

sities)

Small Relatively large Relatively large

Very large Large Large

and re

Non-voluntary Internal and

external quality audits

Non-voluntary Internal and external quality audits

Non-voluntary Internal and external quality audits

Voluntary Internal and strong external refereeing system

Voluntary External

Voluntary Internal system with external audits by accrediting agencies

on Basis for public subsidy Basis for

grant of autonomy Basis for

institutional improvement

Basis for institutional improvement

Basis for public subsidy Basis for grant of autonomy Basis for public institutional improvement

Basis for institutional improvement Basis for public subsidy

Basis for institutional improvement

Basis for grant of autonomy Basis for institutional improvement

Table 1. Quality Assurance Practices in Selected Countries

assurancountritable a

4.0 R

experieseems combin

The current qualityce practices in selectedes are summarized in thebove.

ecommendations

On the basis of thences of other countries, it

feasible to try out aation of program and

14

institutional accreditation in the Philippines, given its huge higher education system. The Commission on Higher Education needs to take a stronger hand in the accreditation process and may consider shifting from pure voluntary accreditation to prescribed accreditation, if public subsidy is to be linked with accreditation results. A more reliable External Quality Assurance

System needs to be put in place inthe country’s higher educationsetting.

References: Barnette , R. (1995). Improving Higher Education (Total Quality Care). London: SRHE.

Bazargan, A. (2000). “InternalEvaluation as an Approach toRevitalize University Systems: TheCase of the Islamic Republic ofIran.” Higher Education Policy,13, 173180. Bazargan, A. (1995). “InternalEvaluation and Its Application inContinuing Quality Improvement.

15

Research and Planning in HigherEducation,” 3(374); 1-22 (inPersian). Brower, M.J. (1994).“Implementing TQM with Self-Directed Teams.” in H. Costin(ed.), Readings in Total QualityManagement, pp. 403-420. FortWorth, TX: Dryden Press. Dhanarajan, G., and A. Hope (1992). “Quality Assurance at the Open Learning Institute” in A. Craft (ed.), Quality Assurance in Higher Education: Proceedings of an International Conference, Hong Kong 1991, pp. 207-220. London: Falmer.

16

1.0 Introduction

As higher education institutions in many parts of the world face rising costs, shrinking resources, and increasing demands for accountability, their faculty and administrators are giving new attention to the quality of the services their institutions offer. Hand in hand with this enlarged focus on quality comes a new attention to quality assurance procedures: the various systems through which institutions and their staff assess themselves and their instructional and research activities, with an eye toward maintaining and

improving quality. Reviews of the effectiveness of quality assurance procedures, and of the appropriateness of their objectives, have become commonplace in universities. In several countries, procedures for “academic audit” have been introduced, in which an external body (often a national-level regulatory or accrediting agency) reviews an institution’s quality assurance procedures and reports on their appropriateness and adequacy.

Defining the Location of Responsibility for Institutional Quality Assurance

Bruce Taylor

Project Coordinator, Quality Improvement Team Open Learning Institute of Hong Kong

ABSTRACT

The focus of this paper is on defining the location within an

institution where the responsibilities of internal quality assurance are lodged.The paper shows three “pure” alternative arrangements for locating theseresponsibilities, and notes their potential advantages and disadvantages. Thisis followed by a discussion of arrangements at one institution, the OpenLearning Institute of Hong Kong (OLI), which in the past year has undertakena review of its quality assurance procedures including the question of definingthe focus of responsibility for quality. Keywords and phrases: location of responsibility, quality assurance, QualityImprovement Team (QIT)

These quality systems do

not appear from thin air; nor can they be entirely self-regulating

even in an institution where all staff have a demonstrated commitment to quality. Some person or body, either internal or external to the institution, must take the initiative to conceive of and develop a system of quality assurance procedures that is best suited to the context of the institution. Once adopted and implemented, some person or body must monitor the system’s functioning: both in the narrow sense of monitoring compliance with the system’s procedural requirements (producing reports, etc.) and in the broad sense of determining whether the system provides benefits commensurate with the time and resources invested in its operation. Some person or body must also address the question of follow-up: do academic units, and individual teachers use the outcomes from the system to improve their own performance - or are the various reports given a cursory review and then set aside? Finally, some person or body must review the structure of the quality system itself and propose modifications when needed. In this paper, these various responsibilities are grouped together under the heading “responsibility for institutional quality assurance”. 2.0 Alternatives for Locating

17

Responsibility for Quality

Within the organizational structure of a typical higher education institution, there are a number of possible places to lodge responsibility for academic quality assurance. Three conceptually distinct (so-called “pure”) arrangements, in particular, suggest themselves. These are discussed in turn, with an indication of the favorable and unfavorable consequences that might arise in an institution as a result of their adoption.

a) One arrangement entails assigning responsibility for academic quality assurance to a senior academic officer - perhaps a provost or pro-vice-chancellor, who might assume the additional title of “Director of Quality Assurance” or something similar. This officer becomes the focal point for the institution’s quality assurance activities, and assumes responsibility for monitoring their functioning and assessing their performance. Such an arrangement can be termed “managerial” or “executive”, in the sense that administering academic quality assurance becomes, under these conditions, principally a management function.

arrangefollowi

•

•

•

•

•

•

Pros and cons of thisment might include the ng:

The presence of an individual with responsibility for the functioning of academic quality assurance allows for a uniform and coordinated institutionalresponse if, say, external pressure to demonstrate “accountability” is appliedby a funding agency or an accrediting body. There is no question as to accountability, or ultimateresponsibility. In the eyes of the institution’s chief executive and its Council or other governing board, the responsibility for maintaining academicquality is clear. By virtue of his/her senior position, this officer can act as a powerful champion within the institution for development of a culture focused on quality. Proponents of Total Quality Management agree that to be successful, aninstitution’s efforts at promoting a quality culture

18

must receive strong, visible endorsement from the managers who stand at thetop of the organization. Depending on the attitudeof the individual in the position and the constraintsplaced upon him/her by the institution’s governance structures, there may be considerable latitude for him/her to experiment, innovate, carry out pilot investigations or case studies, and so forth. (Care must be taken, however,that new or innovative procedures do not become “the Director’s thing” - implying a lack of ownership on the part of the affected staff).

With so much authority for quality assurance matters invested in a single, highly visible “quality czar”, there is an understandable tendency for staff to consider quality as “that person’s job”, when it is more appropriately viewed as each person’s job.

The inevitable tendency for decisions on quality to becentralized and

•

gives qualitybody (often Academsubcombody. termedon pcollegimost already

personalized under thisarrangement runs contraryto the view that effectiveimplementation of a qualityassurance system requires aprofessional commitmentby all participants in thesystem - and theempowerment of thoseparticipants to demonstratethat commitment.

Problems may arise incommunication if themanager is unable toarticulate effectively toother staff the purposesserved by academic qualityassurance processes, or thebenefits gained from them.In the extreme, this maylead to a loss of legitimacyfor the entire qualityassurance system.

b) A second arrangementresponsibility for academic assurance to the centralregulating academic affairscalled a Faculty Senate oric Board), or to amittee reporting to such a

This arrangement might be “collegial”, in that it buildsre-existing structures ofal governance present ininstitutions - oftentimes empowered with certain

19

review functions, in areas such asthe curriculum - and enlarges theirresponsibilities to cover the entirespectrum of academic qualityassurance activities.

This arrangement also has

its favorable and unfavorableaspects:

• Ownership of academic

quality assuranceprocedures by the keyacademic body fits wellwith the traditional form ofgovernance accepted inuniversities, wherein theteaching faculty assumeresponsibility for decisionsmade on academic matters.

• If it is accepted that at its

heart, quality teaching andlearning ultimately dependon the attitudes ofindividual staff (andstudents), collegialassumption ofresponsibility for academicquality assurance makesevident to staff theunbroken line ofresponsibility extendingfrom the individual (whoaddresses quality issuesaffecting his/her ownteaching) through theacademic unit and the

•

•

•

•

institution seen as whole.Quality assurance systemsare less likely to be viewedas “impositions” and morelikely to be viewed asflowing organically from aconsideration of theinstitution’s academicmission.

A collegial arrangementalso favors thedevelopment of critical,continuous internal reviewprocedures for theinstitution’s academicactivities, carried out bythose who are closest to theprocesses of teaching andlearning. Collegial decision-makingbodies may be especiallyeffective in monitoringdevelopments in areaswhere they traditionallyhave held authority (forinstance, review of thecurriculum). They may beless effective in addressingquestions regarding, say,the effectiveness of follow-up actions by academicunits, owing to a naturalreluctance to sit injudgment concerning theactions of their peers.

20

In most institutions, thecentral academic body hasmultiple responsibilitiesand considerable demandson its time. In manygovernance structures, themembership of such a bodyis disproportionatelycomposed of senioracademic administrators(e.g., Faculty/SchoolDeans) who also haveburdensome administrativeresponsibilities. It might bequestioned, especially in alarge institution, whetherthe Senate or AcademicBoard can offer effectiveoversight of academicquality assuranceprocesses. Even asubcommittee can easily beswamped withresponsibilities.

Many (not all) collegialdecision-making bodiesseek to attain consensus, orat least to minimizedisagreement, and in thisrespect are inherentlyconservative. One mightexpect that under theseconditions, incrementalmodifications to well-established qualityassurance procedureswould be favored over new

academbecomeindepespecifi“QualitSuch autonograntedmanageregulatboth. interdismay amembeinstitutrepreseconstitgradua

approaand mi

•

•

•

•

initiatives that represent asignificant break withtradition.

c) In a third model,ic quality assurances the responsibility of an

ndent group createdcally for this purpose (ay Assurance Committee).a group typically operatesmously \, under authority by an institution’s seniorrs, or the central body

ing academic affairs, orAlmost invariably it isciplinary in membership; itlso have a complement ofrs from outside theion, and it may includentatives of suchuencies as students,tes, or employers.

Like the other twoches, this one has its plusesnuses:

The “empowered team”approach embodied in thisarrangement is one thatfinds favor with manyproponents of TotalQuality Management inboth business andeducational settings (e.g.,Brower, 1994) - who citethe ability of such teams to

21

bring multiple perspectivesto bear on the task ofmaking institutionalprocesses work better, andovercome restraints createdby bureaucratic controlsystems in their pursuit oftheir own and theinstitution’s goals.

The sharing of experiencesamong members of thegroup is an aid indisseminating knowledgeof good practice to allteachers, administrators,and managers in theinstitution. The potential involvementof representatives of arange of constituencies,including student“customers”, as an integralpart of the academicquality assurance processcan only benefit theinstitution - at least in thelong run. Unless a Quality AssuranceCommittee enjoys genuineautonomy and a grant ofexecutive authority, it mayhave to rely on othergoverning bodies (forinstance, a Faculty Senate)to implement its

•

•

institutmixturarrangeFacultyultimatrelatinginstrucadvice Improvacadem

recommendations. Thismay lead to problems if thegroup is not represented asof right on those governingbodies, and must dependon the good-will ofsympathetic members toplace its findings andrecommendations on theiragenda.

Defining the membershipof a Quality AssuranceCommittee must be donewith care. If it is top-heavywith Deans and senioracademics, it may beviewed with cynicism assimply an extension of theestablished academichierarchy. If its membersare mostly junioracademics (who may inany event be reluctant toserve in such a potentiallytime-consuming capacity,given the need in manyinstitutions to focus onresearch and publication),there is a risk itssuggestions will not betaken seriously.

Although most academicsare used to serving oncommittees, the concept ofautonomous, self-directed,interdisciplinary teams will

22

be new to many. To avoidconflicts andmisunderstandings, participants need to beeducated as to the properfunctioning of teams, theboundaries of theirauthority, and the extent oftheir accountability. Thisimplies that the decision tocreate an independentgroup to oversee academicquality assurance shouldonly be taken after carefulstudy. Members of suchgroup will also benefitfrom opportunities toeducate themselves, tobuild up both their own andthe group’s competenciesand, over time, to enablethem to contribute to thereview and modification ofthe group’s own operatingprocedures and, indeed, itsoverall mission.

In practice, mostions will operate with ae of these three “pure”ments. For instance, a Senate may assumee authority for decisions to the institution’s

tional function, but rely foron a permanent Quality

ement Team chaired by anic vice-president. The

precise balance of authority willdepend on the institution’s size, itshistory, the degree of centralizationor decentralization evident in otheradministrative processes, and otherunique aspects of its context.

Particularly interesting to

observe are cases where changes inconditions lead to changes in thebalance of responsibility for qualityassurance within an institution.Changes may result from internalfactors: a period of rapid growth,say, or a shrinkage in resources. Orchanges may stem from theexternal environment in which aninstitution functions: say, a demandon the part of a governmentfunding agency for“accountability”, perhapsmanifested in formalizedprocedures for quality audit or evenassessment of outcomes. Recenttrends in any higher educationsystems towards managerial andmarket-driven, rather parallelchanges may occur in both theoperation and the ultimateownership of academic qualityassurance systems. Shifts from onebalance of responsibility to anotheralways bring the potential fordifficulties. In some cases they maycall into question fundamentalpremises of institutionalgovernance and the nature ofrelationships between faculty

23

members and administrators - arecipe, quite often, for openconflict within an institution. 3.0 An Example: The Open

Learning Institute ofHong Kong

Opened in 1989, the Open

Learning Institute of Hong Kong(OLI) is Hong Kong’s distancelearning institution. The OLI offerssome 36 different academicprograms at degree and sub-degreelevel to mostly working adults. Itsenrollment as of early 1996 wasslightly over 20,000 students (orabout 4,900 FTE). OLI makes useof a combination of locally-developed and externally-developed course materials: thelargest provider of external coursesis the Open University in Britain.The Hong Kong Governmentrequires the Institute to be self-financing in respect of its operatingbudget: this means that the full-time academic staff complement isrelatively small (68 staff as of April1996, supplemented by more than750 part-time tutors who are themain point of contact for students).OLI’s self-financing status alsomeans that a continual tensionexists between achieving itsmission of offering high qualitylearning opportunities on the onehand, and accepting the limits of

financial reality on the other - including the need to keep fees at a level that local students can afford, in light of the Institute’s commitment to offer education for all.

Assuring the quality of its academic programs has been a key concern of the OLI since even before the start of operations in 1989. In an inherently conservative community, many had difficulty accepting that an institution that did not impose strict entry requirements could offer a high quality education. Also the concept of distance learning was basically new to Hong Kong and was met with some degree of skepticism. To overcome these concerns the Institute undertook from the outset to demonstrate that its educational offerings were equivalent in standard to those available elsewhere. It did so by a variety of means (Dhanarajan and Hope, 1992): inviting external participation in the review of program content and of the standards of locally-developed courses; using External Examiners to monitor course assessment practices; developing a quite elaborate hierarchy of internal review committees at program, School, and Institute-wide levels; and welcoming external review of

24

its overall quality assurance systems by outside bodies (initially the UK’s Council for National Academic Awards, later the Hong Kong Council for Academic Accreditation). Speed was of the essence in OLI’s academic program planning, and quality assurance systems were at the early stages mainly imported from other institutions - notably the Open University in Britain - where they had demonstrated their validity (see Michael Robertshaw’s paper contributed to this conference).

Under the OLI’s original structure, ultimate authority over decisions relating to the academic program, and to instructional matters, rested with the Academic Board (the central academic body). The Associate Director (Academic) oversaw the operations of OLI’s academic quality assurance systems and produced a twice-yearly report on the standards of course presentation for consideration by the Academic Board. In a 1992 restructuring, the post of Associate Director (Academic) was eliminated and its quality-related responsibilities were transferred largely to the academic Deans, who now serve dual (and somewhat contradictory) roles: as the responsible officials managing quality assurance processes

operating within their Schools, andas coordinators and facilitators ofInstitute-wide quality mechanisms,some of which function as checksand balances on the activities ofSchools.

A further quality-relatedinitiative occurred in 1994, whenthe OLI created a QualityImprovement Team (QIT) with thedeclared goal of promoting theestablishment of a “quality culture”throughout the Institute. The QIT,composed of middle-rankingacademic and administrativepersonnel and chaired by a long-serving member of the academicstaff, undertook to educatecolleagues on matters relating toquality and carried out pilotprojects which reviewed bothacademic and administrativeprocesses. One such project, forinstance, focused on the OLI’sprocedures for reviewing andreporting semester by semester onthe standards achieved in coursepresentation. QIT’s creation,though helping to raise the profileof quality issues within the OLI,did not alter the balance ofresponsibility for quality matters.QIT was not invested with anyimplementation authority, andrecommendations on academicmatters derived from QIT projectshad to receive assent from

25

Academic Board. Such was the situation

when in June 1995 the Instituteunderwent an Institutional Reviewconducted by the Hong KongCouncil for AcademicAccreditation (HKCAA), for thepurpose of determining whether itshould become a “self-accrediting”institution (i.e., one with theauthority to validate its own degreeprograms). In its report of theInstitutional Review, the HKCAAexpressed a concern that OLI hadno individual or group with specificresponsibility for academic quality.QIT’s activities were noted, but itslack of authority to initiate orstimulate change was alsohighlighted. The report elaborated:

… “the panel recommendsthat the OLI creates a focusof responsibility for itsquality assuranceactivities… (I)t isconsidered that it is crucialto locate a focal point forits quality assuranceefforts, where authoritygoes together withcommitment andresponsibility, and whereimprovement andinnovations can becoordinated.” (HKCAA,1995)

Clearly this was asuggestion that OLI designate asingle official with overallresponsibility for academic qualityassurance, along the lines of the“managerial “ arrangementpresented above. This call did notmeet with universal acceptance atthe OLI. “In establishing theQuality Improvement Team weopted to follow the growing trendin quality assurance whichemphasizes the ‘bottom-up’ ratherthan ‘top-down’ approach”, wrotethe Deputy Chair of the OLICouncil, in the Institute’s officialresponse to the HKCAA’s report.“Currently the focus ofresponsibility for quality assurancewithin the academic arena lies withthe Academic Board and ultimatelywith the Director”. The Institute’sHKCAA Institutional ReviewProject Team was also skeptical inits Final Report:

… “we should… worktowards developing an institutionalculture in which self-reflection andcritical analysis of our systems andprocedures are institutionalized -not in the sense of being reduced torituals of little meaning, but in thesense of becoming second nature.The recommendation… that theOLI establish a focus ofresponsibility for its qualityassurance activities needs to be

26

viewed in the light of this largerpurpose, in that it is all too easy toabdicate all concerns over mattersrelating to quality to the person orgroup who is the ‘focus ofresponsibility’. A focus on qualitycannot be seen as ‘their’ job; itmust be everyone’s job.”

Even before theInstitutional Review, somemembers of QIT had formed theview that existing quality assuranceprocesses were adhered toritualistically, without a true senseof ownership by the academic staffwho dutifully produced descriptive(not, for the most part, “self-reflective and critical”) reports buthad little understanding of thepurposes of the system or howtheir actions contributed toachieving those purposes. QITitself adopted early on the premisethat achieving “quality” does notmean attaining a fixed target or aset standard and maintaining thatlevel of accomplishment: rather, itis more concerned with achievingconstantly improving standards.Incentives to attain this latter goalare precisely what are felt to belacking under the current system.From this standpoint, the adoptionof a managerial model for thecentralized control and monitoringof academic quality assuranceprocesses represent a potentially

retrograde step - certainly not onein keeping with recent moveswithin OLI to developresponsibilities for qualityassurance systems, at least at thelevel of the individual course, tothe staff of individual programswho bear the most responsibilityfor delivering those courses (amore “collegial” approach). Ifcountervailing forces ofcentralization and devolution pullstaff in two directionssimultaneously, the danger is thatacademic quality assuranceprocesses will lose whatever focusthey have and degenerate into atangle of related, uncoordinatedreporting activities.

Balanced against theseconcerns are practical questionsregarding the workload imposed onacademic staff. For financialreasons, OLI has always operatedwith a lean staffing structure.Heavily involved with their course-related responsibilities, OLI’sacademic (and administrative) staffare skeptical of committingthemselves to ventures - even thosewith the best of intent - thatconsume their scarce time. Thus thedesirability of empowering staff tocritically review their ownactivities with an eye toward theircontinued improvement must beweighed against the time

27

commitment required: at least asizable minority of colleaguesbelieve they cannot afford theluxury of engaging in leisurelycritical reflection even on their ownwork, let alone on the broaderpurposes of the Institute. (The sameargument can be made in relation toInstitute-wide reviews of qualitysystems, as Robertshaw has notedin his paper contributed to thisconference). At the senior level, theworkload problem is even moreacute, given that Deans havemyriad administrative duties -traceable partly to the fact that thegrowth of Schools has not beenaccompanied at OLI by the creationof “departments” with formalizedpowers and responsibilities, as wellas to the devolution of time-consuming supervisoryresponsibilities formerly belongingto the Associate Director(Academic), as noted above.

The Quality ImprovementTeam raised other issues in itsJanuary 1996 report of activitiesover its first year. One of itsconcerns was that at its creationQIT had been superimposed on topof, rather than integrated into,OLI’s existing structures ofgovernance. (It is not an“empowered team”, to use thephrasing introduced earlier). Forinstance, the QIT is not represented

as of right on the Academic Board.Beyond this, there was a sense thatin excluding Deans andadministrative unit heads from itsmembership - a purposeful decisionat the time the Team was created -QIT was marginalizing thosecolleagues who were potentially thestrongest advocates within theInstitute for building a culture ofquality (the “champion” effectnoted previously). The Team alsoquestioned whether a separately-constituted, permanent QualityImprovement Team could make asignificant independentcontribution to the OLI’s objectivesin the area of quality assurance,above and beyond what itsindividual Project Teamscontributed through their activitiesand their reports.

The QIT’s proposals toaddress these concerns wereradical: in essence, the Team calledfor its own dissolution andreconstitution in a different form. Itproposed that the Chair of QIT (orco-Chairs: the Team left open thepossibility for joint academic andadministrative heads) be a staffmember ranking at the level ofDean or above. This person wouldassume overall responsibilities forInstitute activities in the area ofquality assurance, including theQIT’s own project activities. Whenrequired, the Chair would be

28

assisted by an ad hoc team ofadvisors drawn from the Chairs ofactive Project Teams, who wouldthemselves be drawn from theranks of unit heads (academic andadministrative). As a result, apermanently constituted QualityImprovement Team would nolonger exist, although the group ofadvisors to the Chair(s) might takethat title. Through these proposals,the QIT hoped to cement its place(or its successor’s) in the life of theInstitute more firmly, and bringquality-related issues to centerstage at a time when, perhaps forthe first time in the Institute’s shorthistory, there might be the chancefor critical reflection on how theOLI chose to organize itself tocarry out its academic qualityassurance activities.

The initial response to thisproposal by Deans was that theyhad no time to take on theresponsibilities for institutionalquality assurance, as the QIT wassuggesting. This is probably a fairassessment given the presentdemands on Deans’ time. Notcommented on, but somewhatironic given the context, was thatQIT was advocating thedesignation of an individual (ortwo) as the focal point for theInstitute’s quality assuranceactivities - the “managerial” model,essentially, despite the fact that

much the same arrangement had been criticized earlier by Institute staff including members of QIT! The tension here is perhaps best viewed as one between idealism (the recognition that an Institute-wide commitment to building a culture focused on quality requires a professional commitment by all staff to that end, and the creation of systems that facilitate making and keeping that commitment) and practicality (the recognition that governance of the Institute is centralized, and that at present there is a reluctance to consider “empowerment” [even of the high-profile QIT] to the extent that would allow for the flourishing of a culture where the individual or small work unit can effectively address quality issues independently, as envisaged by the mostly Western proponents of TQM in higher education). Given a conflict between the two, the Team has opted for practicality as a means of ensuring that quality issues remain, so to speak, “in the public eye”. 4.0 Postscript

At the time of writing, the question of locating the responsibility for institutional quality assurance at the Open Learning Institute remains unresolved. The latest development

29

is a proposal to reinstate the post of Associate Director (Academic), partly as a means of liberating the OLI’s Director for more externally-focused activities. If this suggestion is adopted, the Associate Director (Academic) might assume the quality-related responsibilities proposed by QIT for the Chair of that body. Further action on this front must await the deliberations of the OLI Council, the Institute’s highest governing board.

Certain aspects of the OLI’s situations are of course peculiar to it: the constraints imposed by the Government’s self-financing requirement, the need to demonstrate quality to a skeptical community to guarantee the Institute’s survival, the impetus given by external reviewers to the Institute’s reconsideration of the operations of quality assurance systems. There are several elements of the OLI’s recent experience, however, that are likely to have broad relevance to institutions elsewhere. These include:

• the conflicting impulses to make quality processes, as well as other administrative processes, more “efficient” (often meaning under more centralized managerial control) versus more

•

•

•

•

educatito addeducati

“inclusive”

the concern amongacademic staff that theirempowerment to addressquality issues serve ameaningful purpose thatjustifies the time takenfrom their immediateresponsibilities

the need for an investmentin “quality time” - inreflection, thought, andcritical analysis - by seniorstaff in particular to ensurethe success of a “qualityculture”

the role that independentteams are able to play inconsidering qualitymatters, if these are createdwithout reference to theexisting system ofinstitutional governance

the value in many contextsof an institutional“champion” for quality-related issues - no matter,perhaps, that he or she is asenior administrator in analready-centralized governance system.

As institutions of higheron choose, or are compelled,ress the quality of theironal provision, they will

have to make their own choices asto where to locate the responsibilityfor academic quality assurance.The choices will oftentimes beconfusing; their relative merits notclear; yet the requirement iscompelling - for, in the end, qualitydoes not arise spontaneously, butmust be brought to life through thework of an external hand.

References:

Brower, M.J. (1994).“Implementing TQM with Self-Directed Teams”. in H. Costin(ed.), Readings in Total QualityManagement, pp. 403-420. FortWorth, TX: Dryden Press. Dhanarajan, G. and A. Hope(1992). “Quality Assurance at theOpen Learning Institute”. in A.Craft (ed.), Quality Assurance inHigher Education: Proceedings ofan International Conference, HongKong 1991, pp. 207-220. London:Falmer. HKCAA (1995). “InstitutionalReview Report: Open LearningInstitute of Hong Kong.” HongKong: Hong Kong Council forAcademic Accreditation. Kogan, M., I. Moses, and E. El-Khawas (1995). Staffing Higher Education: Meeting New Challenges. London: Jessica Kingsley.

Balancing Autonomy and Accountability in Higher Education: Quality Audit at City University of Hong Kong

David Mole and H. K. Wong

City University of Hong Kong

ABSTRACT

The emergence of different styles of quality management raised apositive and normative issue. The focus of the paper is to make clear whyCity University of Hong Kong adopted as its particular strategy of qualitymanagement an aggressive drive toward quality assurance, and to argue thatthis strategy has generally supported the strategic objectives of the University. Keywords and phrases: quality management, quality audit, quality assurance

1.0 Introduction

“Quality management” has become an inescapable project for university staff and for university managers in Hong Kong, as it has in other parts of the world (Barnett, 1990, 1992; Caldwell, 1992; HKCAA, 1994; Loder, 1990; University of Northumbria, 1993; Vroeijenstijin, 1993). Quality management at a university may fall into three main patterns: accreditation, assessment, and quality assurance. Accreditation provides for an internal, or external, estimate of whether the unit, or programme assessed has reached a threshold standard. Assessment provides an evaluation on some scale of the quality level being

31

achieved. Quality assurance seeks toestablish systems that ensurecontinuous improvement (Ashworthand Harvey, 1994; Bourke, 1986;Craft, 1992; Harris, 1990;Vroeijenstijin and Acherman,1990).

The positive issue is the

role of the particular circumstancesand history of university systems inleading them to adopt one of thethree quality strategies. The relatednormative policy issue is the meritof these strategies in achieving theobjectives of stakeholders in theuniversity system across a variety ofconditions.

The centerpiece of the paper

is an account of the City

University’s experiment with“quality audit” as part of its qualityassurance system. The first sectionof the paper outlines the contextthat led to the adoption of a qualityaudit scheme. This is followed by ageneral account of the operation ofquality audit at the University, andan assessment of the benefits of thesystem. 2.0 Quality Management

Increased attention to thequality of teaching and learning inhigher education, alongsideresearch activities, has become aninternational trend. This trend is theresult of a number of pressures.The most obvious pressure is fromgovernments seeking to ensure thatthey get value for money from theuniversities they fund. Otherexternal pressures have come fromemployers, and the broadercommunity. As universityeducation becomes accessible tomore of the population, moreexpensive, and more important foreconomic and social development,these pressures are bound to grow.

The external pressures arecomplemented by significantinternal pressures. Modernuniversities have been forced toadopt a managerial approach totheir operations. Greater

32

expectations are now placed inacademic managers, and greaterclaims are made by these managers.Managing quality has come to beseen increasingly as a necessaryproject for a universityadministration.

The quality of teaching and

learning has moved steadily up tothe agenda of universities for twoother reasons. Firstly, a gulf hasopened between the professionalaims and incentives of academicstaff, on the one hand, andinstitutional aims and incentives onthe other. While academic staffhave an incentive to gain the kindof academic reputation, and evensurvival, that comes only throughactive research, universities mustshow that they are serving theircore function of providingundergraduate students with a highquality educational experience.

Secondly, there has been a

general loss of confidence intraditional styles of teaching.“Chalk and talk” and standardmethods of assessment have comeunder increasing scrutiny, to bereplaced by an emphasis on“learning” and the “studentexperience”. An emphasis on thequality of teaching and learning hasbeen an important mechanism formaking an appropriate adjustment

to this changing pedagogy.

Hong Kong universitieshave to varying degrees felt allthese pressures. In particular, theHong Kong government has takenpublic sector reform very seriously,making client orientation,performance pledges, and value formoney key features of thesereforms (Hong Kong Government,1995). Closer to home, theUniversity Grants Committee, thearms-length agency through whichlocal universities are funded, hasset in motion a series of “teachingand learning quality processreviews”.

Quality Assurance and Quality Management

City University’s decisionto move toward quality assuranceas a style for managing the qualityof its teaching and learning was inlarge part driven by some basicconditions shared by mostuniversities in Hong Kong andoverseas. Three factors are worthnoting.

Firstly, the economics of

tertiary education are such that theservice provided is very difficult tovalue. Measuring value addedrequires measurement of the valueof both inputs and outputs. A

33

university has only very imprecisemeasures of the value of its rawmaterial - the fresh intake ofstudents, and a similarly imprecisemeasure of the value of the output(Bibby, 1993). Nor can there beany certainty about the contributionof the university to any observedincrement in value. Students growup whatever the university does.

This problem of

measurement might in othercircumstances be resolved by theestablishment of a marketvaluation, but the extent of publicsubsidies of both schools andstudents make this impossible.Certainly these market signals arevery weak in Hong Kong which iswithout private universities.

Thus, the economics of the

university means that assessment isa difficult and controversialenterprise. It is convenient to shiftthe focus of quality managementaway from outcomes and towardprocesses (Higher EducationQuality council, 1994). It is worthnoting that this “solution” givesrise to deeper difficulty, that ofmaking rational judgments aboutthe use of resources to increase‘quality”.

Secondly, as organizations,

universities group together

professional staff whose privilegesand claims as experts must be takeninto account in any move to assessquality, or evaluate a process. Inthese circumstances, the only wayforward is to make “self-assessment”, or “peer assessment”the cornerstone of qualitymanagement. In Hong Kong, theprofessional privileges ofacademics are widely respected,perhaps more widely respected insociety in which teachers havetraditionally enjoyed high status(Acherman, 1990; Kells, 1992).

The lesson to draw from

this observation is not thatintrusive, top-down, assessment isnot a viable long-term option,although this is probably true, butrather that maintaining andimproving quality is more easilyachieved when staff are directlyinvolved in the process of qualitymanagement.

Finally, universities in

many societies play an importantpolitical and cultural role as centersfor knowledge and expertise andfor the free, or freer, expression ofopinion. This role requires ameasure of autonomy. A balancemust be struck between anappropriate independence fromexternal control and the reasonableclaims of stakeholders to a credible

34

assurance about the quality of theservice being provided. The wayforward here was to provide for thewell-constrained involvement ofexternal expertise in the evaluationof quality. It need hardly be addedthat the autonomy of universities isat least as sensitive an issue inHong Kong as in other parts of theworld.

The next result of theseenvironmental conditions has beenthe development by manyuniversities of a quality-management style that emphasisesthe process for maintaining quality,not the quality level, that makesself-appraisal a key element inevaluation, and involves externalexpertise, but where this expertiseis drawn into the process on theuniversities’ own terms. This hasbeen the approach at CityUniversity of Hong Kong. 3.0 Quality Assurance at City

University of Hong Kong

A number of specificfactors have influenced theparticular style of qualitymanagement at City University.The most obvious of these has beenthe shift to university status and“self-accreditation” after a periodof ten years as the City Polytechnicof Hong Kong. Such a shift is by

no means unique, especially in tertiary systems influenced by British models. However, City University took advantage of this shift to accomplish two important ancillary objectives: a move toward a devolved, collegial, decision-making culture; and move toward a much greater emphasis on the research role of the organization, the local echo of general trend noted at the beginning of the paper.

The deliberate decision to foster a new style in university governance is related to the shift from external to internal accreditation of programs. In the early period during which the University was obliged to seek external accreditation, external demands became a source of disproportionate influence for managers controlling the link between the University and external bodies. A much tighter and more centralized system of internal quality control came into existence than could be justified by the demands of external agencies. Once self-accreditation had been achieved, the centralized system came under immediate attack and crumbled very rapidly.

The effort to dismantle centralized quality control, while retaining some of the checks and

35

balances required by management, if it is to provide stakeholders with credible assurance that quality is being maintained, has become one of the main dynamic tensions in the University’s development of its quality assurance system.

The adoption of an institutional mission that provides for an emphasis on research, as well as on effective teaching, has provided much of the background to current inbternal discussion of quality assurance. Initially the adoption of a new research orientation, designed to increase the prestige and funding of the University, led to considerable stress and confusion. However, staff have quite quickly grasped the new rules of the game, and recognized the congruence of institutional and personal objectives under these rules. This has left the management, and the other staff committed to the teaching role of the University, forced to wage an internal campaign for quality teaching, and to seek a new consensus mong academic staff about the institutional mission.

4.0 The Introduction of Quality

Audit Because the conditions of

the quality management problem facing City University were substantially shared with universities in other jurisdictions, the University has been able to benefit the experience of others. What has made the experiment in quality management at City Univesity particularly interesting is the recognition that one of the most valuable instruments in the effort to refine devolved systems and to promote a quality culture is the internal, peer review of quality assurance systems, i.e. quality audit. While quality audit is a familiar feature of the external scrutiny of universities in Britain and elsewhere, there have been very few efforts to introduce this instrument internally (Jermyn, et al, 1994; Navaratnam, 1994),

Working from models

explored in business and by other universities, City University concluded that a “mature quality assurance system would be characterized by two components; (1) a se of devolved quality assurance processes designed to facilitate continuous improvement by the staff responsible for the actual work; and (2) a pervasive “quality culture” that makes the devolution of such systems both possible and appropriate.

36

At a very early stage, the University’s newly established Quality Assurance Committee recognized that its role would be to develop policy in the area of quality assurance and to provide support and advice to staff seeking to implement that policy. The Committee did not regard itself as playing any direct role in assessing, monitoring, or managing quality. A Set of “quality principles” was adopted, placing responsibility for quality with staff themselves. Meanwhile, the Committee set about providing templates for quality assurance systems to operate at the programme level. This required substantial devolution of control ovr rogramme design, over the evaluation of teaching, over staff and student induction and so on.

This devolution was

welcomed, but it raised a critical question. How was the University as a whole to continue to provide the necessary quarantees that the quality of its teaching and learning was infact being maintained?

While it is probably true

that staff rapidly noticed and accepted their “empowerment” and the new role that this implied, it cannot be said that many members of staff grasped much of the

“theory” of quality assurance. It isnot often recognized that a qualityculture is not simply a culture inwhich staff accept theirresponsibility for quality (Cope andSheer, 1991; Gilbert, 1992; Sarahand Sebastian, 1993; Westbrook,1993). A quality culture must alsoincorporate a broadly sharedappreciation of the requirements fora system capable of deliveringcontinuous improvement. Theeserequirements are: that the systemprovide for review and evaluation,including input from users; that thesystem generate agenda for actionpromoting good practice andaddressing defects; that actions arefollowed-up; that the impactr ofactions is checked; and that with anew round of evaluation a newcycle begins.

This is pretty much

common sense, but like much thatlooks like common sense, it takessome absorbing. Even onceabsorbed, implementation is noeasty task. It is always easier toevaluate than to figure out what todo about problems, easier toidentify action than to follow up onit, and easier to devise remediesthan to show that they have animpact. An organization does nothave a quality culture until itsmembers have grasped what isrequired and understand their own

37

role in the implementation of thequality system.

The terms of the policy

problem were therefore to find theright balance between central anddeveloped responsibility for qualitymanagement, while promoting agreater comprehension of themeaning of quality assuranceamong members of the university.

The Quality Audit Scheme

A solution turned out tohave been built into terms ofreference of the Quality assuranceCommittee. The Committee isinstructed to “audit the systems thatassure quality”. A number ofquality audit models are available.The Committee recommended anapproach similar to that taken bythe United Kingdom HigherEducation Quality Council, butsubstantially scaled down andmodified in some importantrespects (Higher Education QualityCouncil, 1995a, 1995b).

The fact that the internalquality audit exercise in theUniversity was able to take offowed much to the vision andcommitment of the Vice-Chancellor;s Office and the QualityAssurance Committee. It was alsoblessed with enthusiastic

professional support and advicefrom a small secretariat which haddeveloped considerable expertise inthe field of quality assurance sincethe inception of the Committeethree year ago. The impendingvisit to the University in a year’stime of the funding body, theUniversity Grants Committee, onteaching and learning qualityprocess review provided theimpetus for the general acceptanceby academic colleagues of thedesirability and necessity forintroducing the internal qualityaudit scheme as part of our qualityassurance system.

The Level of the Audit

One major decision was tofocus the audit on the faculties,rather than on the University as awhole, or on departments.Departments certainly haveimportant responsibilities for thequality of teaching and learning,but these units are too small. Auniversity-level audit is better doneby an outside agency. In fact,“quality process reviews” are nowgoing forward in Hong Kong. Afaculty-level audit provides asuitable compromise. Audit of afaculty also assists in clarifying therespective roles in the qualityassurance system of the center, thefaculties and the departments.

38

Broad Approach of the Audit

The audit scheme can bedescribed under three mainheadings: the broad approach, thefocus, and the process. Theapproach taken is first to emphasizethe key role of the “critical self-appraisal”. The production of theself-appraisal is in itself one of themost valuable parts of the audit. Itprovides an opportunity for thefaculty to review its approach toquality management, to reaffirm afaculty consensus on this approach,and to provide for internal andexternal use a clear statement of theprocesses at work. In the longerrun, as the audit cycle goesforward, it is expected that facultiescan return to their self-appraisals,review their progress and makeadjustments as required.

A second important aspect

of the approach to audit is that it is“peer review”. The audit isundertaken by a small team ofacademic staff members. Onemember of the team is from thefaculty that is the subject of theaudit. The team also includes anexternal member, normally anacademic from another overseas, orlocal, university familiar withquality management in highereducation. However, on one auditteam, a quality assurance

professional from a major localpublic utility joined the team,providing very useful input.

Focus of the Audit

The audit is focused on

three main issues: course design;teaching and learning; and staffappraisal and development. In eachof these areas, an effort is made toensure that quality assurancesystems are operating. The auditteam checks whether feedbackbeing sought, whether problems arebeing addressed, whether actionstaken are having the expectedimpact and so on. As well asinvestigating the operation of thesystems, the team is connected withthe extent to which staff andstudents understand how the systemworks and their own role in thesystem. The Audit Process

The audit process has threemain steps. The faculty provides itscritical self-appraisal and otherrelevant documents to the auditteam. Although assemblingdocuments such as committeeminutes, course reports, teachingevaluation materials, staff appraisalschemes and so on, can betroublesome and gives rise tocharges of bureaucratic excess,

39

these documents are the “trace” leftby the operation of qualityassurance mechanisms. Theyprovide much of the evidencerequired to confirm the existenceand evaluate the effectiveness ofthese mechanisms. The teamreviews the documents anddetermines which issues it will takeup on the audit visit.

The team then spends a

full-day with the faculty,supplemented by an eveningmeeting with part-time staff andstudents. Given the timeconstraints, audit teams are obligedto “sample” programmes and to“track” particular issues, ratherthan review of the documents. Onthe visit a balance must be struckbetween general questions ( if youwish to make a suggestion aboutthe course, how would you goabout it?) and questions aboutparticular issues (we note that youincreased the size of tutorialgroups, how did you manage theimpact of this on learning?).

The final step is the

production of an audit report. Thereport is essentially a report to thefaculty. It is intended to assist inthe faculty’s own effort to reviewand enhance its qualitymanagement. At the same time, thereport provides the University with

an assurance that the faculty has inplace an effective quality assurancesystem, a system that is being used,and is capable of maintaining andenhancing quality.

5.0 Assessment

Striking a proper balanceand maintaining a sense ofproportion has been the guidingprinciple for the planners inconceiving the configuration of theinternal audit scheme and insteering its implementation. Aproper balance has to be struck inrelation to many key attributes ofthe scheme: the time and financialresources to be invested, the role ofexternal input, thecomprehensiveness and scope ofcoverage and involvement, thedegree of training for the auditorsbefore the actual start of the audit,and the application of variouslevers as influence - moralexhortation, reasoning andpersuasion, inducements, authority,the pressure of public knowledge ofrelative performance, and so on. Acolleague dramatized one suchdilemma vividly; “we cannot spendall the time watching over eachother and neglect the moresubstantive task of meeting withthe students, making teachingpreparations and follow-ups, andengaging in other scholastic

40

pursuits, including research andpublications.” Two half-dayworkshops with external experts asfacilitators were conducted forpotential auditors nominated bydepartment heads among theiracademic colleagues. The feedbackon adequacy of training was thatthe actual experience in doing theaudit was the most important thing.

An early assessment

suggests, however, that internalaudit pay off over the longer run.This pay off has come in threeways. Firstly, as expected, audit hasprovided an essential basis for a“mature” quality assurance system.A system is mature because itprovides for the maximum ofdevolution, without sacrificingexternal scrutiny of qualitymanagement, and matures becauseit provides for the continuousreview and improvement of thequality assurance system itself.This review has a very importantrole in identifying and spreadinggood practice, in refiningmechanism that are working well,and in calling into question routinesthat are ill-focused or unhelpful.

Secondly, the audit hasbecome the basis of a trade off. Tothe extent that quality audit can bedeveloped as a useful instrument,central agencies are able to

delegate control over coursedesign, teaching evaluation and soon. As a result, the burden of therelated documentation andcommittee work can besubstantially reduced. An audit-based system turns out to besimpler and more convenient tooperate than other qualitymanagement systems.

Finally, and perhaps mostimportant, quality audit has beenthe tool in the work of creating aquality culture. As noted above, aneffective quality culture combines awidely shared sense ofresponsibility for quality with awidely shared grasp of the wayquality assurance systems aresupposed to work. When the firstcycle of quality audits is complete,about twenty-five members of theacademic staff will have acted asauditors, about fifty will have beendirectly involved in drafting criticalself-appraisals, while about twohundred staff and fifty students willhave participated in audit visits.Involvement in audit must lead to amuch greater understanding ofquality assurance and of the role ofindividual members of theUniversity in quality assurance.

It is still premature to makea final assessment of CityUniversity’s experiment with

41

internal quality audit. However, asindicated by initial review sessionswith some key participants in theaudits, there is little doubt that itsimpact has been positive. Indeed“quality audit” appears to haveprovided a neat, and cost effectivesolution to a difficult set ofproblems.

One reading of theUniversity’s current arrangementsfor quality management is that theyrepresent a modern version of someolder values: a respect forprofessional expertise andautonomy, alongside, recognitionof the need to provide forprofessional self-regulation is seento be effective; demands forexternal scrutiny and control can bemore easily resisted. Acommitment to establish andoperate systems that facilitate thecontinuous improvement of thequality of teaching and learning is avery low price to pay for theprivileges that come withinstitutional autonomy. 6.0 The Way Forward

The internal audit wasconducted in a supportive, frankand constructive manner. With theexercise now about two-thirdsthrough, the general feedback ispositive. These suggest that there

was no “over-kill”, and it was cost-effective addition to our qualityassurance system. The auditedfaculties have taken ownership ofthe exercise and have setthemselves an action agenda indealing with the shortcomingsbrought out by the audit reports,which they publicized widelyamong their colleagues. The auditsteering group will in due coursedocument and publicize the goodpractices in the units to the wholeinstitution. It is also activelyplanning other ways and means tosustain the momentum forcontinuous quality improvement.The separate efforts in qualitymanagement of the academicsupport centers and centraladministration units will becoordinated and dovetailed withthose of the academic units.

The audit exercise hasconcentrated by and large onteaching. This also generallyreflected our standing practices andconcerns. Not enough attention hasbeen paid to the factors andenvironment contributing toeffective learning, deep learningand life-long learning. Nor, wasthere active exploration of the waysand means of engaging students inreflecting on, and committingthemselves to, their own role andobligations in the pursuit of

42

excellence in learning and qualitystudent life. An interim review ofthe audit exercise has brought outthe shortcoming. It, however,confirmed our observation that afairly advanced and sophisticatedquality assurance system and set ofprocedures in academicadministration were already inplace in the University.Nevertheless, it also indicated thatwe are only in the early stages ofthe arduous road in building a deepand prevalent quality culture inteaching, learning and otherscholastic pursuits- a culture wheremembers respect, cherish andinternalize the values of goodpractices, and reflect this in theirbehavior.

The major conclusion and

the most important thing appears tobe that we are on the right trackseeking continuous qualityimprovement.

References: Archerman, H.A. (1990) “QualityAssessment by Peer Review: ANew Era for University Co-Operation”, Higher EducationManagement, 2(2), pp. 179-192. Ashworth,A. and Harvey, R.C.(1994) Assessing Quality in Furtherand Higher Education, London:

Jessica Kingsley Publishers. Barnett, R.A. (1990) “TheMaintainance of Quality in thePublic Sector of U K HigherEducation”, Higher Education, 16,pp. 279-301. Barnett, R.A. (1992) ImprovingHigher Education, Buckingham:SRHE and Open University Press. Bibby, D. (1993) “ Absolute andRelative Quality in HigherEducation” in Proceedings of theFifth International Conference onAssessing Quality in HigherEducation, Bonn, F.R. Germany,19-21 July, pp. 89-98. Bourke, P. (1986) QualityMeasures in Universities,Belconnen: CTEC. Caldwell, E.M. (1992) “Quality inHigher Education: A TheoreticalStudy”,

, (7), August. Educational Research

Journal Cope, R.and Sheer L. (1991) TotalManagement for Organizations:Concepts and Tools, Adelaide:TAFE National Centre forResearch and Development Ltd. Craft, A. (ed.) (1992) QualityAssurance in Higher Education,London: Falmer Press.

43

Gilbert, A.D. (1992) “Total QualityManagement and the Idea of aUniversity” in Proceedings of theFourth International Conference onAssessing Quality in HigherEducation. Enschede, Netherlands,28-30 July, pp 29-50. Harris, R.W. (1990) “The CNAAAccreditation and QualityAssurance”, Higher EducationReview, 23 (33), pp 34-53. Higher Education Quality Council(1994)

, Higher EducationQuality Council.

Guidelines on QualityAssurance

Higher Education Quality Council(Quality Assurance Group) (1995a)Notes for the Guidance of Auditors,Higher Education Quality Council. Higher Education Quality Council(Quality Assurance Group) (1995b)Notes on the Role of AuditSecretaries, Higher EducationQuality Council. HKCAA (1994)

, Hong Kong: HKCAA. Hong Kong Government(Effeciency Unit) (1995) GainingCommitment to ContinuousImprovement. Hong KongGovernment.

Hong KongCouncil of Academic AccreditationHandbook

Jermyn, K., Hall, C., and Burns, G.(1994) “Preparing for AcademicAudit: A Case Study of a NewZealand University”, NZOAConference, New Zealand. Kells, H.R. (1992) Self-Regulationin Higher Education. London:Jessica Kingsley Publishers. Loder, C. (ed.) (1990) QualityAssurance and Accountability inHigher Education, London: KoganPage. Navaratman, K.K. (1994) “QualityAssurance Audits in Technical andFurther Education”, Total QualityManagement, 5(4), pp. 219-225. Sarah, J.V. and Sebastian, R.J.(1993) “Developing a QualityCulture”, Quality Progress,September, pp. 73-78. University of Northumbria (1993)“Quality Assurance at the

44

University of Northumbria“Newcastle, University ofNorthumbria. Vroeijenstijn, A.I. (1993) “CurrentDutch Policy Towards AssessingQuality in Higher Education” inProceedings of the FifthInternational Conference onAssessing Quality in HigherEducation, Bonn, F.R. Germany,19-21 July, pp. 285-300. Vroeijenstijn, A.I and Archerman,H. (1990) “Control-Oriented versusImprovement-Oriented QualityAssessment” in Goedgebuure,L.C.J., Maasen, A.M. andWesterheijden, D.F. (ed.), PeerReview and PerformanceIndicators, Utrecht: UitgeveruLemma. Westbrook, J.D. (1993) “Organizational Culture and Its Relationship to TQM”, Total Quality Management, January/February.

cmo Kb

1.0 Introduction

The accrediting organization for state universities and colleges was born in 1987 with the original name, State Colleges and Universities Accrediting Agency in the Philippines (SCUAAP). However, when it was submitted for registration in the Securities and Exchange Commission, it was required that the word “state” be dropped as the Agency is a private agency. It was finally registered on September 4, 1989 with the name of the organization being changed to Accrediting Agency of Chartered Colleges and Universities in the Philippines (AACCUP), Inc.

Thus, AACCUP joined the

three other accrediting agencies, the oldest of which existed as early as 1957 while the two others started in the mid-1970’s.

In search of a model, AACCUP sought the assistance of the older agencies and practically “copied” from them, the most important of which was accreditation by program. Though accreditation in state colleges and universities was supposed to start in 1987, the initial years were devoted to the organization of the agency, the development of manual of procedures, the preparation of accreditation instruments, the recruitment and training of accreditors, and the task of selling the ideology of accreditation to the clientele (SUCs) which were not

Redesigning the Philippine Quality Assurance System

Manuel T. Corpus Executive Director

Accrediting Agency of Chartered Colleges And Universities in the Philippines (AACCUP)

ABSTRACT

There are certain concerns or issues on accreditation by program that annot be ignored. This paper introduces a framework of new accreditation odel which will depart from program and shift to the institution, as the unit

f assessment.

eywords and phrases: unit of assessment, the framework of accreditation y institution

pre-disposed to plunging on a new endeavor.

It was only on September 15-17, 1992 when the first preliminary survey visit was conducted under AACCUP. What started quite slowly, and even reluctantly, has gained momentum after mid-1990s to the extent that as of the end of the year 2000, there were 65 (among the 108) SUCs that have accredited programs.

Ten years of experience, albeit fruitful, in accreditation has led to the emergence of certain concerns or issues on accreditation by program particularly its relevance, usefulness and the operational practicality of the model culminating in the decision of the AACCUP Board on March 1, 2001 to explore other models of accreditation, and commissioning this writer to lead this project. 2.0 The Unit of Assessment

Different countries the world over adopt varying structures, functions, mechanisms, and practices in their accreditation (or quality assurance) programs. But one of the most serious issues that is currently subjected to review is the unit of assessment.

What is the unit of

46

assessment used globally? Almostall countries have certainmechanisms in place that assess theinstitution as a whole, theindividual academic programs, or afew others using a mixture of both.A few examples may be recalled: inIndia, they assess institutions; inthe Philippines, programs; and inHong Kong, they begin withinstitutional review, and latervalidate the individual programs.The Presidential Commission onEducational Reform (PCER)recommended a modification of themodel in the Philippines byrecommending accreditation bydiscipline, the concept being usedto mean, a cluster of programs,like, engineering and technology,sciences, and others.

What unit of assessment

should be adopted in thePhilippines? Shall we stick toaccreditation by program, or shallwe explore other models assuggested by the CHED-organizedTechnical Working Group onQuality Assurance and by Dr.Marian Phelps, a consultantrecently commissioned by theAsian Development Bank to studythe quality assurance program inthe Philippines?

I would like to offer three

criteria in selecting the unit ofassessment, namely: 1) the number

accredithe acaits colaccredioperatisegregainputs these athe proshared of the the libclassrofacilitiethe stuthe admextend

the inassessmrelevanstakehogovernfunds supporstudentinstitutetc. In and demanassistanand noWhen hearingquestiowould

and size of higher education institutions, 2) the purpose of accreditation, and 3) the feasibility of using the model.

Accrediting by program

has strong points. As it is reviewing a small unit, it enjoys the advantage of being well-focused; it looks into details. However, it is too fragmented, and in a country with over a thousand higher education institutions, it would take many years, perhaps even a century to accredit all programs even in just one cycle. This may be one of the major explanations why in spite of a history spanning more than four decades of accreditation, we can claim a coverage of less than 20%. Obviously, accreditation by program is not the practical approach suited in the Philippines unless we are prepared to accept the continued weakening of our educational system.

Even with only 110 state

universities and colleges, the toll caused by the increasing traffic for applications for accreditation is now being felt by the AACCUP. This alone justifies a need to seek for other accreditation models, one of which points to increasing the scope, shortening the cycle, and improving the quality of the process.

47

One practical advantage of tation by institution is that demic quality is defined by lective impact. Indeed, in tation surveys, it is onally strenuous to te the evaluation of certain to the academic program as re not used exclusively by gram under survey, but are with other units or programs institution. Take the case of rary, the laboratories, the oms, and other physical s. Even the services, such as

dent services, not to mention inistration of programs, are

ed institution-wide.

Another advantage of using stitution as the unit of ent is its usefulness and

ce to the major lders, such as the

ment, which provide the (quite relevant to state-

ted institutions), the s, employers, aid-granting ions, donors, foundations, those cases, the commitment the accountability are

ded from the recipient of the ce, which is the institution

t from individual programs. in legislative budget

s, the legislators ask ns on accreditation, they

like to refer to the

A

De

accreditation of the whole SUC. Itis unfair to say: “we areaccredited” when in fact, one isreferring to only the accreditationof one or two of the over 20programs offered by the institution.

3.0 Towards a New Framework

of Accreditation

A shift from program toinstitutional accreditation calls fora new framework, such as: 1) aredefinition of the scope or focus,

2. Process Focus 3. Performance

a. Outcomes b. Leading

Indicators c. Learning

Indicators

I

Model: ccredited SUC

48

and precisely defining the pictureof the accredited institution; 2)benchmarking what will bemeasured; 3) a new system ofmeasurement; 4) a new breed ofaccreditors; 5) a partnership (withthe SUCs) approach in institutionaland program accreditation; and 6)the leveling of accreditationawards. Figure 1. The Framework of Accreditation by Institution 3.1 Model: The Accredited SUC

pply e

nstru-ent

Award the Level (Status) of Accredita-tion

What are the General Factors (Areas) that form this “Model?”

What are the Specific Indicators? (Quantifiable / Behavioral 1. Policies 2. Best

Practices) 1. Input Focus

Scale of Values What weight will be allocated per 1. General

Factor 2. Specific

Marking System 1. Point

System 2. Likert

Scale

AthIm

Design the Accredita-tion Instrument

Measure-ment

Benchmarking. What will be measured?

Define the model. What is the SUC we want?

ndicator

Level of Accreditation

and oevaluatin defin 3.2 W

be settby indroppinevaluatbe the

by prog10 crigoals acurricustudentand clibrarylaboratadmini

Group to derankingeducatiprivateevaluatadministudentinstrucfacilitieresourc8) res

Is the currently used one-size-fits-all approach still relevantin an institutional accreditation?(The relevance of this approacheven in program accreditation wasalready under question).

How do we define the

institution to be evaluated? Oneway of defining the parameters ofthe evaluation is to assess theinstitution on the basis of itsmission, goals and objectives.Thus, in the scheme, an institutiondefines its mission and itsperformance is evaluated only as itis related to its mission. Thus, aprimarily teaching institution isassessed differently from researchinstitutions. A problem may cropup - how do we handle theprograms pursued by an institutionwhich are not within the ambit ofits mission? How about SUCs withmulti-campuses? Let us admit thatone weakness in the choice of theinstitution as the unit is that in theassessment of the whole, it may notbe able to distinguish between thegood and the bad sub-units, orthe good and the not-so-goodcurricular programs. Of course,certain pre-determined thresholdqualifications or compliance levelsfor all campuses and programs as asine qua non for the award of anaccreditation status may berequired.

49

Given the mission, goalsbjectives, an appropriateion scheme may be adopteding the focus of assessment.

hat will be Measured?

One misconception mustled right way. Accreditationstitution does not meang programs in theion. The programs will stillmajor foci of evaluation.

The present accreditationram under AACCUP adopts

teria, namely: 1) mission,nd objectives, 2) faculty, 3)lum and instruction, 4)s, 5) research, 6) extensionommunity involvement, 7), 8) physical facilities, 9)ories, and 10)stration.

The Technical Workingcommissioned by the CHEDvise a system for the/classification of higheron institutions, public and, recommended 12ion criteria, viz: 1)stration, 2) faculty, 3) services, 4) curriculum andtion, 5) physical plant ands, 6) library and mediaes, 7) laboratory resources,earch, 9) extension and

community outreach programs, 10)performance indicators, 11)information, communication andtechnology, and 12) linkages andnetworking.

This design for theranking/classification of HEIs isinstitution-based; it also includes acriterion on performance oroutcomes.

A paradigm shiftnecessarily needs a dramaticchange in the standards used. Thecriteria adopted by the TechnicalWorking Group on theRanking/Classification of HEIsillustrate attempts to align theaccreditation system to globalpractices, particularly in theinclusion of performance andinformation, communication andtechnology in the evaluation byinstitution, even while it still putsprimary emphasis on inputs.

A new accreditation

paradigm needs to still include theinputs (e.g., facilities), andprocedures (e.g., teaching-learningtransactions and administration ofservices). However, there must bean emphasis on measuring thesuccess of past activities (laggingindicators) through outcomes aswell as the dynamism of theinstitution (leading indicators) as

50

measures of future performance asmay be illustrated by theinstitution’s planning andinformation management systems;and how fast the institution makesadaptations to innovations andchallenges (learning indicators). 3.3 The System of Measurement

The present system adopts

a system of values in theassessment based on the set criteria.Weights are allocated to thedifferent criteria, and evaluation ispursued both qualitatively andquantitatively. The specific tool forevaluation is the accreditationinstrument.

The new measurement

system will contain similarfeatures, viz, adoption of scale ofvalues, assignment of weights tothe different criteria, a mix ofquantitative and qualitativeevaluation, and use of accreditationinstruments. But, the measurementwill depart from the almostexclusive reliance on inputs, to anassessment of outcomes, qualitymanagement practices, thedynamism of the institution, and itsactual performance in adapting tonew challenges. This will requirethe use of more open-endedquestions particularly in evaluatingquality management and services.

3.4 A New Breed of Accreditors

This new model conceivesof drawing some accreditors fromthe private sector or end-users ofthe graduates of the stateinstitutions to join the currentcream of peer accreditors from theSUCs. This will be a newexperience for the host institution,the current crop of accreditors, andthe AACCUP itself.

Another new feature will

be the special qualification of theaccreditors. Under programaccreditation, the accreditors musthave the particular program (e.g.,agricultural technology) to beaccredited as his area ofspecialization based on hiseducation and experience. Ininstitutional assessment, accreditorswill be selected on the basis of theirqualifications in being able toassess certain criteria (e.g.,financial management, research, orlibrary) of the institution as a wholerather than of a particular program.

Aside from the accreditor’s

qualifications to evaluate a certaincriterion (or “area” as we refer to inthe accreditation instrument nowbeing used), the model accreditorwould still be sought:knowledgeable and competent; ableto make rigorous and objective

51

assessments; able to relateprofessionally with the hostinstitution officials, faculty andstaff and to be a worthy teamplayer; personal decorum (e.g.,dressing) with no undesirable orimproper habits; observing ethicalstandards; and other related traits.

The shift in the unit of

assessment and the correspondingchanges that it entails, necessitatesa new program for trainingaccreditors. This calls for theassistance of experienced expertsincluding foreign consultants toenrich this new scheme ofaccreditation with globalexperience. 3.5 A Partnership Approach

The goal of accreditation isto develop and sustain the qualityof the educational servicesoffered by an institution. Thus, itwould enhance the success of theprogram if the institution developsan internally-driven initiative,and/or be a partner in the pursuit ofthe accreditation program. The newscheme proposed here is to adopt apartnership between the hostinstitution and the externalaccreditors. Under this system, theaccreditation of an institution,including the assessment and theaward of accreditation will be the

3.6

othadoacchasschdifdif

adoof accthisincinssta

obscouclamaacctheasc

whinsassof stapro

exclusive role of the AACCUP. Along-side external

assessment, the individual SUCswill be encouraged and assisted indeveloping their respective internalassessment systems manned bytheir accreditors, all or some ofwhom, are AACCUP accreditors.The internal assessment system(body) will be tasked to:

1. conduct self-surveys

using the AACCUPinstrument which maybe validated byexternal accreditors;

2. conduct internal

assessment bydiscipline, or even byprogram, for the use ofthe individualinstitutions, and holdfollow-up activitiesafter an institutionalaccreditation isconducted; and

3. plan, provide technical

advice, and monitorimplementation ofsurvey teamrecommendations made by externalaccreditors.

aw

52

The Levels of Accreditation

In AACCUP, as in theer accrediting agencies, we havepted four (4) levels ofreditation (Incidentally, PASUC adopted a similar classificationeme for SUCs although usingferent criteria and designed for aferent purpose).

Most European countries

pt only two (2) classificationsinstitutions: accredited or notredited. The disadvantage of scheme is that it removes theentive of the accreditedtitutions to aspire for a highertus, to be different from the rest.

India, taking notice of this

ervation in the Europeanntries, has modified the

ssification scheme. She stillkes a distinction between theredited or not accredited, but for former, is a leveling from 1-starending to 5-star institutions.

Under this new scheme

ere we intend to use the wholetitution as the unit ofessment, there is an advantageusing a 4-level accreditation

tus to classify accreditedgrams, provided that we doay with the confusing label of

level I as only granting candidatestatus, not yet accredited. It wouldbe advisable to use level I as theinitial accreditation, and thenimprove upwards to level IV.

It is conceived that the

grant of accreditation, valid onlyfor a certain period (say five yearsfor re-accredited status), may stillbe retained. However, the practiceof requiring that accreditationstatus must pass through pre-determined stages must be re-examined. For example, whyshould an institution be required toqualify first for level II if it isalready qualified for level III? 4.0 Conclusion As mentioned earlier, thiswriter has been commissioned toprepare the new accreditationdesign. This paper is an initialapproximation of a framework of anew accreditation model which willdepart from program and shift tothe institution, as the unit ofassessment. In preparing this paper,this writer has profited fromcountry papers on accreditation orquality assurance, the ideas of afew foreign experts that this writerhas met at the INQAAHE(International Network on QualityAssurance Agencies in HigherEducation) and other foreign

53

conference, from stakeholders likeofficials from CHED and SUCs,private sector education experts andthe officials of AACCUP.Certainly, comments andrecommendations from this groupof senior accreditors would be mostwelcomed. References: Harman, G. (1996). QualityAssurance for Higher Education(Developing and Managing QualityAssurance for Higher Educationsystems and Institution in Asia andPacific). Bangkok: UNESCO-PROAP. HKCAA (1995). InstitutionalReview Report: Open LearningInstitute of Hong Kong. HongKong: Hong Kong Council forAcademic Accreditation. Indhapanya, W. (2000). “Standardfor Quality Assurance: A Case ofThai Higher Education.”International Conference onQuality Assurance in HigherEducation in Bangkok, Thailand. Kogan, M., I. Moses, and E. El-Khawas (1995). Staffing HigherEducation: Meeting NewChallenges. London: JessicaKingsley.

Lee, H.C. (2000). “QualityAssurance in Higher Education:Standards and Mechanisms inKorea.” International Conferenceon Quality Assurance in HigherEducation, Bangkok, Thailand,November 2000. Tavakol, M. (1999). HigherEducation Status in Iran. Researchand Planning in Higher Education,6(4), 1-26.

54

Vroeijentijin, A.I. (1994).“External Quality Assessment,Servant of Two Masters? TheNetherlands Perspectives.” in A.Craft (ed.). Quality Assurance inHigher Education: Proceedings ofan International Conference.London: The Falmer Press. Webb, C. (2000). “Mechanisms forQuality Assurance.” InternationalConference on Quality in HigherEducation in Bangkok, Thailand,November 2000.

The Thai Standards for Quality Assurance

W. Indhapanya On behalf of the Council of University Presidents of Thailand

(CUPT)

ABSTRACT

The old system of quality control in Thai higher education is considered not good enough. The new law requires that an internal quality assurance system must be employed by the university, an external quality assurance will be introduced, and report will be made available to the public. The purpose of the paper is to review and assess the standards for quality assurance system in Thai higher education. Keywords and phrases: internal quality assurance.

1.0 Introduction

In 1999, the new NationalEducation Act was announced andquality assurance for all levels ofeducation was required by law forthe first time in Thailand.Universities in Thailand areclassified by ownership into privateand public universities. Both areunder supervision of the Ministry ofthe University Affairs, but there wasno quality assurance as a system liketoday. Private universities are underclose supervision by the Ministry ofUniversity Affairs as they have to gothrough several steps of approvaland review by external committee. Itmay take years before a privateuniversity is accredited, even so ithas to go through the process of

quality control again for any new degree program. At the other end is a much less control for public universities. They were established by law with certain powers under their own legislations. The Ministry of University Affairs provides guidelines and final approval, but does not have any audition or review after the approval.

Like many countries, quality control in higher education depends heavily on the university itself. Certainly, it does not mean that Thai universities do not take quality of education seriously. They always look for academic excellence, unfortunately they may see this as an internal management rather than an information made

55

2.

available to the public. Universityranking has never been done inThailand but academic reputation isknown in each area of study amongall universities. There seems to bean implicit ranking known by thepublic.

2.0 Quality Assurance Under

the Ministry of UniversityAffairs

The idea of having quality

assurance system for Thaiuniversities originated from theMinistry of University Affairs. InJuly, 1996, the Ministry announcedfor the first time about its policy onquality assurance. The Ministry hasput effort to encourage universitiesto make academic standards. Thepolicy has required all publicuniversities to install the internalauditing system as a means tomaintain the quality of education atpresent and to improve it in thefuture. This policy has been statedas quoted below.

“1. The Ministry of

University Affairswill provide anddevelop the qualityassurance systemand mechanism as aninstrument inmaintaining theinstitution’s

56

academic standards.The principle ofstimulating institutions is toestablish anacademic qualitycontrol systemincluding theircontinuous missionimprovement on thebasis of academicfreedom and of anautonomous nature.This is a way toensure publicaccountability requirements whichlead to wideacceptance inacademic standardsand internationalcompetency in aworld competition.To make thismeasurement work,the subcommittee ofeducational standards has beenestablished forsupervision, providing academicstandards, administration, andaccreditation.

The Ministry ofUniversity Affairswill enable

3.

4.

5.

6.

institutions todevelop their owninternal qualityassurance system asa tool in thedevelopment ofeducational management. It is away to create aninternal qualitycontrol mechanismwith highereducation institutionsthat would beefficient. Therefore,any institution is ableto establish its ownappropriate internalquality control for itsimplementation andevaluation system.

The Ministry ofUniversity Affairshas established theprinciples and theinitial procedure ofpractical measures ineducational qualityassurance which eachinstitution candevelop appropriately inaccordance with eachinstitution condition.By this measure, aquality assurancemanual is used to

57

provide detailedinformation for theimprovement of eachinstitutional qualityassurance system.

The Ministry ofUniversity affairswill encourage eachinstitution toestablish its ownquality auditmechanism at boththe institution anddiscipline levels inorder to gain wideracceptance.

The Ministry ofUniversity affairssupports andencourages bothpublic and privatedepartments /institutions includingacademic orprofessional associations toparticipate in highereducational qualityassurance activities.

The Ministry ofUniversity Affairswill facilitateinstitutions to widelyand publiclydistribute their

The M

affairs has workshops anto implemenassurance. Auniversity learned newquality assessment, assurance, assurance, squality assurof them are iespecially frand Australiimprove the and academpublic univeformal assedeclining

information and the results of institutional quality assurance activities for society’s acknowledgement of higher educational standards. It also helps parents and students to decide on which educational institution to choose for study, including details of financial support for the institution’s administration.”

inistry of University organized several

d seminars since then t its policy on quality long the way, Thai

administrations have concepts such as auditing, quality

internal quality external quality

elf-assessment report, ance index, etc. Most mported from abroad, om United Kingdom a. The policy is to standard of teaching ic environment in

rsities. There was no ssment showing the quality of public

58

universities, but the policy indicated clearly what the problems in Thai higher education were. This was very unfortunate for all public universities as one could misunderstand that university education in Thailand had poor quality. In fact there is other reason for public universities to adopt fund in the manner that best quality can be assured. Evidence of quality can not be certified solely by the universities. It has to be done through the process of external evaluation.

To implement its policy, the Ministry has proposed several indicators of quality assurance. Finally, the indicators have been grouped into 9 aspects of higher education criteria as the following: *

- Mission, objectives, planning

- Teaching and learning - Student recreational

activities - Research - Social academic service - Preservation of arts and

culture - Administration - Budgeting - Quality assurance and

enhancement It should be noted that the

idea of having external assessmentis quite a radical change to allpublic universities which have longenjoyed the privilege of academicfreedom and self-regulated system.It needs a lot of effort and goodstrategy to have a smoothoperation. Above all, the policymust be cooperated by universitycommunity. Knowing the nature ofuniversity people, the ministry hasselected a good strategic approachleading to a successfulimplementation. But this is just thebeginning; there are a lot more tobe done. However, the Ministry hastaken a significant step inintroducing quality assurance tohigher education in Thailand. It canbe seen as a milestone in thehistory of Thai educational system.

There is one more reason

for the Thai public universities tobe cooperative under the newquality assurance schemeintroduced by the Ministry ofUniversity Affairs. The universitiesalways want to be free fromgovernment regulations on budgetand employment. As a governmentunit, the budgeting is controlled bythe Budget Bureau and theComptroller General’s Department.The university people always lookat these regulations as inefficientand an obstacle to good universityadministration (though they may

59

not agree on what should be a gooduniversity administration). Theywould prefer funding by blockgrant to the existing budgetingsystem. The government can give ablock grant to the universities onlywhen there is something to ensurethat the universities will speed thegrant to produce good qualityeducation. Therefore, to beautonomous, the universities mustbe under quality assurance systemwith external review. Under thenew law, the public universitieswill have to choose betweenbecoming an autonomousuniversity or staying under thenormal budgeting procedure. Atthis point, all of them are scheduledto be autonomous (with the blockgrant funding) by 2002. So far,there is no resistance from theuniversities to join the qualityassurance system. The Ministry hasset schedule for first round visit byexternal reviewers at thedepartment level.

There are three

components within the externalquality assurance. Firstly, theMinistry will send a team to auditthe internal quality assurancesystem. Secondly, the externalreviewers will assess the qualitybased on 9 aspects of highereducation criteria. Thirdly, arecognition (or accreditation) will

be made for the institution. Thepurposes of external control are:

- to approve the quality

index used internally. - to approve the system of

internal quality assurance. - to audit the effectiveness of

internal quality control. - To assess the quality and to

provide recommendationsfor improvement.

The external reviewing

team consisting of 3 - 5 persons,appointed by the Ministry ofUniversity Affairs, will visit thedepartment for 3 - 4 days. The teamwill report to the EducationalStandard Sub-Committee. Thereport will be available to thepublic. Any university that does notpass the assessment must showimprovement within 2 years, therecognition will last for 5 years forthose that pass the assessment.

At this moment the systemof quality assurance for publicuniversity in Thailand is not in acomplete cycle. Recently, about 6medical schools have been visitedto audit their internal qualityassurance systems. Thailand hasnot yet experienced the full processof quality assurance. No one reallyknows what will happen in thefuture when the new Accreditation

60

and Quality Assessment Officebegins its operation. For Thailandonly part of the lesson has beenlearned. 3.0 The Internal Quality

Assurance

The internal quality controlis seen by many as a vital part ofquality assurance. What can bedone at most by an externalreviewer is to reflect what theuniversity is, he can not make anychange in it. The strength of auniversity must come from its owninternal management. Poor qualityis caused by the production processnot from the outside examiner. Agood quality control by theuniversity itself can ensurestudents will graduate with goodquality as demanded in the labormarket.

Public universities havetheir own quality control for a longtime, but the system employed cannot produce supporting evidenceready for external review. Theyscreen the lectures, they alwayswant to recruit good students, andthese are part of quality control.There are several committees in auniversity working on how toimprove teaching, curriculum, orresearch. There is a committee totake care of the environment and

cleanliness on campus. These aresomehow related to qualityassurance, but when asked toproduce a report on how the qualityis controlled, most of us just do nothave such report on hand. Theevidence are scattering in allplaces, unorganized, and perhapsunwritten. The internal qualityassurance should help us to do agood documentation on what wehave done to maintain or toimprove the quality. It allows youto take a good look at yourself. Ifyou look good, you will havesupporting documents ready forexternal reviewers. If you look badyou will know what should be fixedup so that by the time the externalreviewers come you will be in goodshape.

The Ministry of University

Affairs has encouraged theuniversities to do the self-studyreport at the departmental level andat the institutional level. Guidelinesand handbooks have beendistributed, and workshops havebeen organized to make sure thatwe can produce the required self-study report. Basically, a self-studyreport will contain a) introduction,b) analysis of strength andweakness, c) summary, and d)appendix, which contains appointedquality assurance committees at thedepartmental level. The committee

61

may be responsible for planning foractivities related to qualityassurance such as necessarytrainings, workshops, disseminationof good practice, provision ofhandbook, etc. The committee willdecide on how to do the internalquality control, but no matter whatsystem is chosen, the Self-StudyReport must be prepared forexternal review. This report is aself evaluation by all internal units.It covers from objectives or targetsto final output, from professors tostudents, and from programs tosupporting facilities.

Many universities

expressed some disagreement withthe quality assurance at thebeginning. To be evaluated issomething that Thai universitylecturers are not used to. Ironicallymany of them have Westerneducational background whereclass evaluation is a normalpractice. This is the part whenuniversity administrators have tofind the best way to sell the idea ofquality assurance. It certainly cannot be rushed, and it should bemade clear that there is nonegative measure or penaltyinvolved in this process.

4.0 The Lessons Learned

Most of the public

5.0 Th

problemproblemqualitycycle. the qtransfeUniverof EdproblemseriousqualifieTraininexterna

universities are in the stage of setting up their internal quality assurance system; only a few have experienced the external audition. The lessons learned are mainly for internal auditing and assessment. The following lessons may not exactly happen in all universities, but should describe the general situation shared by public universities.

Lesson #1 The quality assurance needs administrative leadership. It would be difficult to start the issue from bottom up. It should be seen as a challenge to university society rather than a mandatory measure. Lesson #2 To be successful the quality assurance should gain participation from all groups of personnel in the organization. Once the system has been installed, the bottom up process should be encouraged. Lesson #3 The quality assurance will be very effective if there is a strong culture of a) quality first, and b) evaluation in the organization.

62

Lesson #4 There is a cost and time involved in setting up the quality assurance system. The university has to provide manpower and budget to support it. Another significant supporting factor is a good data base. Good quality assurance also requires resources.

Lesson #5 It should be made as clear as possible that the quality assurance is a long term commitment included in the university’s planning and monitoring. By doing so the quality assurance will become a routine job. e Problems

There is no report of major s up to this point. But some s are expected when the

assurance comes to its full Under the new legislation, uality assurance will be rred from the Ministry of sity Affairs to the Ministry ucation. Some transitional

s may arise. The more problem is the shortage of d external reviewers. g can be provided to train l reviewers but not all of

them will be respected by theuniversities. Thai universitycommunity is a small world, thosewho are in the same area oftenknow each other. There might besome conflict of interest. There is aquestion on standard and quality ofthe reviewing team. The standardof quality assurance also needs aquality control of externalassessment. How could this bedone effectively?

Another standard problemis what we do not have anycommon understanding on whatshould be a standard. Thesuggestion is to set a minimumstandard for a field of study, butone can imagine how hard it is toset a measurable standard inacademic world. It is possible for auniversity to do a survey onemployer’s satisfaction and use itfor quality improvement. But themarket demand will changeovertime. Academic quality can bedynamic, and so to set a standardwill be more difficult.

In sum, the quality

63

assurance for public universities inThailand has just begun. We do nothave many lessons of our own, andit is too early to conclude that wedo not have any problems. But wecan say that there is a breakthroughin Thai higher education, and thatwe have moved into the rightdirection. In the world ofcompetition, quality is always theanswer.

References: Ashworth A, Harvey R (1994)“Assessing Quality in Further andHigher Education.” HigherEducation Policy Ser. No. 24 ISBN1-85302-539-9. Parsons C ed. (1994) QualityImprovement in Education: CaseStudies in Schools, Colleges andUniversities. ISBN 1-85346-327-2. Rieley J B (1994) Total Quality Management in Higher Education. Diane Publishing ISBN 0-7881-1293-7.

Unit of Assessment for Accreditation

Antony Stella and A. Gnanam

ABSTRACT

One major aspect that needs careful attention is the choice of the “Unit”for assessment. Almost all countries have some mechanisms already in placewhich would assess the institution as a whole. This paper gives the descriptionof each of the units of assessment in which the choice seems to depend on manyconsiderations. The size of the national system of higher education as a whole,the specific purpose for which the assessment has been commissioned, thesignificance of the outcomes to the stakeholders, its viability, and above all, thefeasibility of such reviews are some of them. Keywords and Phrases: unit of assessment, institution, faculty, academicprogram

1.0 Introduction

An analysis of the current practices of the national accrediting agencies of different countries reveals a great deal of diversity. They vary in structure and function. Some of them are established and supported by their respective governments. Others are either independent or quasi-government bodies and, in a few countries, they are formed as an institutional consortium. These bodies may undertake only the accreditation on a two- point scale or carry out an elaborate assessment and grade the units or restrict themselves to the overall academic audit without grading. Likewise, many of them

6

may confine assessment to thereview of either teaching andlearning or the research activities ofthe unit or do both. Other aspects ofsuch variations are many, and,therefore, one cannot say which isthe best. However, for a given frameof reference, one can attempt toidentify the best model that is mostsuitable for the national context. 2.0 Institution as the Unit of

Assessment

While evolving andoperationalizing quality assurancemechanisms, we may come acrossmany issues, the relative level ofimportance which will fallsomewhere between the macro and

4

micro approaches. The overall sizeof the higher educational networkvaries with the countries. The totalnumber of students enrolled, andnumber of institutions - universitiesand colleges - involved are some ofthe factors that will determine thesize of the system.

It ranges from a singleUniversity (Mauritius) to hundredsof them as in USA, Russia andIndia. India which has the secondlargest network of higher educationsystem, has 245 universities andabout 11,000 colleges with morethan 8 million students enrolled.Obviously, for any country whichhas a large sized educationalsystem, choosing any unit smallerthan the institution, will have manypractical difficulties. If one choosesthe department of study or thediscipline as unit, in a country likeIndia with its large number ofuniversities and colleges, thenumber of units to be assessed willrun into a few hundreds orthousands, a stupendous andpractically difficult job to be donewithin the normal assessment cycleof 4 to 5 years. In a country likeUK, where there is a relativelysmaller number of largeinstitutions, the choice was madesome years ago to use the programas the unit of assessment and twocycles have been completed so far.

65

In a country like India where thereis a large number of smallerinstitutions, the institution happensto be the obvious choice. InCanada, education is a provincialresponsibility and there is no‘national level’ mechanism in placefor quality assurance. In fact, someof the provinces follow ‘qualityassurance’ procedures, quite similarto our ‘affiliating functions’. At theprovincial level, it is possible forthem to focus on programs. Even inNew Zealand and Hong Kong sincethe number of institutions to becovered is very low, program levelor aspect level assessment likeassessment of Teaching-LearningQuality Review could be possible.

Apart from the practicalaspects, the use of an institution asa unit of assessment has manyadvantages. The academic qualityis justifiably defined as, not thequality of individual teachers butthe collective impact of an overallacademic program designed anddelivered by the institution as awhole for providing the desiredknowledge, skills and competencies(QAA document). It is true, as it isindeed, then the assessmentstrategy should focus on thecollective impact. In that case, theinstitution will be the obviouschoice as a unit of assessmentbecause institutions are responsible

for the introduction of newprograms based on the currenttrends and changed expectationsand for their design and delivery ofthe course. Individuals docontribute but they cannot do sowithout the academic soundness ofthe institution. It is theresponsibility of the institutions toprovide the academic environmentthat helps in developing thecognitive and general skills that fallbeyond the realm of subjectspecialization and classroomteaching. Likewise, only theinstitution can facilitate the multi-disciplinary and inter-disciplinaryprograms through coordinationamong the various constituentdepartments of studies.

It is also the responsibilityof the institutional management toprovide the various infrastructureand learning resources such ascentral library, computer centers,residential halls, facilities for sportsand games, cultural activities,academic activities like seminars,debating etc. These significantlycontribute to self-learning,acquisition of self-confidence andleadership qualities. Institutions actas the training ground for severalskills such as communication,capacity to work in a team,citizenry and other expected orimplicit maturity among the

66

students. The ideal and richcorporate life of the manycampuses generally providesopportunities for students todevelop their inherent talents andinstitutions are empowered toprovide the needed rich ambiancein the campuses for this purpose. Infact, the institution as a whole isresponsible for providing all theimplicit and explicit provisions fordeveloping desired knowledge,values and skills among students.Consequently, only when theinstitution is assessed for itsmission, objectives, policies,principles, processes and variousinputs, one can get an insight intothe quality of education offered.

Another advantage of usingthe institution as a unit ofassessment is its direct usefulnessand relevance to the majorstakeholders, such as thegovernment which funds, theprospective students and theemployers who hire the graduates.The institution is the unit offunding by the government or theprivate foundations and Trusts allover the world. Though certainspecific programs of studies andmany research projects may befunded by different individualsponsors to a faculty or adepartment of studies, the lastingand sustainable funding -

maintenance or block grants - comes only to the institution from the state and not directly to any of the subsystems. Even the agencies that provide support to individuals insist on institutional commitment for accountability and in many cases the institution is the proper channel through which the transaction takes place.

Likewise the public consciousness of academic quality is built around the institution rather than any specific course or degree even though the faculty and the quality programs contribute to the image of the institution. After all, the departments form the backdrop for the institutional assessment and without them the assessment is impossible. The students and parents choose an institution for study mainly because of the standing, reputation and the tradition of the institution and not based on the quality of discipline-based academic program(s), when they complete their high school or graduation. Even the pubic philanthropy seems to favor the institution as a whole.

Although the situation may be different in the developed world, looking for the critical size of the unit is an important factor for undertaking assessment work. In

67

countries where undergraduate education is separated and offered in affiliated colleges, the size of the individual unit is generally reduced. Even the institution as a whole may find it difficult to qualify if a critical number of students on the roll is insisted upon.

Many state universities in India and in the Indian subcontinent, have a large number of small colleges affiliated to them which take care of the undergraduate education, while the universities provide only post graduate and research programs. The result of this bifurcation is that neither the colleges nor the affiliating universities normally have more than 1000 to 1500 students. The few unitary universities may be exceptions. In such cases, for the extent of efforts and the time involved for the assessment work, the outcome may not commensurate with the efforts. It is a futile task to take up a unit which is smaller than the institution as a whole viz. a university or a college.

There are also certain disadvantages in the choice of the institution as a unit of assessment. An overall assessment-based single grade for the whole institution may not distinguish between the good

•

•

•

•

and bad sub-units. At the surfacelevel, it may appear that even thenot so well performing groups areprotected under the institutionalumbrella. If this is the onlyreservation against institutionalassessment, the reporting can besuitably adapted to includedepartmental evaluation on specificpre-determined aspects. A morerealistic and practical difficulty willbe in assessing institutions that arelarge, and offer hundreds ofprogram options with manythousands of students in one ormore campuses. 3.0 Academic Program as Unit

of Assessment

The academic programs asunit of assessment will have theadvantage of being well focusedand will provide opportunities tolook into the micro details. It canprovide the right kind of inputs todiscriminating beneficiaries like theemployer and students. It will nothave the difficulties one wouldencounter in assessing theinstitution as a whole. However,the disadvantages of choosing thisas a unit of assessment are:

• At the national level, thenumber of such unitsoffering the variousprograms will be too largeand consequently one

68

would end up with a largenumber of such units toassess in a 5 or 7 pointscale.

Normally any program at agiven institution may notinvolve more than a fewfaculty and a few studentsand consequently the unitsize will be too small forsuch a big effort. Compared to the institutionas whole, the compositionand character of the groupoffering a program mayundergo frequent changes.If a faculty or two moveout of the institution, thequality of the offerings willsuffer and hence, theoutcomes of the assessmentmay not be tenable for anyextended time.

The programs of interest tothe students and theemployers at one time maynot be so in a few years. Itis likely that some suchprograms will not even bethere after one or two yearcycles, thus making theassessment effort futile.

Even though the programofferings are done by thedepartment/school, most of

•

4.0 FaA

where departmsuch aLanguaetc. homogsize to

the infrastructural facilitiessuch as library, computercenters and other learningresources may be sharedwith others and theirquality is determined bythe central governancestructure rather than by thedepartments of studies.

Many programs that areinter-disciplinary in natureand offered by more thanone department with thevarying responsibilitiesmay not pose a problemwhen everything is inorder. If there happens tobe a bottleneck at anyjuncture, identifying thecause and takingappropriate valuejudgments become difficultand this will pose a seriousproblem in objectiveassessment.

culty as the Unit of ssessment Apparently, the faculty,

related discipline-basedents of studies are grouped

s Faculty of Arts, Science,ges, Medicine, Engineeringconstitute a relatively

eneous group with an ideal be used as a unit of

69

assessment. Unfortunately such aunit is practically unknown to thestakeholders and therefore may notbe useful. It is neither a unit offunding by the public nor a unit ofacademic offering. Therefore,except for its practical advantage asa manageable sub-unit of thesystem for assessment, it can offervery little to the stakeholders likethe funding agencies, employers orto the students.

An entirely different

approach in selecting the unit ofassessment will be, to take up thedifferent academic aspects like theteaching and learning process orthe research component of eitherthe institution or the department ofstudies. In England, the HigherEducation Funding Council(HEFCE) directly assesses thequality of the research in thedepartments or schools of studiesand rates them for differentialfunding. This process is done by anin-house group called RAE(Research Assessment Exercise)unit with well laid out protocolsand uses the outcomes to fund theresearch activities. The QualityAssurance Agency (QAA) which isan autonomous body partiallyfinanced by the HEFCE undertakesthe assessment work of theacademic aspects of the programsof studies offered by all the

institutions. In Canada, Commission for Evaluation of College Education (CEEC) of Quebec attempted a different approach to begin with and now it is moving towards ‘institutional review’. CEEC once carried out two types of assessment - the general component of the college studies and the specific component of the college programs - for the select programs in all the 47 colleges of the Quebec province. After involving a cross section of faculty in all the colleges, now CEEC is promoting institutional assessment.

While the discipline-wise

or aspect-wise assessment provides a horizontal review of the select aspect across institutions, the institutional review yields the top down vertical assessment. While the former may be good for well-developed institutions in the not so advanced nations.

5.0 Conclusion

To sum up, it is important that we choose the unit of assessment in the national context. One has to look at the various

70

factors such as the size of higher education system as a whole, feasibility of completing the assessment cycle within a pre-set assessment cycle, viability in terms of having a minimal critical size for assessment, and the sustainability of the unit. References: Astin, A.W. (1991). Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education. New York: Macmillan. Craft, A. (1992). “Quality assurance in higher education.”

Hong Kong, 1991. Bristol, PA: The Falmer Press, Taylor and Francis, Inc.

Proceedings of an International Conference,

Curry, W., & Hager, E. (1987). “Assessing General Education: Trenton State College.” in D. F. Halpern (Ed.).

No. 59 (pp. 57-65). San Francisco, CA: Jossey-Bass.

Student Outcomes Assessment: What Institutions Stand to Gain: New Directions for Higher Education

issfaCC KF

1.0 Introduction

The sheer number of stateuniversities and colleges in thePhilippines demands a morerationale approach to budgeting.For FY 2002, the appropriation forthese 110 state institutions ofhigher learning amounted toP12.6.B. Such huge investmentoutlay invites public pressuredemanding for accountability aspart of the institutions’ qualityassurance system.

71

The main rationale forrationalizing the budgeting processof state universities and colleges ison grounds of efficiency. Certaininequities in the allocation of stateresources point to the inefficientoperation of the institution. Twocolleges that offer the sameprograms and have similarconditions obtaining in themreceive differentiated fundinglevels. The variances that areobserved in these institutions areoften so wide that the budgetsappropriated defy logical

A Quality-Based Normative Financing for State Higher Education Institutions in the Philippines

Roberto N. Padua

CHED Representative For

Technical Working Group on Normative Financing

ABSTRACT

The annual government subsidy given to state colleges and universities often characterized as a negotiated funding scheme. Historically, such

ubsidies have been incremental in nature responsive to economic inflation ctors but not necessarily to quality consideration. A recent proposal of the ommission on Higher Education, following the pattern of the U. K.’s Funding ouncil, considers the link between funding and quality in higher education.

eyword and Phrases: normative financing, state universities and colleges, unding Council, block grant.

inefficioveralleducatithe samSUCs of SUCindividshrink.feel theless dethat ult

Commisince 1consideIn FY maintenexplainas: a)examinCenter Develonationaexaminthe peexplainbe incrThe plobjectithrough

2.0

explanation. Such inequalities and inefficiencies are to be expected in a situation where funding is negotiated.

In a negotiated funding

model, the state schools prepare an annual budget proposal. The budget proposals are submitted to the budget agency, the Department of Budget and Management, which scrutinizes the proposals based on an approved budget ceiling. The DBM-received proposals are then incorporated in the President’s proposed National Expenditure Program (NEP) which, in turn, is submitted to Congress for deliberation. Budget negotiations begin at the DBM level when political figures try to intercede in behalf of the concerned SUC’s. The negotiations continue until the budget process culminates in the signing of the annual General Appropriations Act (GAA).

The “winners” in this

negotiated funding process are the SUCs with strong political backing while the “losers” are those without political clout. Consequently, the resulting budgetary appropriations do not reflect the actual needs of the SUCs, are insensitive to quality parameters and often, defy logical criteria for funding.

(80%)

72

Budgetary inequities and encies translate into poor performance of the higher on system. For instance, for e total national budget for

and with increasing number s, the per capita share of the ual institutions generally The more established SUCs budgetary cuts less but the veloped ones are the ones imately suffer the most. Realizing this situation, the

ssion on Higher Education, 999, began inputting quality rations in the SUC budgets. 2002, exactly 20% of the ance budget of SUCs are

ed by such quality measures passing rate in board ations, b) identification as

of Excellence or pment, and c) above the l passing rate for licensure ations. It is envisioned that rcentage of SUC budgets ed by quality measures will eased to 100% by FY 2004. atform through which this ve can be achieved is normative financing.

Analysis of SUC Budgets: FY 2002

More than eighty percent of the SUC budgets are

dedicated to support salaries, wagesand benefits of personnel (PS cost),15% are allocated for maintenanceand operation (MOOE) and 5% forcapital or infrastructure outlay(CO). This simple breakdown ofSUC budgets shows that theoperations of these state schools arejeopardized. For instance, the 15%allocation for MOOE are almostconsumed by the mandatoryexpenditure on light and water,gasoline and lubricants, travel, andnecessary supplies and materials.The subsidy provided by thenational government to SUCs isalmost devoted entirely to cover thesalaries and wages of SUCpersonnel. The FY 2002 situation isa replica of the SUC budgets overthe last decade.

The inequities are observed

by analyzing the individual SUCbudgets. In the National CapitalRegion (NCR), seven (7) SUCsshare an MOOE budget of overP800M. The largest, in terms ofenrollment, is the PolytechnicUniversity of the Philippines(enrollment = 65,000), followed bythe University of the PhilippinesSystem (enrolment = 56,000), yetmore than 75% of the MOOEbudget is allocated for the UPsystem with the remaining 25%shared by the six other SUCs. Thelopsided budgetary allocation

73

becomes even more pronounced inthe Capital Outlay (CO)appropriations: more than 85% ofthe CO budget for SUCs areallocated for the UP system theremaining 15% are shared by theother 109 state schools.

Perhaps these inequities

can be explained partly by qualityconsiderations. However, sinceonly 20% of the MOOE budgets inFY 2002 are explained by qualityparameters, the remarkablebudgetary differentiation betweenSUC cannot be fully explained.

There are other features of

the SUC budget for FY 2002 thatdeserve further scrutiny. The pro-poor and Mindanao-focus of thepresent administration’s agenda arehardly reflected in theAppropriations Act for SUCs. Forinstance, CARAGA, the poorestamong all the regions in thecountry and host to two (2) of themost depressed provinces in thePhilippines, received the lowestSUC appropriations. There are four(4) state colleges in the region, onefor each province. Of this number,three (3) are newly-establishedSUCs whose operating standardsare way below what is expected ofhigher education institutions. In allthree new SUCs, there are nolibrary buildings (except for

makeshift libraries out of oldclassrooms), the laboratoryfacilities are out dated or non-existent; faculty expertise is notcongruent to higher educationstandards. All these should havebeen concrete considerations if thebudgets were made sensitive toquality and development concerns.

3.0 Normative Financing for

SUCs

In 2001, the AsianDevelopment Bank (ADB)provided Technical Assistance tothe Commission on HigherEducation to study and reform theSUC budgeting system. The ADBconsultants with the experts fromthe Commission submitted a reportto CHED recommending the use ofa normative financing model forthe state universities and collegesto address the issues of inequityand inefficiency in the use ofgovernment resources.

The use of a normative

financing model (NFM) for SUCsis anchored on the premise that theCommission on Higher Education(CHED) will take the role of aHigher Education Funding Council(HEFC). This premise is based onthe fact that among all agencies ofthe national government, it isCHED that directly deals with the

74

SUCs. CHED is supposed todevelop and implement a fundingformula that will be more equitableand will lead to greater internal andexternal efficiency among theSUCs.

The funding formula takes

into account parameters related to:a) quality, b) typology, and c)demand for higher educationcourses. In particular, the formulais given by:

Number of student Places

= a x Quality + b x Typology + c xDemand

(1) … Budget for the ith

program = No. of Student Places xCost Per Student Per Program

where a, b, c, are positive

constants, a + b + c = 1. The parameters Q (quality),

D (demand) and T (typology) areexplained in detail:

Quality Measurement of quality (of

SUC programs) is a difficult, albeit,tedious exercise. At best, qualitycan be inferred from somesurrogate measures, the crudest ofwhich is to say “1” if quality is

The

ticklish onebe classifieof thought classified Science andschools Universities

If t

SUCs is ththen all SUcolleges atherefore loon the basisThus,

program is w

Typ

The

positive wapplied to tT. The dweights sho

Cost Per S

Ideaper programinvestment (1) student level compa

perceived “0”, otherwise. Forexample:

Quality = 1, if program is

accredited at least level II = 0, else or Quality = 1, if passing

percentage in board exam is abovenational average

= 0, else Whatever measure is

adopted, CHED needs to decideand make public its own quality-assessment procedure.

Demand Demand is, theoretically,

measured from the point of view ofthe end-users of the highereducation graduates. Thus, demandneeds to be based on a labor marketinformation system (LMIS). In themeantime that such an informationsystem is not available, CHED mayopt to adopt a ranking of highereducation courses based on priorityneeds. Thus,

1, if program is within CHED’s

priority list Demand =

0, else

75

Typology

issue on typology is a. How should an SUCd? A prevailing schoolavers that SUCs can be

into: Agricultural, Polytechnics, Normalor Comprehensive

.

he basis for classifyingeir respective Charters,Cs are comprehensive

nd universities. It isgical to classify SUCs of their official names.

1, ifithin an SUC typology

ology = 0, else

Constants

constants a,b, and c areeights (percentages)

he parameters Q, D andetermination of suchuld come from CHED.

tudent Per Program

lly, the cost per student should account for theneeded to educate onein a given program at arable with international

an indrequirerole asof SUoppositRepubl

which amendmdoubtfuits powSUCs. will refigure changeinitiativ

Issue SUCs

applicaassumpis eveneven, alevel oA strawipe oexistinggenerattherefoNF, if a

Issue 3Not?

standards. This can be a staggeringfigure by Philippine standards. Forexample, an Engineering student inHongkong pays at least $6,000 perannum (PhP 300,000); in theUnited States the same student pays$7,500 per annum (PhP500,000).The cost per student if thePhilippine varies from a low ofP9,500 per annum to a high ofP97,500 per annum, both figuresbeing way below internationalrates.

The idea of internationalcomparability of costs needs to beabandoned. The CHED may haveto compromise costs observed inthe public school system and theprivate school system. 4.0 Issues, Problems and

Prospects of NormativeFinancing Normative financing does

not purport to cure all the ills ofpublic higher education funding.On the contrary, it is recognizedthat the implementation of such afinancing scheme in the Philippinesetting is replete with pitfalls anddifficulties. Some of the issues thatare raised relative to this financingscheme are:

Issue 1. Legislative and ExecutiveCompatibility of NF

76

The new role of CHED asependent funding councils that CHED relinquishes its Chairperson of the BoardsCs. This is in direct

ion to the provisions ofic Act 8292. Apart from this difficulty

can be remedied through anent of the law, it is also

l that Congress will give uper relative to the budgets ofA strict application of NF

sult in an objective budgetfor an SUC which cannot bed by a Congressionale.

2. Comparability of the

A major difficulty in thetion of NF is the basictions that the “playing field”. The playing field is nots can be gleaned from thef development of the SUCs.ight application of NF willut more than 50% of the SUCs with no capability toe their own income. It isre important to implementt all, in phases.

. Should NF Include PS or

cost pShouldthe PShow wthe gcannotother hthen Nof the S

avers ta “shathat thegiven the bethe basthe wo 5.0 C

SUC b

How does one compute theer student per program? this computation include or not? If PS is included,ill the government deal withovernment officials that

be funded by NF? On theand, if PS is not includedF will rationalize only 15%UC budgets.

One school of thoughthat PS should be included asdow budget”. This means full subsidy to PS normally

to SUCs will continue, butnchmark NF results will beis for the gradual attrition ofrk force.

oncluding Statement The need to rationalize the

udgets in the Philippines is a

77

concrete and urgent concern of thegovernment. Normative financingprovides an avenue for suchreforms in resource allocation forhigher education. Perhaps, the fulleffect of NF will not be felt in theimmediate future but will certainlybe appreciated one or two decadeslater.

References:

Preddy, J. et al. “ADB-PPTA onNormative Financing”. (ATechnical Report Submitted toCHED, 2001).

Padua, R. et al. “Legislative andExecutive Compatibility ofNormative Financing ”. (A Reportof the TWG on NormativeFinancing Submitted to CHED,2002).

instrumand aassessmodel Keywoquality

1.0 Introduction

Since the last three decadesof the past century, higher educationis confronted with the rise of aquality culture. Quality, qualityassurance and quality assessment, isa booming business. Highereducation is not only confronted withthe requirement to assure its qualitybut also show its quality to theoutside world in the framework ofaccountability.

Of course, higher education

did look around to see what could belearned from industry and if it waspossible to apply models, like, theEFQM. Already at an early stage it

78

became clear that, for example, theEFQM model could be difficult toapply in higher education withoutsubstantial changes. The reasons are:

• A Higher Education Institute

is not a firm; it is not acookie factory producinggraduates.

• Talking about HigherEducation, it is not clearwhat the product is. Is it thegraduates? Or is the productthat provided courses?

Also the client is not clear.

Who is the client ofHigher Education? The

Towards a Quality Model for Higher Education

A. I. Vroeijenstijn Senior Consultant

Quality Assurance and Quality Assessment Association of Universities in the Netherlands (VSNU)

ABSTRACT

The quality movement in Higher Education caused the development of ents for internal quality management and external quality assessment

ccreditation. Models are always helpful instruments in auditing or ing the institutions and disciplines. The paper aims to introduce quality s that would help an institution in quality management.

rds and phrases: quality assurance, EFQM-model, SWOT-analyses, models

2.0 Q

E

centuryand qutopics ihad alre100 yquality in Weconsidethe goHigher This chautonomorientatorientatforced internalexternafar, quaresponsnow taccounto seeoutsidealso manage

Educati

student? The employer?The Minister? The societyat large? Higher Education is aprofessional organizationand not a hierarchical one.A Higher EducationInstitute depends on thequality of experts, noteasily to lead or to steer.

The EFQM-model is moreof a model aiming atorganization developmentand organizationimprovement and is notaiming at the improvementof quality per se.Therefore, the EFQM-model might be useful toanalyze organizationmanagement, but not toassess the quality.

The concept of quality ineducation is much morecomplicated than theconcept of quality inindustry.

Quality control in industryhas not the specific featureof accountability, like it isintroduced in HigherEducation. This means thatmodels for qualitymanagement and self-

79

assessment should fit withthe methods andrequirements set byagencies for externalquality assessment.

uality Assurance in Higherducation

Since the 80s of the past, quality, quality assuranceality assessment became hotn Higher Education. The USady a tradition of more than

ears of accreditation andassessment, but for examplestern Europe quality wasred to be present becausevernments were steeringEducation very centrally.anged in an area of morey, more consumer

ion, more marketion. Higher Education wasto take care of its quality byly assuring the quality andlly showing the quality. Solity has been the individualibility of every professor,he institutions are heldtable for it. It is interesting that the pressure from, asking for accountabilitycaused internal qualityment.

Interestingly, Higheron did choose an approach

that differs from the approach in industry. Industry has already a long tradition in quality control. One of the gurus, Demig, developed quality

Figure 1. European Quality Model

Enables

People People Results

Strategy Processes Policy & Key Performance

Results

Customer Results

Results Partnerships &

Resources Society

Innovation and Learning

Leadership

Results

models and ideas for Total Quality Management. In the ISO-9000 series, both a definition for quality has been given and criteria for quality control. Industry has developed the ISO-certificate, recognized all over the world and telling the client that the firm is meeting certain assessed criteria for quality control. There are

80

several quality models in use.Maybe the best known is theEFQM-model (E= EuropeanFoundation for Quality

Management). (see figure 1). In the past decades Higher

Education developed its ownworking process. Nowadays, manycountries in the world have anagency for accreditation and/orexternal assessment of the HigherEducation institutions. This mightbe on institutional level or at

generafollowi

•

•

program/discipline or subject arealevel. Of course, all agencies areworking in their own nationalcontext. There are differences butalso a lot of similarities in theapproach. In general there are verylittle differences in answering thequestion how it is done. Differenceshave much more to do with thewhy. In general the qualityassurance model in HigherEducation looks like the model,given in figure 2.

INTERNAL EXTERNAL

SELF-ANALYSIS

QUALITY PLAN

EXTERNAL ASSESSMENT

Figure 2. The Quality Assura

QUALITY ASSURAN

81

The characteristics of thel EQA-model are theng:

It is based on self-analysisand external assessment bypeers.

The external assessment isorganized by anindependent agency. If theagency is not independent,at least the committees can

IMPROVEMENT

ACCREDITATION

ACCOUNTABILITY

nce System

CE

•

•

•

apply EducattemptaapplicaOne ofthe faguideliassessmof aspeaccounthe disprovidaspectsattractithat it aspectsget mo

act independently.

It has both internalfunctions (accountability,quality label,accreditation).

There is generally spoken,a public report of theexternal assessment

It looks not only at qualityprocedures, but tries tocatch quality itself bylooking to input quality,process quality and outputquality.

As said, it is difficult tothe EFQM-model in Higherion, although we see a lot oftions to make the modelble to Higher Education. the main reasons for this isct that all manuals andnes in use for self-ent are only providing a list

cts or topics to be taken intot, analyzing the institution orcipline. The manuals do note a clear relation between the and the topics. Theon of the EFQM-model isoffers a relation between the and offers the possibility tore grip on the reality.

This had been one of the

82

reasons to start to think about aQuality model in HigherEducation. Another reason hasbeen that it is necessary to preventboth the internal and externalassessment becoming a de-fragmented assessment of someaspects or topics without takinginto account the correlationbetween the different factors.There is a need for a model forhaving an overall view on qualityinstead of a scattered one.

Another important reason

is the internationalization ofHigher Education and the risingneed of recognition ofqualifications and degrees. A basiccondition for mutual recognition ofdegrees is the need to assess thequality on a generally acceptedapproach. There is no need tomake the assessments uniform.However, it is important that atleast the same criteria have beentaken into account. 3.0 A Quality Model for Higher

Education

As already said, lookingaround in the world, one will find alot of guidelines, manuals andprotocols for self-analysis and self-assessment. They all have a lot incommon. They all try to describethe most important quality criteria,

quality indicators and qualityaspects. However, what is missingis a model, in which aspects are allcorrelated, like the quality modelused in other sectors of society.For example, the EFMQ-model(see figure1) helps to structure theself-analysis of a company andhelps to discover strengths andweaknesses. Also for HigherEducation, there is a need for sucha model for SWOT-analysis. Weneed to analyze our Strengths andWeaknesses, to look atOpportunities and to see what arethe Threats.

In most cases, the self

analysis or self-assessment in

INSTITUTIONA

ASSESSMENT CO

RESEARCH EDUCAT

Product Quality

Figure 3. Correlation

83

Higher Education is connected withan external assessment. This meansthat in some cases, the self-assessment is only seen asproviding information for theexternal committee. This is a pity,because a self-analysis might be apowerful instrument to assure thequality. In fact, an HE-instituteshould adopt the self-analysisregardless of a formal, organizedexternal quality assessment.

Just as a model may help

an institution for a good self-analysis, it may also help theexternal review committees toanalyze the quality.

L AUDIT

RE ACTIVITIES

ION SERVICE

Process Quality

Between the Models

To analyze the quality of aninstitution, we have to make adistinction between

ourselv

•

output process input Realized mission

graduates

management study

programs Mission statement policy

Achieved goals goals

& aims

scientific

production

research projects

staff

expected outcomes students

Achieved outcomes

community services

funding services Satisfaction

stakeholders

facilities

Figure 4. Quality Model for the Analysis of the Organization

•

the analysis of the organization (might be the institution as a whole, a faculty, department or school)

84

community services.

In this paper we restrictes to the analysis of the

organization and the core activity ofteaching/learning.

In figure 4, a model for theinstitutional analysis has been given.Figure 5 is a model for the analysisof program quality.

4.0 A Quality Model for

Institutional Analysis

4 may as a worganizdepartmfollowi Mission

generaldefinitiEducatseen asdimensof the ideas a

define three mtalking

•

•

•

qualityanalysiformulformulformulthe stanWithou

The model given in figurebe applied to the institutionhole, but also at a lowerational level, like, a School,ent or faculty. In the

ng the model is clarified.

Statement

Nowadays, there is a agreement that an objectiveon of quality in Higherion does not exist. Quality is context-bound and as multi-ional. Quality is in the eyesbeholder who has his or herbout quality.

Although it is difficult toquality, there are in factain questions, when we are

about quality.

Are we doing the rightthings? Are we doing the rightthings in the right way? Do we achieve what we areclaiming to achieve?

This means that every assessment and self-s have to start looking at theated mission statement, theated goals and aims and theated expected outcomes (=dards set by the institution).t a clear picture about why

85

one is doing what one is doing,each assessment of quality isimpossible. Is the mission clearlyformulated and well known toeverybody? Is the missionstatement operationalized in cleargoals and aims? Input

The second group of topicsto be analyzed has to do with input.They may be seen as the boundaryconditions to achieve out goals. Itconcerns:

• Management

An analysis isneeded of the managementof the institution: role ofthe management; top downmanagement or bottom up?Analysis of the personnelmanagement and financialmanagement. The leadingrole of the management.

• Policy

How are missionstatement, goals and aimstranslated in a policy? Isthe policy plan the basis forstrategic management?How is the feedback on thepolicy plan organized?How is the involvement of

•

•

•

•

Proces

the different levels informulating the policy? Staff

Quality of aninstitution will be decidedby the quality of staff. Amore in-depth goinganalysis of the staff willtake place during theanalysis of the corebusiness. The analysis hereconcerns the qualificationsof the staff in general (e.g.number of Ph.D.), thecompetencies of the staff tocover the programs offered.Are there problems withstaff concerning aging ofstaff; competition with theoutside world to attractgood staff members?

Students

The qualitydepends too on the qualityof the entering students. Isthere any selection? How isit to be done? What do we

86

think about the startinglevel of the students? Funding

Talking aboutquality, of course fundingis important. Too often,one forgets that everyquality has its price. Thismeans that the formulatedgoals and aims should beachievable in the givenconstraints of funding. Facilities

One has to analyzeif the available facilitiessupport the mission. Is theequipment for teaching andlearning adequate? Lecturehalls, small workingrooms? Computerfacilities? Are theequipment for the researchactivities sufficient?

s

The cells under the heading

G O A L S & A I M S

program contents

organization

Didactic concept

Curriculum design

assessment

preconditions

students

staff

management

funding

facilities

Internal Quality

Assurance

output

Achieved Standards

Pass rate Drop out

Graduation time

Cost per student

feedback

Opinion students

Opinion Alumni

Opinion labor

market

Opinion society

Opinion staff

Figure 5. Quality Model for the Analysis of Programs

‘process’ contain the core activities of the institutions. Information for the SWOT-analysis of the educational activities, the research activities and community service, should be based on the SWOT-analysis carried out by the departments. The model for an analysis of the teaching/learning activities is given in figure 5. For the institutional analysis at central level or faculty level, one will summarize the strengths and

87

weaknesses found by the analysisof the core activities. One shouldlook at how each educationalprogram contributes to the statedmission. How does the researchcontribute to realize the mission?And what to think about thecommunity service. What is theoverall quality of the corebusiness? What fields needstrengthening? What may beclosed?

figure 5the teaeducatimay undergpost-gr

assessmactivitian analthe pobjectitake inof all requireformulof our clear stfrom tthe poutcomattitudethe proand aimlike:

•

•

•

•

5.0 The Quality Model for theEducational Activities

Output

What has been said aboutthe process is also applicable to theoutput. The departmental analysisshould provide information andshow strengths and weaknesses.Here an overall analysis will bemade because one has an overallpicture of the faculty and even ofthe whole institution Realized Mission

This last column in fact isthe most important one, talkingabout the analysis of our institution.Do we indeed realize what we haveexpressed as our mission, our goalsand our aims? How do we knowthat we have achieved what hasbeen formulated as desirableto achieve? And of course, just asin the industrial models we have tolook at the satisfaction of theclients. However, it is not quiteclear who our clients are.Therefore, we have to look for thesatisfaction of all stakeholders:students and parents, employers,academic community, government,society at large. How do we knowhow they think about ourperformance.

88

The model showed in is useful for the analysis ofching/learning process (theonal activities). The model

be applied to theraduate, graduate, as well asaduate programs.

Also, the model for theent of the educational

es starts at the left side withysis of the goals and aims ofrogram. Formulating theves of a program, one has toto account the requirements

stakeholders, weight thements and balance them inating the expected outcomesprogram. There should be aatement of what is expectedhe graduate after finishingrogram. Those expectedes (knowledge skills and) are the standards set forgram. Analyzing the goalss one has to ask questions

Is there clearly formulatedpolicy on what theexpected outcomes of theprogram are? What are you expectingfrom the graduates? How did youoperationalize the expectedoutcomes? How do we take into

The Pr

have to • Th• Th• Th• Cu• Th

goals aprogramor not; course general

importadeliverbehindcurricuorganizIs the accounis the are innprogram

reflecti(standaprogramthe staattentio

account the requirements of the stakeholders?

ogram Analyzing our program, we

look at several aspects:

e content e organization e didactic concept rriculum design e assessment/examinations

One has to ask how the nd aims are translated into a

; if the program is coherent and the contribution of each to the achievement of the mission of the institution.

Not only the content is nt. Also the way of

ing the didactic concept the organization of the lum is important. Is the ation structure satisfactory? didactic concept taking into t new methodologies? How curriculum designed? How ovations introduced in the s?

Having goals and aims ng the expected outcomes rds) and having designed a , which aims at achieving

ndards, one needs to pay n, too, to the way one can

89

assess the achievement of the students. One has to analyze the procedures and organization on the one hand. On the other hand, one has to look at the level of the examinations and put the question, if the examinations indeed reflect the content of the program.

Boundary Conditions (Pre-Conditions)

Although the standards set for the program and the way of delivering it will decide the quality, it is not enough to look at the program only. One has to analyze the environment, too. One has to look at the constraints and the boundary conditions. Students

One has to analyze the student population; the way the students are selected; and the way students are followed during their study. How is the student counseling done? Staff

The quality of a program depends strongly on the quality of staff; the competencies and the way staff members are co-operating. Facilities

be useSWOTinternaconnecorganiznot. Done mprogramdepartmor on it is analysianalysi

followi

•

•

facilitieshouldintrodulearnincompuwork. Output

self-assof tHowevpuddinoutcomachieveHow mstandarcost? H Feedba

do witof theopinionprogramofferedabout topinionlabor mSociety 6.0 T

IA

It will be clear that thes to deliver the program

be satisfactory. Thection of Computer-basedg without sufficientter equipment does not

The starting point for theessment is the formulation

he expected outcomes.er, the proof is in eating theg: What about the achievedes? Do our graduates the standards set indeed?any students did achieve theds? In what time and at whatow high is the drop out?

ck From Stakeholders

The last column has also toh feedback and satisfaction stakeholders. What is the of the students about the offered and the way it is

? How do the alumni thinkheir education? What is the of the employers and thearket about our graduates?

-at-large?

he Quality Model asnstruments for Self-ssessment

90

The described model mayd for a self-analysis or a-analysis, whether thel assessment is directlyted with an outside-ed external assessment orepending on the situation,ay start with an analysis at

level only, onental level, on faculty level

institutional level. However,clear that an institutionals has to be based on thes of the core activities.

To use the model, theng steps have to be taken:

Formulate for each cell ineach column questionsrelated to the topic. Onemay use existing manualsfor self assessment forformulating adequatequestions, fitting the needs Look at each question andgive:

• a clear description ofthe current situation

• a critical analysis andreflection on thecurrent situation

• the way you think the

1. The instituformulated

2. The goals purpose to

3. The goals in consultastakeholde

4. The goals

5. Institutionadecision mthe goals.

6. The institugoals regu

7. The goals institution.

8. The goals measurabobjectives

9. The goalstranslated

• Be

seonobexto

•

problem may be solved

fore one starts with the lf-analysis, one has to set e’s own criteria because jective criteria do not ist. For example, one has fix the pass rate and the

1 2 3 4 5 6 7 8 9 10

tion has clearly goals.

express clearly the achieve.

have been formulated ncy with all rs?

are well known to all.

l planning and aking are guided by

tion reconsiders the larly.

show the profile of the

are translated in le goals and .

and objectives are in clear policy plan.

Figure 6. Example of a Checklist to be Used

drop out rate that might be seen as satisfactory. Based on benchmarking one may have set the standards for the Bachelor or Master Degree. Of course, one has

91 •

to take into account criteriaor standards laid down inlaw or regulations byprofessional bodies.

Design a checklist basedon the questions. Afterdescription and analysis,

one will mark each topicwith a mark between 1 and10, where 6 meanssatisfactory, below 5 isunsatisfactory and 8 andmore is good or excellent. Calculate for each cell in

the checoncernthe inst

followeassessmbasis quality years tmodel othe proall leve

assessmit. If onin the ran hassessmafraid. not disnot knpossibiimprov

each column the average. Do the same for each column and fill the marks in the cells and columns. By doing so, one sees at a glance the strengths and weaknesses of the programs, the department, the faculty or the institution as a whole.

In figure 6 an example of cklist to be used is given. It s the column “goals” for itutional assessment.

If the self-assessment is not d by an external ent, one still has a firm

for the formulation of a improvement plan for the

o come. You may use the r parts of it again to control

gress of the quality plan at ls in the institution. In case of an external ent, one must be ready for e has done the self-analysis ight way, and if it has been onest and open self-ent, one does not have to be

The external committee will cover weaknesses one does ow about. One has a

lity to start already with ements before the

92

committee arrives. One has also thepossibility to show and emphasizethe strengths. If the externalassessment also will be done in anunconstructive way and not only isseen as inspection, one will getvaluable information andsuggestions for keeping the qualitywhere necessary enhancement ofthe quality is needed.

7.0 The Quality Model as

Instrument for theExternal Audit or Review

It will be clear that the

given model not only can be usedfor the self-assessment, but also forthe external assessment by theexternal experts in the reviewcommittees. Looking at thedevelopments in the field of qualityassurance, it will become more andmore important that bothinstitutions and disciplines will beassessed in a comparable way.

Although it is not possible

to define quality in an objectiveway and although assessors alwayshave to take into account thespecific context, it is important toknow that institutions anddisciplines are being audited orassessed, using a model, containingthe most important aspects. The use of the model might help to promote(mutual) recognition of the external

quality assessment agencies and by doing so to promote transparency of the Higher Education system and its core activities.

93

References: Elam L C (1991) “Values, Leadership and Quality: The Administration of Higher Education.” The David D. Henry Lectures University of Illinois Press ISBN 0-252-01893-1. Kells H R (1992) “Self-Regulation in Higher Education: A Multi-National Perspective on Collaborative Systems of Quality Assurance and

94

Control.” Higher Education Policy Ser. No. 15 Taylor & Francis ISBN 1-85302-528-3. Queeney D S (1995) “Assessing Needs in Continuing Education: an Essential Tool for Quality Improvement.” Higher & Adult Education Ser. Jossey-Bass ISBN 0-7879-0059-1.

Institutional Members

AbrTec

Akl Bas Bat Ben Bic Buk Bul Cag Cam Cam Cam

Col Car

Col Cat Cav Ceb Ceb

Tec Cen Cen Cen Cen

Col Cot

Scie Dav

Scie Don

Art Don

Uni Dr.

Stat Eas Eul

of S Ifug

and

Iloc Iloi Isab Kal Lag Ley Ley Ley Ma Ma Min Min Min

and Mis

Agr Mo

Col Nav Neg Nor

Col Nor

Scie Nue

Tec Nue

Tec Nue

Col Occ Pala Palo Pam Pan Pan Par Phi Phi Phi Phi

a State Institute of Science and hnology an State College of Agriculture ilan State College angas State University guet State University ol University idnon State College acan State University ayan State University arines Norte State College arines Sur Polytechnic College arines Sur State Agricultural

lege los A. Hilado Memorial State lege anduanes State College ite State University u Normal University u State College of Science and hnology tral Luzon State University tral Mindanao University tral Visayas Polytechnic College tral Visayas State Polytechnic lege abato Foundation College of nce and Technology ao Oriental State College of nce and Technology Honorio N. Ventura College of

s and Trades Mariano Marcos Memorial State versity Emilio B. Espinosa, Sr. Memorial e College of Agriculture tern Samar State College ogio “Amang” Rodriguez Institute cience and Technology ao State College of Agriculture Forestry

os Sur Polytechnic State College lo State College of Fisheries ela State University inga-Apayao State College una State Polytechnic College te Institute of Technology te Normal University te State University riano Marcos State University rinduque State College danao Polytechnic State College danao State University doro State College of Agriculture Technology amis Oriental State College of iculture and Technology untain Province State Polytechnic lege al Institute of Technology ros State College of Agriculture thern Iloilo Polytechnic State lege thern Mindanao State Institute of nce and Technology va Ecija University of Science and hnology va Vizcaya State Institute of hnology va Vizcaya State Polytechnic lege idental Mindoro National College wan State University mpon Institute of Technology panga Agricultural College

ay State Polytechnic College gasinan State University tido State University lippine Merchant Marine Academy lippine Military Academy lippine Normal University lippine State College of Aeronautics

Institutional Members continued . . . Pol

PolPhi

Qui Ram

Uni Riz Rom Sam Siq Sor Sou

and Sou Sou

MaTec

Sta Sul

Col Sul Tar Tar Taw

TecPhi

TibSci

Tom Uni Uni Uni Uni Uni Uni We We We

Tec Zam

Sci La Lip City

The

ytechnic State College of Antique ytechnic University of the lippines rino State College on Magsaysay Technological

versity al Technological University

blon State College ar State Polytechnic College

uijor State College sogon State College thern Leyte State College of Science Technology thern Luzon Polytechnic College thern Philippines Agri-Business, rine and Aquatic School of hnology te Polytechnic College of Palawan tan Kudarat Polytechnic State lege u State College lac College of Agriculture lac State University i-Tawi Regional Agricultural

College hnological University of the lippines urcio Tancinco Memorial Institute of ence and Technology

as Oppus Normal College versity of Eastern Philippines versity of Northern Philippines versity of Rizal System versity of Southeastern Philippines versity of Southern Mindanao versity of the Philippines st Visayas State University stern Mindanao State University stern Visayas College of Science and hnology boanga State College of Marine

ences and Technology

Carlota City College a City Public College College of Urdaneta

Queen’s University of Brighton