Top Banner
EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA
56

EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Dec 22, 2015

Download

Documents

Joel Carson
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

EVALUATION OF THE NATIONAL HIGHER

EDUCATION RESEARCH AGENDA –I: 1997-2008 and

NHERA-2 RESPONSESA. C. ALCALA, R.N. PADUA, O.D. LACHICA

Page 2: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

INTRODUCTION

The purpose of this presentation is to examine and evaluate the outcomes of the National Higher Education Research Agenda –I, which was implemented from 1997 to 2008. This research agenda was finalized and implemented in 1998 after a series of consultations with higher education stakeholders and contained the following basic provisions:

  National Higher Education Research Goal: to establish

and inculcate a culture of research in Philippine higher education institutions;

  National Higher Education Research Objectives:

aimed to: (a.) increase the research productivity of Philippine higher education institutions and individuals, (b.) establish a system of research-based policy environment through periodic commissioned researches, and (c.) establish support structures that would ensure long-term sustainability of research activities in Philippine higher education institutions.

 

Page 3: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

In pursuit of the explicit objectives of NHERA-I, the Commission on Higher Education formulated key strategies to respond to the three major areas of concern, namely: (a.) increasing research productivity and establishing a culture of research by capacitating and empowering individual academics in Philippine HEIs, (b.) commissioning reputable higher education individuals or groups of individuals to conduct researches which will become bases for higher education policies (commissioning), and (c.) decentralizing the level of administrative operations for higher education research in order to make the logistical support for research more accessible to researchers in various regions of the country.

Page 4: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

DECENTRALIZATION OF THE LOCI DIRECT RESEARCH SUPPORT OF RESEARCH OPERATIONS TO INDIVIDUALS AND INSTITUTIONS

RESEARCH CAPABILITY BUILDING

ESTABLISHING A RESEARCH CULTURE IN PHILIPPINE HIGHER EDUCATION

FIGURE : LOGICAL CONNECTIONS OF NHERA-I AREAS OF CONCERN

Page 5: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

PRESENTATION OF RESULTS

INPUTS

The total support for the implementation of NHERA-I over a ten-year period amounted to over PhP254M or an average of about P25 million per year to support the various activities outlined in the plan.

Page 6: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Table 1 shows the breakdown of expenditures by areas of concern: Table 1 AREA OF CONCERN AMOUNT PERCENTAGE

1. Direct Support to Research for Individuals or Institutions (thesis grants, dissertation grants, grants-in-aid or GIA, Commissioned Researches) 1.1 Thesis Grants 1.2 Dissertation Grants 1.3 Grants In Aid 1.4 Commissioned Research

P85,804,246 P4,021,950.00 P8,490,936.05 P50,848,944.10 P22,442,416.40

33.71%

2. Research Capability-Building and Enhancing Research Productivity (Visiting Fellows program, trainings/ seminars, presentation in international conferences) 2.1Support to Presentation of Papers in Local and International Forums 2.2 Visiting Fellows Program 2.3 Conferences and Seminars Conducted 2.3.1 National 2.3.2 International

P20,986,073.00 P13,127,948.00 P1,858,125.00 P3,000,000.00 P3,000,000.00

8.24%

3. Decentralizing Locus of Research Administrative Operations (support to Zonal Research Centers)

4. TOTAL

P147,763,380.00 P254,553,699.00

58.05%

Page 7: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

The establishment of structures to ensure that higher education support was within reach of the stakeholders constituted the greatest bulk of the funding for the operationalization of NHERA-I accounting for over 50 percent of the total amount for research. This funding went to the operations of the various Zonal Research Centers, which, in turn, were responsible for reviewing all research proposals from the various zones and passing judgment on the merit of these proposals. The Zonal Research Centers also spearheaded most of the research capability-building activities in their zones of influence.

Page 8: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Expectedly, direct support for research to individuals and institutions accounted for the next largest bulk of funding (33.71%) under NHERA-I. This amount supported selected theses and dissertations of graduate students, grants-in-aid to higher education institutions doing research and commissioned researches. The purpose of providing direct research support to such individuals and institutions was to stimulate more quality research outputs from colleges and universities. It may also be noted that within this funding class, support to small colleges and individuals through Grants–in-Aid accounted for over 60% of the total reported expenditure of P85,804,246.00. 

The remaining 8.24% of the funding allotted for the implementation of NHERA-I was earmarked for research capability-building through trainings, seminars, presentation in international conferences and visiting fellows.

Page 9: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

OUTPUTS

The infusion of over P254 million over a ten-year period to implement the National Higher Education Research Agenda resulted in tangible outputs in line with CHED’s objectives for research

Page 10: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

AREA I: DECENTRALIZATION OF THE LOCI OF ADMINISTRATIVE OPERATIONSA major premise for the successful

implementation of NHERA-I was the decentralization of the administrative/logistical support to research operations in the various regions of the country. Decentralization was seen as a major step towards making research support more accessible to the researchers working in the various HEIs in the country

Page 11: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

To this end, Zonal Research Centers were identified by CHED. The Zonal Research Centers served as the administrative/technical arm of CHED to implement the various aspects of the plan. The ZRC’s were identified on the basis of their: (a.) technical capability in conducting research in at least three (3) of the major higher education disciplines, and (b.) administrative capability to implement the contents of NHERA-I in terms of proposal reviews, assistance to other HEIs, monitoring and evaluation and others.

Page 12: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

ROLE INDICATORS EXPENDITURE1. Trainings/seminars

and capability-building activities

160 short-term trainings conducted to enhance research capabilityClose to 4,000 higher education faculty /researchers trained in research

P20,000,000.00@ P5,000 per person13.54%

2. Number of research proposals reviewed and evaluated

1,200 research proposals reviewed or an average of 12 proposals per ZRC per year

P2,400,000.00@ P2,000 per proposal for 2 experts’ honoraria1.62%

3. Number of Completed Researches Monitored and Reported

Thesis/Dissertation: 230GIA: 102Commissioned Studies: 28SUBTOTAL OF RESEARCHES MONITORED: 360

P1,800,000.00@ P5,000 per research monitored to cover travel and experts’ honoraria1.22%

4. Monitoring and assistance to HEI’s in zone of influence

1,730 HEIs monitored and assisted or an average of about 173 HEIs per year

P17,300,000.00@P10,000 per HEI assisted through:Research Mentoring: P5,000 for one(1) expertTravel: P5,00011.71%

5. Administrative Support PS

20 ZRC job-orders or an average of 2 per ZRC for 10 years

P24,000,000.00@P10,000 per job order per month16.24%

6. Support to Research Commissioned to ZRC’s (in the form of GIA or Commissioned Studies)

200 studies conducted by the ZRCs or an average of about 2 studies per ZRC per year e.g. graduate tracer studies, compilation of research statistics per zone, listing of all researches conducted in HEIs

P82,263,380.00@ P500,000 per study conducted55.67%

GRAND TOTAL P147,763,380.00

Table 2: Outputs of the Zonal Research Centers

Page 13: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Apart from the administrative cost of operations through payment of personal services or human resources in the ZRC’s, the greater bulk of the total allocation for the operations of ZRCs went to support the researches commissioned to the ZRC’s themselves (either GIA or special studies needed by CHED). This aspect accounted for 55.67% of the entire amount allotted followed by expenditures for trainings and capability building (13.54%) and monitoring/assistance to smaller HEIs (11.71%).

 It may be surmised that from the output of 200 research

studies alone, ranging from a compilation of research statistics (efficiency, faculty to research ratios etc.) to a compilation and analysis of research outputs of HEIs in the country, the existence of the ZRC’s is well justified since the conduct of such studies is not within the administrative purview of the regular staff of CHED. Moreover, capability building trainings were more extensively and intensively implemented by the ZRC’s within their zones of influence than would have been the case had the operation been centralized in Manila.

Page 14: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

AREA II: DIRECT SUPPORT TO RESEARCHWhile the major concern for the plan period was

the establishment of a conducive research environment for research in Philippine higher education, a sizeable portion of the total funding infusion for NHERA-I was devoted to direct grants and support to research for individuals and institutions. As listed in Table 1, a total of over P85 million was earmarked for this purpose. Table 3 shows the tangible outputs for the infusion of P85 million for direct research support:

Page 15: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Table 3. Outputs for Area II Type of Support Output Indicator Amount

1. Thesis Grant 84 grantees P4,021,950 (4.69%) 2. Dissertation

Grant 146 grantees P8,490,936 (9.89%)

3. Grants in Aid 102 institutions/ individual grantees

P50,848,944(59.26%)

4. Commissioned study grant

28 commissioned studies

P22,442,416 (26.16%

TOTAL P85,804,246

Page 16: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Tabular values show the inclination of the plan to encourage and motivate promising individuals and institutions to conduct world class research as can be inferred from the fact that over fifty percent (50%) of the allocation went to support institutions or individuals expected to do good quality research through CHED’s grant-in-aid program. Only a little over 25% went to support studies that CHED needed for policy development in the form of commissioned studies e.g. normative financing studies, amalgamation and typology studies etc.

 As a developmental agency for the furtherance of

higher education research, CHED’s focus shifted away from supporting graduate thesis and dissertations toward more support for institutional level research through grants in aid.

Page 17: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

CAPABILITY BUILDING DIMENSION

AMOUNT INDICATORS

Research Capability-Building and Enhancing Research Productivity (Visiting Fellows program, trainings/seminars, presentation in international conferences)2.1Support to Presentation of Papers in Local and International Forums2.2 Visiting Fellows Program2.3 Conferences and Seminars Conducted2.3.1 National2.3.2 International

P20,986,073.00P13,127,948.00 (62.56%)P1,858,125.00 (8.84%)P3,000,000.00 (14.30%)P3,000,000.00 (14.30%)

124 paper presenters or about 12 per year4 visiting fellows supported3000 participants3000 participants

TOTAL P20,986,073

Hand in hand with the two other major areas of concern is the concern for developing the capability of HEIs and individuals in research.

AREA III: CAPABILITY BUILDING

Page 18: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Support for research capability building appears to have been concentrated in supporting completed papers for presentation in local and international forums (62.56%) while only 8.84% of the amount in this dimension was earmarked for the Visiting Fellows Program, a program which could have been an effective strategy for assisting smaller HEIs develop their capabilities for research. As earlier mentioned, actual trainings and seminars for research capability enhancement were devolved to the Zonal Research Centers.

Page 19: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

OUTCOME AND IMPACT ANALYSIS

Given the outputs earlier presented, we now attempt to evaluate the implementation of the NHERA-I in terms of verifiable outcomes and impact on the entire higher education system in terms of research.

Page 20: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

AREA I: DECENTRALIZATION OF THE LOCUS OF ADMINISTRATIVE OPERATIONSThe impact and outcome analysis for this area of concern were

guided by the following key questions:

With the establishment of the ZRC’s as a strategy for decentralization, did “access” to research logistics (technical and financial) improve?

 Was greater efficiency achieved through decentralization in

terms of more prompt service to higher education stakeholders in research? Was there a more equitable allocation of resources for research across the different regions of the country?

Was the higher education system assured of better quality researches produced across the different regions of the country?

 

Page 21: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

1.1 Improved “Access” to Research Logistics. In order to assess this dimension, it is necessary to know the “access” situation prior to the implementation of NHERA-I. However, except for the EDCOM(1993) report, no comprehensive data base on higher education research funding from the defunct Bureau of Higher Education could be obtained. Thus, we resorted to interviewing key informants and learned that no program was ever available for funding higher education research prior to CHED but instead, the few researches conducted were funded by external funding agencies.

Page 22: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

As a surrogate measure of “access” to research logistics, we did a time-series analysis of the “number of research proposals reviewed and funded” as submitted to the ZRC’s for review over the past ten years to the “number of proposals reviewed”. A positive upward trend in this index would be indicative of greater access to research logistics in the sense that more and more higher education academics are getting seriously involved in research presumably because of the availability of research support in the regions i.e. more researches are actually getting funded.

Page 23: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Another surrogate measure of “access” to research logistics (in terms of technical support) would be the ratio of the “number of research authors who were trained under the capability building program” to the “total number of persons trained under the program” to demonstrate how the training programs successfully capacitated the individual researchers.

Page 24: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

YEAR **INDEX 1: No.of Proposals Submitted for Review to the Number of Proposals Actually Reviewed and Funded(Financial Access Index)Submitted *Funded Index

***INDEX 2: NUMBER OF RESEARCH AUTHORS TRAINED UNDER THE PROGRAM TO THE TOTAL NUMBER OF PERSONS TRAINED(Technical Access Index)Authors Trained Index

1998 212 2 .0094 10 400 .0251999 260 6 .0231 25 400 .06252000 308 4 .0129 27 400 .06752001 360 8 .0222 32 400 .0802002 420 5 .0119 37 400 .09252003 460 16 .0348 48 400 .120020042005200620072008

500 14 .0280520 12 .0231550 10 .0182680 22 .0324740 44 .0595

67 400 .167578 400 .195083 400 .207591 400 .227598 400 .2450

Table 4: Indices of “Access” to Research Logistics

*Funded: Analysis limited to GIA and Commissioned studies** Index 1:Positive upward trend = more and more researches are getting funded out of

submissions*** Index 2: Positive Upward Trend = trained researchers are becoming more engaged in the conduct of actual research and publication

Page 25: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Figure 1: Time Series Trends for Index 1 (Financial Access Index)

Figure 2: Time Series Trends for Index 2 (Technical Access Index)

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

1 2 3 4 5 6 7 8 9 10 11

Series1

0

0.05

0.1

0.15

0.2

0.25

0.3

1 2 3 4 5 6 7 8 9 10 11

Series1

Page 26: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

The establishment of the ZRC’s had greater impact on the accessibility of research support (for the HEIs and researchers) in terms of the centers’ ability to provide technical assistance than on the centers’ ability to process and fund research proposals. This can be gleaned from the more pronounced slope of the second graph on “technical Access index” than the corresponding slope of the first graph (financial access index).

Page 27: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

1.2 Enhanced Efficiency and More Equitable Resource Allocation. A good measure of efficiency would have been to calculate the average “waiting time” of a researcher from proposal submission to funding decision. This information, however, is not currently available. However, since decisions are now devolved to the zonal research centers, it is conjectured that such waiting times have been considerably reduced compared to a “central office decision” paradigm.

 As a surrogate measure of efficiency , we calculate an index

defined by: Efficiency (%) = output resources/input resources x 100

In particular, we calculate efficiency for just one aspect, namely the inputs and outputs for the “Direct Support to Research” with the following information:

GIA and Commissioned Researches: completed and publishedThesis: 80% completedDissertation: 90% completed

Page 28: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Thus: Input = Total Direct Support for Research =

P85,804,246Output = GIA + Commissioned Research + .80 x

Thesis + .90 x Dissertation = P50,848,944 + P22,442,416 + P3,217,560 + 7,641,842 = P84,150,762

 Therefore, the average efficiency of the system is

about : 98.07%. The establishment of the ZRC’s, with their hands-

on and first-hand management of the funded researches, can be identified as the main reason for this high efficiency level.

 We defer the discussion on equity in the next

section.

Page 29: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

1.3 Better Quality Research Outputs. The establishment of the ZRC’s also ipso facto defined a “quality assurance system” for research in higher education which hitherto was absent in the Philippine higher education system. The proposal reviews conducted by the ZRC’s were rigid and strict peer-review processes, thus, ensuring better quality researches in the end. A quantitative measure of “quality” can be evolved from the percentages of researches funded by CHED and subsequently published in strict peer-reviewed journals. Table 5 shows the figures for this purpose:

 

Page 30: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Table 5: Percentage of Funded Research Published in Refereed Journals Type Number of

Research Funded Number of Researches Published in Refereed Journals

Percentage

GIA 102 25 25.04% Commissioned studies

28 27 96.42%

Thesis 84 9 10.71% Dissertations 146 23 15.00% REPUBLICA AWARDS AVERAGE QUALITY INDEX

45 45 (all published in ISI-indexed journals)

100.0% 49.20%

Page 31: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Of the researches funded by CHED (through the ZRC’s), about half were published in refereed journals or in ISI-indexed journals. This shows that the quality assurance system for research adopted through the ZRC’s has improved the quality of published papers, and the Commission on Higher Education made the right decision to decentralize the focus of administrative operations to improve the quality of research in higher education.

Page 32: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

AREA II: DIRECT SUPPORT TO RESEARCH To assess the outcomes and impact of the funding

infusion to directly support research, a survey was conducted. A survey questionnaire was floated to the individual or institutional grantee of direct funding support from CHED. The key questions that guided us in our analysis of the outcomes and impact of area II funding are:

 Were the resources for research equitably allocated

across regions and institutions? What were the significant contributions of the

funded researches to the discipline or national development?

Page 33: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Presentation of Data

Distribution of grantees by regionA total of 53 CHED grantees responded to

the CHED Research Grant Survey fielded for summing up outputs under the National Higher Education Research Agenda (NHERA): 1998-2007. These grantees were those who received thesis writing grants, dissertation writing grants and the Grants-In-Aid given to faculty and researchers of HEIs. The grants covered the period from 1998 to 2007. Figure 3 shows the distribution of grantees by region.

Page 34: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.
Page 35: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Equity Issue in terms of distribution of grants

As shown herein, Region VII had 12 recipients, Region VIII and CAR had 8 grantees each, Region II had 7, Region I had 6, and Region VI with 5 grantees. The other Regions had 3 or less grantees. No data were received from Regions III and the ARMM, hence these are not included here. This may be because either there were no grantees from these areas, or none of the grantees bothered to respond to the questionnaire sent by CHED.

Page 36: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

There were only 6 recipients of thesis writing grants, 22 for Grants-In–Aid, and 25 got dissertation writing grants. Out of the 6 thesis grantees, two (2) were from CAR , two (2) from Region 7 and one (1) each from Regions I and II.

For the dissertation grantees, the highest number came from Region VII with 8 grantees, followed by CAR with 6 grantees, and Region II with 5 grantees. All the other regions had either 2 or less grantees. The respondents are distributed in Regions III, IV, V, X and the ARMM are not indicated.

Page 37: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

The 22 GIA grantees are distributed only among eight regions. Regions III, IX, ARMM and CAR had no GIA grantees. Region VIII topped the list with 7 grantees. Oddly enough, while Region VIII enjoyed a relatively large number of recipients, the other regions only had 1-3 recipients each. Furthermore, it was observed that the 7 grantees all came from one institution.

Page 38: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

The studies conducted with the three grants covered eight disciplines: Agriculture; Natural Sciences, Biology and Biodiversity; Engineering; Information Technology; Education; Health and Social Sciences.

Page 39: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

The number of grantees for each discipline were: Agriculture-11; Natural Science, Biology and Biodiversity-19; Engineering-3; Information Technology-4; Education-8; Health–3, and Social Science-16 grantees. These disciples were identified as priority areas of the NHERA I.

Nineteen (19) of the grantees had their studies under the areas of Natural Sciences, Biology and Biodiversity. This is followed by the Social Sciences with 16, and Agriculture with 11. Engineering and Health had the least number of studies made, with only three (3) studies each.

Of the total amount of P 6,764,800 that CHED granted, 73% was given to GIAs, 23% for dissertation writing, and 4% went to thesis writing. As indicated by the respondents, the amount can be broken down to: Thesis – Php 294,000; Dissertation – Php 1,551,500, and GIAs – Php 4,919,300.

Page 40: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Table 6: Summary of Productivity

Knowledge Generated & Used

Inputs to Instruction

Policy Impact Research Publication

Type of CHED Grant

A B C D E F G H I J K L M

Thesis 0 0 0 1 0 0 0 0 0 2 0 5 0

Dissertation 2 4 1 7 0 0 0 1 0 7 2 15 5

GIA 2 6 0 8 4 0 4 4 4 7 1 13 2

Legend:

A – Invention made H - Guidelines Disseminated

B – Technology Transferred I - Policy Implemented

C – Product Commercialized J – Refereed Local

D – Theories/Concept Developed K – Refereed International

E – Research Design/Method Replicated L – Presented and Published

F – Book/Instructional Materials Developed M - Poster

G – Research Translated to Policy

Page 41: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

CONTRIBUTIONS TO NATIONAL DEVELOPMENT OR THE DISCIPLINES Table 1 indicates that the thesis grantees showed the least impact on the surveyed

area among the three groups of grantees. As shown, not one of the grantees indicated that he/she had made contributions to the area of “Knowledge generated and used” and impact on “policy formulation, dissemination and implementation.” Only 2 papers were published in refereed local journals and 5 were presented in local conferences.

The Dissertation grantees, on the other hand, contributed more to the area surveyed than the Thesis grantees, except on policy making where only one paper indicated that it had developed guidelines for policy dissemination and implementation. In the areas of “Knowledge generated and used,” Inputs to Instruction,” and “Research Publication” they indicated that they have made significant contributions. In terms of publication, nine (9) have been published in refereed journals, with two published in international refereed journals. Twenty (20) out of the 24 dissertations were presented in conferences.

In terms of the areas surveyed, the GIA grantees did comparatively well. While the other two groups (Thesis and Dissertation) had practically not done anything towards policy making, the GIA studies have contributed to formulation, dissemination and implementation of policy. All the 22 GIAs were either published in refereed journals or presented in a conference and published as proceedings.

Page 42: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

AREA III: CAPABILITY BUILDINGGiven the outputs for the research

capability building program, we now wish to find out the outcomes and impact of these outputs to the overall higher education research system. We were guided only by one question: Did the program significantly impact on the number of active researchers in Philippine colleges and universities? That is, were the capability-building trainings successful in developing new researchers?

Page 43: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Given the huge number of participants in the training programs over the ten-year period, a follow up study of all the participants proved to be infeasible. Thus, we again resort to a “surrogate measure” of impact in this dimension through:

 Training Impact Index (individual) = (# of REPUBLICA

ENTRIES) divided by (NO. OF PARTICIPANTS 2000-2003) X 100%

Training Impact Index (institution) = (# of HEI RESEARCH AWARDEES ) divided by (NO. OF ENTRIES 2000-2003) X 100%

  The logic of this surrogate index is as follows: The

participants to the trainings conducted by the ZRC’s and CHED in the period 2000 to 2003 were the potential pool of researchers who will be capable to compete in the two award systems which began in 2004. Thus, the number of entries to these two award systems is an indication of the success of the training programs. Admittedly, this is a very crude measure of impact in this dimension. In the absence of a tracer study, however, this might be the best measure to consider.

Page 44: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Data:No. of Participants to Trainings (2000-2003): 600

participants (approximate figure)No. of REPUBLICA Entries: 181No. of Entries Best Research: 142No. of Awards for Best Research: 50 Hence, the impact index is: (3.) Training impact index (INDIVIDUAL)= 181/600 X

100% = 30.17% (4.) Training impact index (INSTITUTIONAL)= 50/142

X 100% = 35.21%  The research capability building programs of CHED

seemed to have greater impact at the institutional level rather than at an individual researcher level. Overall, however, the capability building program has an average impact index of only 32.5% or less than a third of the target clients. These indices are low.

Page 45: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

EVALUATION SUMMARY

The paradigm of the National Higher Education Research Agenda- I revolved around three main areas of concern, namely: (1) decentralization of the loci of research administrative operations, (2.) enhancing research productivity through direct support for research to individuals and institutions, and (3.) increasing the pool of capable researchers through a research capability building program. The three main areas of concern attempted to address a 10-year goal of establishing a Research Culture in Philippine Higher Education. The logical connections of the three areas of concern have been shown earlier.

Page 46: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Consequently, the evaluation process consisted of looking into the inputs-outputs-outcome and impacts of each dimension on the overall goal.

 In Area I (on decentralization of the loci of

research administrative operations through the ZRC’s) , results revealed that: (1) greater access to research logistics was realized through the devolution of functions to the ZRC’s with even greater gains in the area of access to “technical expertise”, (2.) a 98.1% gain in efficiency in operations was realized through the ZRC’s through the devolved process of proposal evaluation, and (3.) better quality researches were produced with a 36.8% quality index. The area on decentralization, therefore, positively impacted on the system of research logistic operations in the country.

Page 47: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

In Area 2 (on direct support to research), results revealed that the GIA’s and Commissioned studies contributed to the development of various higher education disciplines and in the formulation of higher education policies respectively. Some dimensions in this area which need to be carefully looked into include: ensuring greater equity (across regions) in the distribution of thesis/dissertation grants and increasing the funds allotted for these two dimensions.

Page 48: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Finally, in Area 3 (on capability building), results reveal a low 32.5% impact of the capability building programs on the intended targets (individuals or schools). This demonstrates a need to intensify the various research capability building programs of CHED with particular emphasis on the Visiting Fellows Program (with fellows assigned to a cluster of schools to lead in the area of research and research management). This strategy has the potential of rapidly increasing the pool of qualified researchers in the various colleges and universities in the different regions of the country.

Page 49: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Table 7: HEIs Engaged in Various Research Activities

Activities NumberResearch Capability Building 283Research Projects Implemented 73HEIs under the ZRCs 511HEIs Engaged in Research 107Researchers Involved in GIA and/or Commission Research

351

HEIs involved in ZRC projects 107

The Table 7 above shows the summary of research related activities of the HEIs under the ZRCs.

Page 50: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

As shown here there were 511 HEIs that has been assigned to 11 Zonal Research Centers. The member of HEIs per ZRC ranges from the smallest of 20 to the largest of 69 HEIs or an average of 46 HEIs per ZRC. Since 1998 to 2007, a total of 283 Research Capability Building activities have been conducted. These were mostly in a form of seminars, workshops, and focused group discussions. The objective of these capability building activities was to equip the participants with research skills to perform researches along the research thrust of the corresponding ZRCs. This also included the discussion of the preparation of research proposal. The effectiveness of the capability building activity is reflected in the number of proposal submitted after the activity.– which eventually be refined and be endorsed for CHED funding. After the project shall have been finished, if necessary, a writeshop on writing of report both for presentation and writing for refereed journal were conducted.

Page 51: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

Since there were 73 projects/researches implemented, indicates that the rate of success of these research capability building activities is about 25 per cent or one research project for every 4 capability building activities. The table above further reveals that there were 351 researchers involved in the these projects. It implies therefore that there were about 5 researchers involved in each research project. This practice is in line with the CHED OPPRI policy of encouraging research projects to be inter/multi disciplinary in nature. As indicated earlier, 351 researchers were involved in the 73 research projects and these researchers were from the 107 HEIs. It is sad to note however, that only 107 of the 511 HEIs were involved in these research activities spearheaded by the ZRC – which only account for 22 per cent of the HEIs. This means only about one in every five HEIs were involve in research involving the ZRCs. I think there is a lot that needs to be done to improve this picture.

Page 52: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

To determine how the grantees of research performed, the CHED OPPRI fielded questionnaires on the impact of funded studies on: Knowledge Generated and Used, Inputs to Instruction, Impact on Policy and Research Publications

A total of 53 CHED grantees responded to the CHED Research Grant Survey fielded for summing up outputs under the National Higher Education Research Agenda (NHERA): 1998-2007. Although this is only about 16 per cent of the total number of grantees, at this juncture and for the sake of the analysis, we assume that the sample gathered is representative of the grantees who received thesis writing grants, dissertation writing grants and the Grants-In-Aid given to faculty and researchers of HEIs. The grants covered the period from 1998 to 2007.

Page 53: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

PART II: RESPONSE OF NHERA-2

NHERA-1 RESULTS NHERA-2 RESPONSE

A. DECENTRALIZATION OF LOCI OF RESEARCH ADMINISTRATIVE OPERATIONS

A. RE-ENGINEERED DECENTRALIZATION PLAN

1. Low effectiveness index for capability building programs for individuals (author:trainee ratio)

1. Capability building programs that will be supported by CHED will henceforth be output-based i.e. should result in publishable papers. Non-output based trainings will be the responsibility of the HEIs. Capability-building programs will be PURPOSIVE rather than ad-hoc.

2. Low effectiveness index for capacitating HEIs

2. Strengthen/reinforce the Senior Visiting Fellows Program (SVFP) through a revised TORe.g.a.Each SVF takes care of mentoring at least three(3) HEIs through regular monthly consultations for two(2) years;b.SVF receives a monthly support from CHED/HEI that will be attractive to retired experts;c.SVFP shall be transferred to the CHED Regional Satellite offices for Research.

Page 54: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

NHERA-1 RESULTS NHERA-2 RESPONSE

3.Low quality index for some ZRC research outputs as evidenced by publication in non-indexed journals

3. Reconceptualize Decentralization viz ZRC’s:a.Identify the country’s Centers for Research Excellence (CRE) in Luzon , Visayas and Mindanao (max: 5). The only TOR for the CRE shall be the annual publication of researches in international indexed-journals.b.Transfer all research administrative functions of the ZRC’s to the CHED Regional Offices by hiring satellite contractual staff for Research in the regional offices in the level of a PDO IV.

B. Capability-Building B. (see A)

Page 55: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

C. Direct Support to Research C. Direct Support to Research

1. Low efficiency index for thesis/dissertation support; low equity response for thesis/dissertation.

1. Reconceptualize “support” concept viz. provide incentives for finished products (outputs) rather than proposals e.g. P100T per completed and published dissertations and P50T per completed and published thesis. Publication must be in a refereed journal or as much as possible in an ISI-indexed-journal.

2. Make distribution of thesis/dissertation incentives more equitable using graduate school population as regional basis for equity.

2. Low and slow turnouts of GIA’s; undetermined quality for some GIA’s

1. Strengthen the monitoring system for GIA’s through the Regional Satellite Research Offices.2. Regularize the publication of “Occasional Papers” and strengthen its refereeing system viz. annual instead of occasional.

3. Make distribution of GIA’s equitable across disciplines (rather than region). More incentives for cross-cutting projects.

Page 56: EVALUATION OF THE NATIONAL HIGHER EDUCATION RESEARCH AGENDA –I: 1997-2008 and NHERA-2 RESPONSES A. C. ALCALA, R.N. PADUA, O.D. LACHICA.

NHERA-1 RESULTS NHERA-2 RESPONSE

3. Erratic and non-periodic grants for Commissioned researches

3. Tie up Planning and Policy Development Office with the Research Office’s functions. The PPDO should determine policy requirements of CHED for the year and communicate these to CHED’s research office for appropriate commissioning.