Indicators of university-industry knowledge transfer ...diffusion of measurable outputs. For example, while the number of consultancy contracts a university stipulates provides some

CIMR Research Working Paper Series

Working Paper No. 13

Indicators of university-industry knowledge transfer performance and their implications for universities:

Evidence from the UK’s HE-BCI survey

by

Federica Rossi Birkbeck, University of London

School of Business, Economics and Informatics Malet Street, London WC1E 7HX

[email protected]

Ainurul Rosli University of Wolverhampton

University of Wolverhampton Business School MN Building, Nursery Street, Wolverhampton, WV1 1AD, UK

[email protected]

19 July 2013

ISSN 2052-062X

2

Abstract:

Focusing on the measurement of universities’ performance in knowledge transfer, we outline some critical issues connected with the choice of appropriate indicators: in particular, we argue that, in order to allow universities to correctly represent their knowledge transfer performance, indicators should include a variety of knowledge transfer activities, reflect a variety of impacts, allow comparability between institutions, and avoid the creation of perverse behavioural incentives. To illustrate these issues empirically, we discuss the case of the United Kingdom’s Higher Education –Business and Community Interaction (HE-BCI) survey. We show that the indicators used to measure and reward universities’ engagement in knowledge transfer are not fully comprehensive, they are better suited to capture the impact of certain types of activities than others and they are influenced by institutional strategies and characteristics rather than simply reflecting different performances. The conclusions explore some promising directions to address some of these problems.

Keywords: Knowledge transfer, knowledge exchange, performance indicators, university-industry relationships, technology transfer

JEL codes: O31, O32, O34, O38

3

1. Introduction

One of the most widely shared views in current economic thinking is that the creation and dissemination of new knowledge underpinning innovation is an important driver of economic growth (Romer, 1990). As a key producer of new knowledge in the economy, the university’s role in supporting regional and national economic growth and development is increasingly acknowledged: universities are now viewed as economic agents that engage with a multiplicity of stakeholders in order to deliver economic benefits, rather than “ivory towers”, producing knowledge in isolation (Etzkowitz and Leydesdorff , 2000). Indeed, knowledge transfer has become a “third mission” for universities, which complements the traditional research and teaching missions.

Consequently, while in the past governments funded universities only for their research and teaching activities, nowadays they are increasingly providing incentives to ensure that universities transfer knowledge to economic agents that can exploit it productively. For example, in the UK, the main public funding councils for higher education have launched a specific stream of funding to support and incentivize universities’ knowledge transfer activities (the Higher Education Innovation Fund, or HEIF) appropriately referred to as “third stream” funding (Molas Gallart and Martinez, 2007; Kitagawa and Lightowler, 2012). In other countries, support for knowledge transfer activities occurs in the form of national project-based funding for knowledge transfer (for example in Spain; Molas Gallart and Martinez, 2007) and support for the development of a knowledge exchange infrastructure, whether at national level (as in Sweden; Sellenthin, 2006), at regional level (as in Germany; Sellenthin, 2006) or at State level (as in the US; PACEC, 2010).

In order to identify if and where interventions are needed, and to design appropriate incentives, the universities’ performance in knowledge transfer activities needs to be monitored and assessed. Monitoring often occurs through systematic data collection exercises, in which universities are requested to provide quantitative information that describes each university’s degrees of engagement in various activities and allows the comparison among universities’ performances. For example, in Spain the Conference of University Rectors distributes an annual survey to the technology transfer offices of universities and public research organizations (Molas Gallart and Martinez, 2007). In the US and Canada, the Association of University Technology Managers (AUTM) runs a yearly survey of university technology transfer offices, mainly focused on technology commercialization activities, addressed to about 200 research universities. At European level, several associations of technology transfer offices such as the European Knowledge Transfer Association (ProTon) and the Association of European Science and Technology Transfer Professionals (ASTP) organize their own surveys, targeting the associations’ members. In the UK a comprehensive survey (Higher Education Business and Community Interaction survey, henceforth HE-BCI) currently managed by the Higher Education Statistics Agency is distributed yearly to all universities, collecting a wide range of information about their knowledge transfer activities. Australia is debating the implementation of a similar exercise, following closely on the indicators being used in the UK (Jensen et al., 2009).

The choice of indicators has important implications for universities. Indicators are recognized to play a performative role (Davis et al., 2010; Merry, 2011): that is, they signal what activities are considered important by policymakers and funding agencies, and what kind of performance may be associated to implicit rewards (such as better reputation and prestige). In some countries, such

4

as the UK, indicators are even used to allocate third stream funds to universities, impacting directly upon the universities’ financial prospects.

However, identifying indicators that allow universities to provide a fair and accurate representation of their knowledge transfer activities, without introducing implicit behavioural incentives, is a complex task; and the indicators that, in practice, are used in order to assess universities’ knowledge transfer performance suffer from numerous limitations.

In section 2, we consider the problem of identifying appropriate knowledge transfer indicators in general terms: we argue that knowledge transfer processes can unfold in many ways, and choosing indicators shaped by a narrow view of the knowledge transfer process limits the ability of universities to accurately represent their engagement in knowledge transfer and the impact of their activities. In section 3, we illustrate this argument empirically by considering the choice of indicators currently adopted in the United Kingdom. We show that the indicators used to measure and reward universities’ engagement in knowledge transfer activities are better suited to capturing the impact of certain types of activities than others and that, since institutions have different knowledge transfer profiles, this may lead some of them to be unable to correctly represent their actual knowledge transfer performance. In section 4, we derive some implications for policy. It must be stressed that the focus of this paper is on universities’ ”third mission” activities, and does not aim to discuss the indicators used to measure universities’ performance in other areas such as teaching and research.

2. The nature of knowledge transfer and its implications for the choice of performance indicators

While the traditional ”technology push” view of innovation focused researchers’ attention on the transfer of knowledge that could be easily patented and commercialized, and easily implemented into new products and processes, recent approaches suggest that innovation originates from a variety of sources that go beyond the development of new technical and scientific knowledge (von Hippel, 1988; Tidd and Bessant, 2009) For example, innovation often originates from the understanding of user needs and the identification of market niches, from production activities, from regulatory compliance, from improved understanding of organizational processes, and so on. Contrary to existing stereotypes, the service industry, now the mainstay of our economy, has been shown to be a highly innovative sector, and one in which innovation very often relies upon the implementation of organizational improvements, new business models and market intelligence, beyond the development and adoption of new technologies (Tether, 2005; Forsman, 2011). In parallel with the increasing interest in non-technological knowledge as a source of innovation, the understanding of universities’ contribution to the economy has evolved too, from an almost exclusive focus on the unidirectional transfer of knowledge embedded in intellectual property rights and other easily quantifiable outputs such as publications, to a broader approach, which recognizes the variety of university-industry interactions and in which knowledge is not just transferred but ”exchanged” (HEFCE, 2012). The question is not the mere transfering knowledge throuh market forces, but through a more complex process of appropriation of knowledge by society, where other stakeholders including the policy makers also come into the picture (Brodhag, 2013).

Indeed, much empirical evidence on university-industry knowledge transfer has shown that universities engage in a plurality of interactions with economic and community stakeholders,

5

only a few of which are based on the sale and licensing of patents and the direct exploitation of academic research outcomes in an industry setting (Bekkers and Bodas Freitas, 2008). Different types of universities tend to engage in different types of knowledge transfer activities, for example according to their research orientation (basic vs. applied), their research intensity (research-intensive or teaching-intensive; Wright et al., 2008), their disciplinary focus (science, technology or the arts and humanities), their geographic localization (urban or peripheral) and their knowledge transfer policies (Di Gregorio and Shane, 2003).

The literature has illustrated that knowledge transfer is a complicated and complex process (Kingsley, Bozeman and Coker, 1996; Bekkers and Bodas Freitas, 2008; Hughes, 2010), in which all the parties involved learn from the interaction, active participation of the receiver is crucial for knowledge transfer to occur, and the respective parties’ prior knowledge base and absorptive capacity strongly influence the outcome of the interaction (Ternouth et al, 2012). Additionally, knowledge transfer processes generate strong spillovers that benefit agents that go well beyond those involved in the initial transfer, including of course the university itself. Hence, the impact of knowledge transfer is not adequately captured by simple indicators of output of the knowledge transfer process, and of their economic value: as knowledge transfer often occurs through interactions rather than simple transactions, the interaction processes themselves matter: that is, the frequency, characteristics and quality of the interactions and the (short and long term) learning processes that all participants in the interactions experience (i.e. with a focus on knowledge exchange rather than just knowledge transfer). Moreover, it has been shown that the specific identities of the parties involved matter for the nature and success of knowledge transfer: for knowledge to be communicated and received properly, the organizations involved must possess sufficient absorptive capacity (Cohen and Levinthal, 1990).

Choosing appropriate indicators in order to measure the intensity with which universities engage in a range of knowledge transfer activities, and the impact of each of these activities, is therefore a complex task which must recognize the variety of knowledge transfer activities performed by universities and the specificity of each of these activities.

In particular, we can identify several general apects that should be taken into account when attempting to measure universities’ knowledge transfer performance in a fair and accurate way.

(i) Variety of knowledge transfer activities. The range of knowledge transfer activities considered must be broad enough to reflect the variety of activities undertaken by universities: if the choice of activities to be measured is not comprehensive enough, the indicators may misrepresent the knowledge transfer performance of universities that engage in activities that are not measured. For example, universities that specialize in the arts and humanities usually do not produce patentable research outputs, so relying upon indicators heavily focused on the filing of patents and the execution of licenses could prevent these universities from correctly representing the knowledge transfer activities they engage in.

(ii) Variety of impacts. Focusing only on output-oriented indicators may penalize universities that transfer knowledge whose social and economic impact is not accurately reflected by the diffusion of measurable outputs. For example, while the number of consultancy contracts a university stipulates provides some measure of the engagement in knowledge transfer of its academics, it does not provide information about the benefits that such activities have generated for the receiving organizations. A particularly problematic indicator is the income that universities receive from their knowledge transfer activities. Income can be considered as an ”output-oriented” indicator to the extent that it is assumed to reflect the value that external

6

partners place on the knowledge they receive from universities (and hence provide a proxy of the value created through knowledge transfer). However, this assumption is questionable, and several arguments can be made to support the view that the income a university receives for knowledge transfer does not necessarily reflect the impact of its activities:

• more prestigious institutions may be able to charge more for their services because of reputation, and not because of the value of the knowledge transferred is greater ( on the economic returns on reputation, see Fombrun 1995; Roberts and Dawling, 2002) ;

• the cost of knowledge transfer projects varies widely, and higher income may simply reflect the higher cost of providing a certain service. For example, the cost of knowledge transfer projects in the social sciences and humanities is often (though by no means always) lower than the cost of projects in areas like medicine and engineering where expensive equipment or complex clinical studies may be required;

• certain forms of knowledge are transferred for free or at a very low price because they are aimed at people who cannot pay for them (such as services to the community) but their value can be high from a social viewpoint;

• universities may choose to disseminate knowledge freely (under open source licenses or other open mechanisms) in order to achieve greater impact without receiving an income

• related to the previous two points, some projects are more likely to generate broad benefits for society and large externalities that may not so easily be captured by individual beneficiaries (e.g. specific individuals or specific organizations) who therefore are less willing to pay for them, even though these projects may have the largest impact;.

• some knowledge transfer projects can be labelled with large uncertainty, especially if they involve a research component (such as collaborative research projects), so that neither party is fully aware at the start of the project of the time span in which benefits will be accrued, and of the value of these benefits, so that the income received is only partially capturing these aspects.

(iii) Comparability between institutions. Output-oriented indicators may be strongly affected by factors related to institutional characteristics such as size, disciplinary orientation, mission. For example, reliance on indicators based on the absolute amount of knowledge transfer activities performed, rather than on measures of engagement in knowledge transfer per unit of staff, could disadvantage smaller universities. The choice of indicators should allow comparability across different institutions, so that differences in the value of indicators can be ascribed to genuine performance differentials rather than to institutional characteristics.

(iv) Awareness of behavioural incentives. The system should be structured in such a way as to avoid the creation of perverse behavioural incentives. If the chosen indicators specifically reward only certain knowledge transfer activities, and not others, this creates implicit incentives for universities to engage only in the activities that are rewarded; but these activities may not necessarily be the most effective ways to transfer knowledge for all universities. For example, if the choice of indicators rewards universities that transfer knowledge via the sale of patents and licenses, this would incentivise universities to apply for more patents, even in cases when this is not beneficial (Lambert, 2003).1 1 The problem of behavioural incentives is very broad, and links to another important issue: the interactions between knowledge transfer and teaching and research activities. It is well known that while the performance of knowledge transfer activities can productively complement and enhance teaching and research activities (Abreu et al, 2008), occasionally it can also give rise to conflicting incentives (Lambert, 2003). Indicators of knowledge transfer

7

3. Case study: the HE-BCI survey and third stream funding allocation in the United Kingdom

Since there are major differences across countries in terms of how knowledge transfer between university and industry is measured, we illustrate the relationship between choice of indicators and universitires’ ability to adequately represent their knowledge transfer performance by focusing on a specific case: the approach used to measure and reward the knowledge transfer performance of universities in the United Kingdom. Several reasons make this an interesting case study. As the United Kingdom is one of the first countries in the world to have launched a comprehensive data collection exercises on universities’ knowledge transfer activities, its choice of indicators is likely to provide a benchmark for other countries in Europe and elsewhere. Therefore, understanding their rationales and drawbacks is relevant beyond the country’s borders. Since this survey is very comprehensive when compared to other systematic data collection exercises on universities’ knowledge transfer activities2, any limitations identified in this survey apply even more strongly to similar exercises implemented in other countries. Finally, the availability of secondary data that are comparable across different institutions and over time allows us to support some of our arguments with quantitative evidence.

3.1. The HE-BCI survey: a critique of the chosen indicators

In the late 1990s, the Higher Education Funding Council for England (HEFCE) began to run a systematic survey (HE-BCI) aimed at capturing the exchange of knowledge between higher education institutions (HEIs), the business community and society at large (HEFCE, 2012). The survey consists of two parts: Part A for data on universities’ knowledge transfer strategies, policies and infrastructures, and Part B for financial and other quantitative data measuring engagement in and impact of knowledge transfer, relative to a specific year.3

The HE-BCI survey is addressed to all universities in the United Kingdom, and it is used for reference towards grants allocations supporting knowledge exchange (the so called ”third stream” funding, to distinguish it between the other two streams of funding, for education and research): the Higher Education Innovation Fund (HEIF) in England, the Innovation and Engagement Fund in Wales, the Knowledge Transfer grant in Scotland and the Higher Education Innovation Fund (HEIF) in Northern Ireland. While, over time, all funds have moved from competitive to formula-based allocation, the English HEIF is the only one that is currently allocated 100% through formula4: since 2012, all third stream funding in England is distributed pro-rata to universities according to their share of overall knowledge transfer income, based on the information presented in Part B of the survey. To compute the institution’s income from performance should therefore take care to avoid incentivizing academic to undertake behaviours (such as excessive patenting and secrecy or allowing excessive influence of industrial sponsors on research design and the selection of research results), which in the long run may undermine scientific credibility and reduce the possibility for others to engage in research. While this issue is very important, it lies outside the scope of the present paper. 2 Rosli and Rossi (2014) present a comparison among the surveys implemented in the UK, USA and Canada, Australia and Europe. 3 Since 2009, the survey has been collected and validated by the Higher Education Statistics Agency (HESA). 4 Northern Ireland allocates 20% as foundation funding (strategic funding) and 80% of the funds through formula. Wales allocates 25% through competitive bidding and 75% through formula. Scotland allocates the funds through base allocation (8%) and formula (92%).

8

knowledge transfer, the incomes derived from each activity considered in part B of the survey are summed over (each activity is weighted equally, although any income obtained from SMEs is assigned double weight). Then, funds from HEIF are assigned to each institution proportionally to that institution’s share of overall knowledge transfer income, as long as the institution reaches a minimum threshold income of £250,000. The actual allocation depends on the HEIF’s annual budget.

In the rest of this paper, we focus on the indicators contained in the 2010/11 edition of the survey, and use information relating to the 131 English universities in order to support our arguments with quantitative evidence. Since only some of the quantitative information contained in Part B is used as a basis to compare and reward universities’ performance, we do not analyze in detail the more qualitative information collected in Part A. Table 1 summarises the areas and indicators measured in Part B of the HE-BCI survey. Each of Part B’s five sections, listed in the first column, includes several key dimensions or sub-areas (listed in the second column). Several indicators are used in order to measure performance in each sub-area (listed in the third column).

Table 1:Activities and indicators included in part B the HE-BCI survey

Sections Sub-areas Indicators Research related activities

Collaborative research with public funding (i)  

Income, in–kind (ii) contribution  

Contract research (iii) Income, total value, number of contracts (by: SME (iv) , Non SME commercial, non-commercial)

Business and Community service

Consultancy contracts Income, total value, number of contracts (by: SME , Non SME commercial, non-commercial)

Courses for business and the community (CPD and CE)(v)

Revenue, total learner days delivered(vi) (by: SME , Non SME commercial, non-commercial, individual)

Facilities and equipment related services

Income, total value, total number of services (by: SME , Non SME commercial, non-commercial, individual)

Regeneration and development programs

Regeneration and development programs

Income from European Regional Development Fund (ERDF), European Social Foundation (ESF), UK Government regeneration funds, Regional Development Agency (RDA) programme, Others Income

Intellectual Property (IP)

Disclosures and patents filed by or on behalf of the HEI

Number of new patent applications filed in year Number of patents granted in year Cumulative patent portfolio(vii)

Licence numbers Number of licenses for non-software and software (by: SME, non-SME commercial and non-commercial)

IP Income Partner type: SMEs, Other (non-commercial) businesses and other non-commercial organisations). IP revenues, Total cost

Spin-off activity Spin-offs(viii), staff start-up(ix) graduate start-up(x) HEI owned, non-HEI owned. Number of active firms, estimates employment, turnover, investment received

Social, community and cultural engagement

Public lectures, Performance arts, Exhibitions, Museum education, Other

Number of Attendees (free events, chargeable events), staff time

9

Note to table: (i) Public funding: UK Department for Business, Innovation and Skills (BIS) research councils, royal society and British Academy, other UK government departments, EU government, and others. (ii) In-kind: contributions to the project from the non-academic collaborators. (iii) Non-public funding and research councils. (iv) Employ fewer than 250 employees worldwide (including partners and executive directors), and has either an annual turnover not exceeding 50m Euros (approximately 42m British Pound), or an annual balance sheet total not exceeding 43m Euros (approximately 36m British Pound), and conforms to the following independence criteria: no more of 25% of the capital or the voting rights is owned by an enterprise falling outside the definition of an SME (HEFCE, 2011). (v) Excluding pre-registration funded by the National Health Service (NHS) or Training and Development Agency (TDA). (vi) One day is equivalent to one person receiving eight hours of teaching/training. (vii) Active (registered under licence to an external party) and live patents. (viii) Spin-offs are defined as companies set-up to exploit IP that has originated from within the HEI. (ix) Staff start up are defined as those companies set-up by active (or recent) HEI staff but not based on IP from the institution. (x) Graduate start-ups include all new business started by recent graduates (within two years) regardless of where any IP resides.

This table suggests that the choice of areas of knowledge transfer activity is very extensive but not quite exhaustive. Although it tries to capture all possible knowledge transfer activities, not all of them are investigated with the same degree of detail and some are overlooked.

The measurement of knowledge transfer via intellectual property rights and spin offs is attributed high importance, as it includes 4 out of the 10 sub-areas measured in part B of the survey, and nearly 36% of the questions included in the survey. This is despite evidence that shows that only few universities use this model with appreciable intensity and success (Litan et al., 2008), as it suitable to a limited number of scientific fields (Harabi, 1995; Brouwer and Kleinknecht, 1999). Moreover, the focus is strongly on patents and copyright licenses: little attention is paid to other intellectual property rights (design rights, trademarks) and to many non-proprietary types of intellectual property that universities produce (materials and artefacts not protected by intellectual property, or protected by open source or creative common licenses such as open source software, blogs, wikis, open source film, open source media, open source pharmaceuticals, etc.) (Andersen et al., 2012; Baghurst and Pollard, 2009)). As some types of disciplines (the arts and humanities, for example) are likely to generate the latter forms of intellectual property rather than patents, institutions that are relatively more focused on these disciplines may end up being unable to correctly represent the amount of knowledge transfer they engage in.

Some attention is paid to forms of knowledge transfer that involve open dissemination: publicly-funded collaborative research with non-academic partners, publicly-funded regeneration programmes and knowledge-dissemination activities in the humanities and social sciences. However, these activities are quite marginal: not only do they represent, together, only around 12% of the questions included in the survey, but the impact of publicly-funded programmes is mostly measured on the basis of the funding they attracted, neglecting other outputs (for example, collaborative research can produce joint university-industry publications, support joint workshops and other openly disseminated outputs, and regeneration programmes can have many valuable impacts on the community). This approach may reflect the view that outputs that are openly disseminated do not produce economic impact (this would be consistent with the choice to neglect forms of intellectual property that are non proprietary), a view that has, however, been disproved by evidence (numerous industry surveys have found that firms consider “open science” channels such as scientific publications and academic conferences as the most important ways to access academic knowledge; see among others: Arundel and Geuna, 2004; Mowery and Sampat, 2005; D’Este and Patel, 2007; Abreu et al., 2008; Bruneel et al., 2009). It may also reflect a concern with keeping a clear distinction between outputs that result from

10

research activities (such as publications) and outputs from knowledge transfer activities, where in practice such distinction is not so easy to make (activities such as university-industry collaborations and regeneration/development programmes often have an important research component).

The survey includes some forms of knowledge transfer that involve direct interactions with industry and the local business environment and community: contract research, consultancy, courses, facilities and equipment-related services. However, several important types of direct interactions between university and industry personnel are not included, such as graduate placements in industry, recruitment of university staff members to industry positions, academics’ participation in industry conferences and workshops, placements of entrepreneurs and industry personnel in universities, visiting scholarships, etc. (other channels or routes to knowledge exchange are presented in Dutrénit et al., 2010; Hughes et al., 2011; Boardman and Ponomariov, 2009; Jensen, R., Thursby and Thursby, 2010; Bekkers and Bodas Freitas, 2008). These may be particularly important in disciplines that are applied in nature (such as architecture, design, engineering, medicine and others). Furthermore, interactions around production and service activities, such as prototyping, clinical testing and design services, would fall within the very generic area of “Facilities and equipment related services” where they would be grouped with standardized, non-knowledge producing services like room and equipment rental activities.

The choice of indicators is strongly biased towards output-oriented measures, quantifying the outputs of knowledge transfer processes and in particular the income received from knowledge transfer activities; income, moreover, provides the only basis for the formula used to allocate third stream funding. The characteristics and quality of the interactions through which knowledge transfer takes place (for example their duration, the number of partner organizations and people involved, the partners’ satisfaction with the interactions, their perception of what they learned from the interactions and the short and long term benefits they received) are usually not considered. The only indicators that capture some interaction aspects measure the number of contracts issued and the types and locations of partners (only considering broad categories such as local region/external). Moreover, although it was claimed that the dataset provides valuable and in depth commentary on the extent of knowledge exchange in the UK, the mechanisms being put in place are only representing uni-directional knowledge transfer from the HEIs, and no attempts are made to explore the benefits that universities derive, besides the income they generate from these activities.

3.2. Are performance indicators neutral with respect to universities’ knowledge transfer profiles?

Using quantitative data drawn from the most recent HE-BCI survey (2010-11) and data on the HEIF funding allocations received by 131 English universities in 2013, we show that the choice of indicators to measure and reward knowledge transfer performance is not neutral with respect to the knowledge transfer profiles of different universities (which in turn depend upon different institutional strategies and characteristics): that is, universities that engage in certain types of knowledge transfer activities are better able to represent the intensity and impact of their engagement in knowledge transfer, than other universities which engage in other activities. First, we present a descriptive analysis that shows that universities with different knowledge transfer objectives have different profiles of engagement, and that different profiles are associated with different knowledge transfer performance, when the latter is measured using income; however

11

the relative performance may change when different indicators are used. Second, we perform a regression analysis on the relationship between the universities’ performance in terms of funding accrued and a range of institutional and strategic variables, showing that these features significantly influence performance.

3.2.1. Universities’ different knowledge transfer profiles

The HE-BCI data from 2010/11 shows that different English universities have different knowledge transfer profiles: in fact, universities have different knowledge transfer objectives and that they focus relatively more on areas of knowledge transfer that are consistent with their objectives. Universities participating in the survey were asked to state their main objectives, choosing three out of 13 possible options.5 By applying a hierarchical clustering algorithm to these 13 variables, we have identified 6 distinct clusters of universities according to their key objectives. Universities in the first three groups (clusters 1, 2 and 3) have a national or global focus, although with different emphasis; the rest of the universities groups have predominantly a local focus. To simplify the analysis, we have further agregated the three smaller clusters with ”local” objectives into a single cluster comprising 25 institutions (cluster 4). Table 2 summarises these clusters6.

Table 2. Clustering universities according to their knowledge transfer objectives

Cluster: focus Main knowledge transfer objectives N. universities

% universities

1. Research and TT Supporting business via research and technology transfer

42 32.06%

2. Education Widening access to education and meeting nation's demand for skills

27 20.61%

3. General Support for SMEs, education and research 37 28.24%

4. Local

Focus on SMEs and local employment; focus on attracting students and building community links; focus on local partnerships and regional skills

25 19.09%

The next figure shows the knowledge transfer engagement profiles of universities in the different clusters. Since we want to compare the various clusters in terms of actual engagement rather than income accrued, we consider all the activities for which data on the intensity of engagement are collected (number of research contracts, number of consultancy contracts; number of facilities and equipment-related services; learner days of CPD courses provided; number of disclosures; number of licenses; number of spinoffs; academic staff days of public events performed). Each

5 The 13 options are: Access to education, Graduate retention in local region, Technology transfer, Supporting small and medium size enterprises (SMEs) Attracting inward investment to region, Research collaboration with industry, Attracting non-local students to the region, Support for community development, Developing local partnerships, Management development, Meeting regional skills needs, Meeting national skills needs, Spin-off activity. 6 According to a Kruskal-Wallis rank test, 8 out of the 13 variables used to construct the clusters have statistically significant means differences across the four clusters.

12

indicator is then divided by the number of academic staff employed by the institution, and normalized to lie between zero and 1.

The figure shows that universities in the various clusters have different profiles, in line with their strategic objectives. Universities in the "Research and TT” cluster are relatively more engaged in intellectual-property related activities (disclosures, licenses, spinoff companies), since they transfer the outputs of their research activities. Universities in the ”Local” and ”Education” clusters are relatively more engaged in consultancies, public events, CPD courses and facilities and equipment-related services, as their missions bring them to engage more directly with their stakeholders using a variety of interactions. The ”Education” cluster also appears to have a more diversified engagement profile than the other clusters.

Figure 1. Knowledge transfer engagement profiles of universities in different clusters

When universities’ profiles of engagement in knowledge transfer are different, it is important to use a broad range of indicators in order to allow all institutions to correctly represent their actual knowledge transfer activities. Instead, the indicators considered not only overlook some important activities (as mentioned in the previous section) but they also capture a limited range of impacts of the knowledge transfer process, being focused on quantifying the output of knowledge transfer rather than the characteristics of the transfer process.

Indeed, output-oriented indicators are prevalent in the survey, and moreover universities’ performance is rewarded only on the basis of one type of output-oriented indicator: the income that universities receive.

3.2.2. The limitations of income as a performance indicator: correlation with institutional profile and characteristics

0

0.02

0.04

0.06

0.08

0.1

0.12

Contract research

Consultancy

Facilities and equipment

related services

CPD courses

Spinoff companies

Licences

Disclosures

Public events

Research and TT Education General Local

13

The focus on income can lead to an inaccurate representation (and possibly reward) of universities’ performance. First, some universities may engage relatively more in knowledge transfer activities that generate low or even zero income but whose value can nonetheless be high in both social and economic terms: this may be the case of activities involving the creation of social and community links, the enhancement of cultural and social capital, the upskilling of the local and national labour force. One indication that universities that focus relatively more on low-income activities may be under-rewarded is the finding that universities in the ”Education” and ”Local” clusters – which as shown in Figure 1 are more likely to perform activities related to education and public engagement – tend to be under-represented among the best-performing institutions, and over-represented among the least performing ones. Universities in the ”Education ” and ”Local” clusters are, together, 40% of the population, but make up 63% of the group of universities that have received zero funding from HEIF, and only 13% of the group that has received more than £2m. Universities in the ”Research and TT” clusters are, by contrast, 32% of the population, but only 7% of the group that has received zero HEIF funding and 73% of the group that has received more than £2m. The mean funding obtained by universities in the “Research and TT” category (£1,974,043) is much higher than that obtained by universities in all the other categories (£829,811 in the “general” group, £743,885 in the “education” group and £775,880 in the “local” group), and the differences are all statistically significant. At the same time, as shown in Figure 1, universities in the ”Research and TT” cluster do not always perform more contracts per academic staff than other universities: indeed, universities in the ”Education” and ”Local” clusters tend to do more contract research, consultancy, facilities and equipment-related services and CPDs per unit of staff: this suggests that these universites receive less income not because they perform less knowledge transfer per capita, but because they are either focusing on less remunerative knowledge transfer activities, or because they are smaller institutions, or both.

To understand how knowledge transfer profile affects performance, we can consider the role of universities’ orientation in terms of scientific disciplines. Very generally, we can expect knowledge transfer activities involving the humanities to generate impacts that are less likely to be quantifiable in monetary terms, because their target clients are more often disadvantaged socioeconomic groups, or the broader community. Moreover, these activities are generally less costly (compared to, for example, clinical trials and prototyping) and so they require less income to cover their costs. However, lower income is not necessarily associated with lower impact of these activities.

Indeed, if we correlate the the institutions’ HEIF funding allocation with the share of staff employed in each discipline in 2010/11 (considering nine main categories) 7 we find that the correlation between funding allocation and specialization in most of the sciences is positive and significant, while the correlation between funding allocation and specialization in the arts and humanities is negative and almost always significant.

Therefore, using the income that universities receive from knowledge transfer activities to assess performance is likely to produce results that overstate the performance of universities that are strongly oriented towards the medical and natural sciences and engineering, and to understate the performance of those that are more oriented towards the humanities.

7 These data referred to staff full-person equivalent (excluding atypical) by cost centre, 2010/11. Source: HESA.

14

Table 3. Correlation between amount of HEIF allocation and share of staff in each discipline

% staff in: Correlation with HEIF allocation in

2013/14 Medicine, dentistry & health 0.375 *** Agriculture, forestry & veterinary science -0.116

Biological, mathematical & physical sciences 0.599 *** Engineering & technology 0.468 *** Architecture & planning 0.064

Administrative, business & social studies 0.125 Humanities & language based studies & archaeology -0.082 Design, creative & performing arts -0.413 ***

Education -0.286 ***

Another reason why it may be misleading to use knowledge transfer income as a basis to assess performance is that this variable often depends on institutional characteristics like size, and does not capture only the effectiveness and efficiency of knowledge transfer performance. As mentioned above, research and technology transfer-oriented universities (and, to a lesser extent, universities in the “general” group) have obtained on average higher rewards for their knowledge transfer performance than universities in the “education” and “local” groups, having performed better in terms of income received. If universities in these clusters turned out to be larger, this would suggest that the current system tends to reward universities with larger scale of operations, and not necessarily those that are more successful and more efficient performers of knowledge transfer. To shed some light on this point, we compute the average sizes of universities (in terms of both the number of academic staff and the number of Business and Community engagement staff) for each one of the different categories of HEIF funding. We find a strong positive correlation between both measures of size and the amount of HEIF funding received. So there is some support for the argument that one of the reasons explaining universities’ greater income is purely their larger size in terms of academic staff8.

8 In this paper, we do not deal with the issue of whether it makes sense to reward the knowledge transfer activities of universities that perform them on a larger scale. There could be arguments in favour of this, for example if there were significant economies of scale (that is, if larger institutions were more productive), or if there was evidence that the knowledge transfer activities performed by larger institutions had somehow greater impact. However, while there is some evidence that the impact of research activities increases more than proportionally with institutional size (Katz, 2000), at present we know very little about how the amount and impact of knowledge transfer activities in a broad sense scale with institutional size.

15

Table 4. Average sizes of universities according to HEIF funding received

Academic staff 2010-11 TTO staff 2010-11

zero 115.8 12.16

<500,000£ 226.8667 16.4

500,000£ - 1ML 534.8636 37.96818

1ML - 2ML 695.125 58.02813

2ML - 2,850,000£ 690.5714 91.57143

maximum funding (2,850,000£) 1095.261 114.013

correlation between amount of HEIF allocation and size 0.7521*** 0.5676***

If the performance of universities was measured using a normalized indicator – such as income per unit of staff, or income per unit of activity - the results in terms of relative performance would change, suggesting that the current approach is not always rewarding the most effective performers. If we rank universities in different clusters on the basis of their average incomes earned for different types of activities, normalized by the number of academic staff, we find that universities in the ”Research and TT” cluster, which performed best in terms of HEIF funding, are ranked first for collaborative research, contract research and sale of spinoff shares, but not for the other activities; the ”Education” cluster is ranked first for CPDs, intellectual property, consultancies, and facility and equiprment-related services, while the ”Local” cluster is ranked first for regeneration programmes. The following Figure shows how the ranking of the different clusters would change if performance in different types of knowledge transfer activities was measured in terms of income per academic staff or income per contract, rather than of total income.

Figure 2. Sensitivity of performance ranking to different types of indicators

16

3.2.3. The limitations of income as a performance indicator: A regression analysis

Finally, in order to capture the extent to which institutional strategies and characteristics together impact the universities’ knowledge transfer performance, when measured on the basis of income, we run a regression analysis on the 131 universities.

Table 5. Descriptive statistics of variables used in the regression

Variable Obs Mean Std. Dev. Min Max

Heif allocation 2013 129 2.31 1.74 0.00 5.00 Academic staff FTE 131 542.05 441.16 0.00 1867.00 Knowledge transfer staff 131 50.20 61.55 0.00 337.00 Former Polytechnic 131 0.24 0.43 0.00 1.00 Modern 131 0.32 0.47 0.00 1.00 Red brick 131 0.21 0.41 0.00 1.00 Old 131 0.05 0.21 0.00 1.00 University college 131 0.18 0.39 0.00 1.00 % academic staff in science and medicine 131 0.31 0.25 0.00 1.00 % academic staff in technology 131 0.11 0.11 0.00 0.69 % academic staff in social science 131 0.20 0.15 0.00 0.95 % academic staff in humanities 131 0.38 0.30 0.00 1.00 Governing body: % business 131 0.39 0.14 0.08 0.82 Governing body: % social, community and cultural 131 0.13 0.12 0.00 0.79 Governing body: % public sector 131 0.31 0.18 0.00 0.75 General 131 0.28 0.45 0.00 1.00 Research 131 0.32 0.47 0.00 1.00 Education 131 0.21 0.41 0.00 1.00 Local 131 0.19 0.39 0.00 1.00 Agriculture and mining 131 0.50 0.50 0.00 1.00 Manufacturing 131 0.74 0.44 0.00 1.00 Utilities 131 0.63 0.48 0.00 1.00 KIBS 131 0.88 0.33 0.00 1.00 Other services 131 0.87 0.34 0.00 1.00

1"

2"

3"

4"

Contract"re

search"

Consultancy"

Facili5

es"and

"equ

ipmen

t:related"services"

CPDs"

Contract"re

search"

Consultancy"

Facili5

es"and

"equ

ipmen

t:related"services"

CPDs"

Contract"re

search"

Consultancy"

Facili5

es"and

"equ

ipmen

t:related"services"

CPDs"

Contract"re

search"

Consultancy"

Facili5

es"and

"equ

ipmen

t:related"services"

CPDs"

Research"and"TT" General" Educa5on" Local"

Income"per"academic"staff"

Income"per"contract"

Total"income"

17

Public sector 131 1.00 0.00 1.00 1.00 % income from collaborative research and regeneration programmes 131 0.28 0.21 0.00 1.00 % income from contract research, consultancies and facilities & equipment services 131 0.40 0.24 0.00 0.97

Our dependent variable is the HEIF funding allocation received by the university in 2013; this is an ordinal variable that takes on six possible values9. The independent variables capture some of the main institutional characteristics (size in terms of academic staff and staff employed in knowledge transfer functions, type of university, shares of staff in different disciplines), and some aspects of the institution’s mission (composition of the governing body) and knowledge transfer strategies (whether the university belongs to one of the four clusters identified, what sectors are targeted in knowledge transfer activities, what share of knowledge transfer income derives from collaborative research and regeneration programmes, what share derives from contract research, consultancies and facilities and equipment-related services). The tables below presents some descriptive statistics on the variables used, followed by the regression results. Due to the nature of the dependent variable we estimate an ordered probit model10. The number of observations in the regression is 129 because two English universities are not included in the 2013 HEIF funding allocation list (Universtiy of Buckingham and School of Pharmacy).

The next table presents the regression results. The two columns present two versions of the econometric model, a version that includes all the basic regressors (Model 1) and a version that includes, besides all the basic regressors, also two variables that try to capture the types knowledge transferred, whether it is characterized by large externalities or more appropriable (Model 2).

The results from Model 1 confirm that the amount of HEIF allocation received by an institution, which depends on the income accrued from a range of knowledge transfer activities, is strongly correlated with the size of the institution and the shares of academic staff in non-humanities disciplines. Old universities are more likley to have received higher funding; since we are controlling for size and research orientation, the positive and significant impact of being an old, established university suggests that reputation effects may play a role. The knowledge transfer strategy of the university institution also matters, with institutions oriented to research and technology transfer and institutions engaging with the manufacturing sector being more likely to receive higher funds. Instead, institutions that engage with knowledge-intensive business services are less likely to receive higher funds. This is in line with our earlier argument that some types of knowledge transfer activities are more expensive than others, so that universities that are strongly focused on the humanities and interact more with the service sector command lower income than universities that are more focused on the sciences and interact more with the manufacturing sector, even though the former may be equally engaged in knowledge transfer than the latter, and they may generate similar impacts.

9 The six values are: 0 = zero, 1 = <500,000£, 2 = 500,000£ - 1ML, 3 = 1ML - 2ML, 4 = 2ML - 2,850,000£, 5 = maximum funding (2,850,000£). 10 We obtain similar results if we run a Tobit model using as dependent variable the actual amount allocated by HEIF to each institution in 2013 (the allocations are comprised between zero and £2,850,000), and the same independent variables.

18

In Model 2, we add two variables capturing the nature of the knowledge transferred by the university, whether it is characterized by large externalities (proxied by the share of income accruing from more ”open ended” and risky forms of research, i.e. collaborative research and regeneration programmes) or whether it is more appropriable in nature (proxied by the share of income accruing from more appropriable forms of research such as contract research, consultancy, facilities and equipment-related services). We find that the latter has a significantly positive effect on income (no effect on the sign and significance of the other variables). This may suggest that businesses are more willing to pay a higher price for forms of knowledge that are more appropriable.

Hence, income from knowledge transfer, which in the UK is used as the key measure of performance, is affected by factors that are not indicative of performance as such, including: the university’s knowledge transfer objectives, its disciplinary orientation and the economic sectors it interacts with, its size, its reputation and the type of knowledge it transfers (with more appropriable forms of knowledge commanding higher income, but it could be argued that more open-ended forms of knowledge, that generate large externalities, have potentially greater impacts in the long run).

Table 6. Regression results Model 1: Model 2:

VARIABLES HEIF allocations

2013 HEIF allocations

2013 Academic staff FTE 0.002*** 0.002*** (0.000) (0.001) Knowledge transfer staff 0.014*** 0.014*** (0.003) (0.003) Former Polytechnic -0.181 -0.013 (0.546) (0.579) Modern 0.244 0.461 (0.463) (0.494) Red brick 0.734 0.751 (0.482) (0.511) Old 1.572* 1.891*

(0.953) (1.028)

% academic staff in science and medicine 1.224* 0.759 (0.654) (0.685) % academic staff in technology 2.628** 2.511** (1.254) (1.265) % academic staff in social science 1.696* 2.045** (0.922) (0.941) Governing body: % business 0.175 0.594 (1.001) (1.016) Governing body: % social, community and cultural -0.738 -0.352

(1.130) (1.128)

Governing body: % public sector 0.973 1.212*

(0.695) (0.710)

General -0.426 -0.446 (0.329) (0.335) Research & TT 0.911** 0.763** (0.354) (0.365) Education -0.368 -0.300 (0.366) (0.370) Agriculture and mining 0.036 -0.004 (0.263) (0.266) Manufacturing 0.718* 0.716*

19

(0.416) (0.421)

Utilities 0.234 0.295 (0.323) (0.331) KIBS -1.438** -1.274** (0.613) (0.648) Other services 0.395 0.245

(0.450) (0.465)

% income from collaborative research and regeneration programmes 0.659 (0.687) % income from contract research, consultancies and facilities & equipment services 1.928*** (0.623) Observations 129 129 Standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

4. Conclusion: the limitations of current indicators

Knowledge transfer activities are many and varied, and universities have different profiles of knowledge transfer engagement. When choosing the set of indicators for the measurement of universities’ knowledge transfer performance, care must be taken in adopting a sufficiently broad approach, because the reliance upon indicators informed by a narrow view of the knowledge transfer process limits the ability of universities to accurately represent their engagement in knowledge transfer and the impact of their activities.

We have illustrated these points with reference to the HE-BCI survey implemented in the United Kingdom, where we have shown that the choice of indicators, while broader than in other surveys (Rosli and Rossi, 2014), is still not sufficiently extensive: many indicators focus on the transfer of intellectual property rights (in particular emphasizing patents and software licenses, and overlooking other important types of intellectual assets), a few indicators capture the transfer of knowledge through open dissemination (but only in relation to the funding attracted to the university and not to the knowledge outputs generated) and some capture the transfer of knowledge through interactions, but not all possible types are included. Moreover, the indicators are almost exclusively output-oriented, and mainly relate to the income from knowledge transfer accrued to the university. We have then argued that income as an indicator is inadequate to capture the impact of universities’ knowledge transfer activities, supporting this argument with some empirical evidence relating to 131 universities in England. If the objective of the assignment of knowledge transfer funding is to reward broad, effective and efficient knowledge transfer, rather than the institutions’ ability to focus on remunerative activities and achieve a large scale of operations, it would be more appropriate to consider more composite ways to assess performance rather than purely income-based measures. In order to choose indicators that are not biased in favour of institutions that adopt specific knowledge transfer strategies, a possible approach could be to recognize that institutions are different and may require different sets of indicators. An alternative approach could be to develop a very broad range of indicators representing all possible activities, and allow universities themselves to choose the profile of knowledge transfer engagement that suits them best (adopting a flexible approach to measurement as suggested, in the more general case of innovation policy indicators, by Rafols et al., 2012). Some of the indicators included should be directed at capturing qualitative aspects of the process of knowledge transfer, and the feedback effects of knowledge transfer activities on universities.

20

Aggregating indicators is also a complex problem, once we move away from simple indicators based on income. It would be unwise to try and derive aggregate measures of institutional performance when the units of measurement are not comparable, or when the measured activities are partly substitute or complementary with one another (Bonaccorsi and Daraio, 2008), and also might be different across economic regions. The need for more flexible ways of aggregating performance indicators has been acknowledged with respect to general indicators of innovation (Stirling, 2003; Grupp and Schubert., 2010) and also to indicators of universities’ performance, where some authors have argued in favour of multidimensional measurements (F. A. van Vught and F. Ziegeleeds, 2012) and positioning indicators (Bonaccorsi and Daraio, 2008).

Finally, the choice of areas and indicators and the use of a single formula to reward knowledge transfer performance may have powerful performative effects, inducing universities to conform to a single model of knowledge transfer (for example, skewed towards research commercialisation and income-producing activities, disregarding the full spectrum of knowledge transfer activities), reducing variety and possibly negatively affecting the amount and quality of knowledge transfer that takes place.

Since the system of knowledge transfer performance indicators adopted in the UK is widely considered as one of the most comprehensive, these general problems are very likely to affect even more strongly other measurement systems that focus on a narrower range of activities (such as technology commercialization or spinoff activities, in the case of the surveys promoted by AUTM in the US and Canada, and by other associations of knowledge transfer professionals in Europe and elsewhere). With the UK being one of the first countries in the world to have launched a comprehensive data collection exercises on universities’ knowledge transfer activities, many countries in South Asia and Europe are having the contention in implementing a similar exercise. Care needs to be taken when emulating the data collection exercises, as selecting appropriate indicators is the most critical aspect of performance measurement process, and policy makers should consider institutional identity as a possible contingency variable in determining knowledge transfer performance. In institutional sectors like academia, the definition of a “successful” universitity in knowledge transfer performance is open to substantial social construction and heavily weighted toward identity-related attributes. References

Abreu, M., Grinevich, V., Hughes, A., Kitson, M. and Ternouth, P. (2008). Universities, Business and Knowledge Exchange. Council for Industry and Higher Education and Centre for Business Research, London and Cambridge.

Andersen, B. (2004). If “Intellectual Property Rights” is the Answer, What is the Question? Revisiting the Patent Controversies. Economics of Innovation and New Technology. 13 (5). p.pp. 417-442.

Andersen, B., Rosli, A., Rossi, F. and Yangsap, W. (2012). Intellectual Property (IP) Governance in ICT Firms: Strategic Value Seeking through Proprietary and Non-Proprietary IP Transactions. Int. J. Intellectual Property Management. 5 (1).

Andersen, B. and Rossi, F. (2012). Inefficiencies in Markets for Intellectual Property Rights: Experiences of Academic and Public Research Institutions. Prometheus. 30 (1). p.pp. 5-27.

21

Antonelli, C. (2005). Models of Knowledge and Systems of Governance. Journal of Institutional Economics. 1 (1). p.pp. 51-73.

Antonelli, C. (2006). The Governance of Localized Knowledge: An Information Economics Approach to the Economics of Knowledge. Industry and Innovation. 13 (3). p.pp. 227-261.

Antonelli, C. (2008), The new economics of university: A knowledge governance approach, Journal of Technology Transfer, 33: 1-22.

Arrow, K. (1962). Economic Welfare and the Allocation of resources for invention. In: Richard Nelson (ed.). The Rate and Direction of Inventive Activity. Princeton, NJ: Princeton University Press, pp. 609-25.

Arundel, A. and Geuna, A. (2004). Proximity and the use of Public Science by Innovative European Firms. Economics of Innovation and New Technology. 13 (6). p.pp. 559-580.

Baghurst, D. and Pollard, T. (2009). A Literature Review on the Efficiency and Effectiveness of University Intellectual Property (IP) Models for the Generation, Identification and Exploitation of “Soft” (Non-Patent and Non-Trademark) IP. SABIP Report.

Bekkers, R. and Bodas Freitas, I.. (2008). Analysing Preferences for Knowledge Transfer Channels between Universities and Industry: To what Degree do Sectors also Matter? Research Policy. 37. p.pp. 1837-53.

Boardman, P.G. and Ponomariov, B.L. (2009). University Researchers Working with Private Companies. Technovation. 29. p.pp. 142-153.

Bonaccorsi, A. and Daraio, C. (2008). The Differentiation of the Strategic Profile of Higher Education Institutions. New Positioning Indicators Based on Microdata. Scientometrics. 74 (1). p.pp. 15-37.

Boschma, R. (2005). Proximity and Innovation: A Critical Assessment. Regional Studies. 39 (1). p.pp. 61-74.

Brodhag, C (2013). Research Universities, technology transfer, and job creation: What insfrastructure, for what training? Studies in Higher Education. 38(3)p.pp. 388-404

Brouwer, E. and Kleinknecht, A. (1999). Innovative Output, and a Firm’s Propensity to Patent. An Exploration of CIS Micro Data. Research Policy. 28. p.pp. 615-24.

Bruneel, J., D’Este, P., Neely, A. and Salter, A. (2009). The Search for Talent and Technology. AIM research paper. (Imperial College London.).

Cohen, W., Nelson, R.R. and Walsh, J. (2000). Protecting Their Intellectual Assets: Appropriability Conditions and Why U.S. Manufacturing Firms Patent (or Not).

Cohen, W.M., Nelson, R.R. and Walsh, J.P. (2002). Links and Impacts: The Influence of Public Research on Industrial R&D, Management Science. 48 (1). p.pp. 1-23.

Cowan, R., David, P.A. and Foray, D. (2000). The Explicit Economics of Knowledge Codification and Tacitness. Industrial and Corporate Change. 9 (2). p.pp. 211-253.

Cowan, R. and Van der Paal, G. (2000). Innovation Policy in a Knowledge-based Economy,. Publication EUR 17023 of the Commission of the European Communities. Luxembourg.

22

Dasgupta, P. and David, P.A. (1994). Toward a new economics of science G. B. A. S. Kuhlmann (ed.). Research Policy. 23 (5). p.pp. 487-521.

Davis, K.E., Kingsbury, B. and Merry, S.E. (2010). Indicators as a Technology of Global Governance. [Online]. 2010. Available from: SSRN. http://ssrn.com/paper=1583431.

Department for Business Innovation and Skills, UK, 2008

Dosi, G., Llerena, P. and Labini, M. (2006). The relationships between science, technologies and their industrial exploitation: An illustration through the myths and realities of the so-called ‘European Paradox. Research Policy. 35 (10). pp. 1450-1464.

Dutrénit, G., De Fuentes, C. and Torres, A. (2010). Channels of interaction between public research organisations and industry and their benefits: evidence from Mexico. Science and Public Policy. 37 (7). p.pp. 513-526.

D’Este, P. and Patel, P. (2007). University-industry linkages in the UK: what are the factors underlying the variety of interactions with industry? Research Policy. 36 (9). p.pp. 1295-1313.

Etzkowitz, H and Leydesdorff , L (2000) The dynamics of Innovation:From National Systems and "Mode 2" to a Triple Helix of University-Industry-Government relations, Research Policy, 29:1098-123

Di Gregorio, D. and Shane, S. (2003). Why do some universities generate more start-ups than others? Research Policy. 32 (2). p.pp. 209-22

Fonbrun, J. C (1995). Reputation: Realizing Value from the Corporate Image. Boston: Harvard Business Press Books:

Grupp, H. and Schubert., T. (2010). Review and New Evidence on Composite Innovation Indicators for Evaluating National Performance. Research Policy. 39 (1). p.pp. 67-78.

HEFCE (2012). Strengthening the Contribution of English Higher Education Institutions to the Innovation System: Knowledge Exchange and HEIF Funding, Available at: http://www.hefce.ac.uk/media/hefce/content/whatwedo/knowledgeexchangeandskills/heif/pacec-report.pdf [Access: 8th April 2013]

HEFCE (2011). Opportunity, choice and excellence in higher education. Bristol.

Harabi, N. (1995). Appropriability of Technical Innovations: An Empirical Analysis. Research Policy. 24. p.pp. 981–992.

Haskel, J and Wallis, G (2013) Public Support for Innovation, Intangible investment and Productivity Growth in the UK Market Sector, Economics Letters, 19(2):195-198

Holi, M.T., Wickramasinghe, R. and Leeuwen, M. van (2008). Metrics for the Evaluation of Knowledge TransferActivities at Universities. Cambridge: Library House.

Hughes, T., Bence, D., Grisoni, L., O’Regan, N. and Wornham, D. (2011). Scholarship that matters: academic/practitioner engagement in business and management. Management Learning. 10 (1). p.pp. 40-57.

Jensen, P.H., Palangkaraya, A. and Webster, E. (2009). A Guide to Metrics on Knowledge Transfer from Universities to Businesses and Industry in Australia.

23

Jensen, R., Thursby, J. and Thursby, M.C. (2010). University-Industry Spillovers, Government Funding, and Industrial Consulting. NBER Working Papers 15732, Cambridge. MA: National Bureau of Economic Research Inc.

Katz, J.S. (2000). Scale-independent indicators and research evaluation. Science and Public Policy 27(1) pp. 23-36

Kitagawa, F and Lightowler, C (2012) Knowledge Exchange: A comparison of Policies, strategies, and funding incentives in English and Scottish Higher Education,Research Evaluation (2012) p.pp. 1-14

Klein Woolthuis, R., Lankhuizen, M. and Gilsing, V. (2005). A system failure framework for innovation policy design. Technovation. 25 (6). pp. 609-619.

Lambert, R. (2003) Lambert Review of Business-University Collaboration, HM Treasury.

Lawton Smith, H. and Ho, KW (2006) Measuring the performance of Oxfortshire's spin-off companie. Research Policy, 35(10) p.pp.1554-1568

Liebenau, J (1985) Innovation in Pharmaceuticals: Industrial R&D in the early twntieth century. Research Policy, 14(4) p.pp.179-187

Levin, R.C., Klevorick, A.K., Nelson, R.R. & Winter, S.G. (1987). Appropriating the Returns from Industrial Research and Development. Brookings Papers on Economic Activity. 1987 (3). pp. 783-831.

Litan, R.., Mitchell, L. and Reedy, E.J. (2008). Commercializing University Inventions: Alternative Approaches. In: A. Jaffe, J. Lerner, and S. Stern (eds.). Innovation Policy and the Economy. pp. 31-57.

Lundvall, B.-Å. (1988). Innovation as an interactive process: from user-producer interaction to the national system of innovation. In: G. et al Dosi (ed.). Technical change and economic theory. London: Pinter Publishers, pp. 349–369.

MacGarvie and Furman, 2005Mansfield, E. (1986). Patents and Innovation: An Empirical Study. Management Science. 32 (2). p.pp. 173-181.

Mazzoleni, R. and Nelson, R.R. (1998). The Benefits and Costs of Strong Patent Protection: A Contribution to the Current Debate. Research Policy. 27 (3). p.pp. 273-284.

Merry, S.E. (2011). Measuring the World: Indicators, Human Rights, and Global Governance: With CA Comment by John M. Conley. Current Anthropology. 52 (3). p.pp. 83-95.

Meyer-Thurow, G (1982) The Industrialization of invention: A case study form the German chemical Industry, ISIS 73 p.pp. 363-81

Molas Gallart, J and Castro-Martinez, E. (2007) Ambiguity and Conflict in the development of 'Third Mission indicators, Research Evaluation. 16(4) p.pp 321-330

Mowery, D. and Sampat, B. (2005). The Bayh-Dole Act of 1980 and University-Industry Technology Transfer: A Model for other OECD Governments? The Journal of Technology Transfer. 30. p.pp. 115-127.

Nelson, R.R. (1959). The Simple Economics of Basic Scientific Research E. Mansfield and E. Mansfield (eds.). Journal of Political Economy. 67 (3). p.pp. 297-306.

24

Nelson, R.R. and Winter, S.G. (1982). An evolutionary theory of economic change. B. Press (ed.). Harvard University Press.

Nooteboom, B. (2004). Inter-firm collaboration, learning and networks. An integrated approach. London and New York: Routledge.

PACEC, (2010) Synergies and Trade-offs between Research, Teaching and Knowledge Exchange, A Report to HEFCE by PACEC and the Centre for Business Research, University of Cambridge

Polanyi, M. (1966). The tacit dimension. New York: Doubleday.

Rafols, I., Ciarli, T., van Zwanenberg, P. and Stirling, A. (2012). Towards Indicators for “Opening up” Science and Technology Policy.

Roberts, P and Dowling, G. (2002) Corporate reputation and sustained superior financial performance, Strategic Mangement Journal, 23(12)p.pp. 1077–1093

Romer, P (1990) Human capital and growth: Theory and evidence, Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 32(1), p.pp. 251-286

Rosli, A., Rossi, F. (2014) What indicators to assess universities’ knowledge transfer performance? Problems in the transition from theory to practice, In Hilpert, U. (ed.) ‘Handbook on Politics and Technology’, Routledge, forthcoming.

Ryle, G. (1949). The Concept of Mind. London: Hutchinson.

Schacht, W. (2005) The Bayh-Dole Act: Selected Issues in Patent Policy and the Commercialization of Technology, CRS Report for US Congress.

Sellenthin, M.O (2006) Beyond the ivory tower: Do patent rights regimes impact on patenting behaviour in Sweden and Germany, VEST Journal of Science and Technology Studies, 19(3-4) p.pp. 27 – 58

Stirling, A. (2003). Risk, Uncertainty and Precaution: Some Instrumental Implications from the Social Sciences. In: I. Scoones, M. Leach, and F. Berkhout (eds.). Negotiating Change: perspectives in environmental social science. London: Edward Elgar, pp. 33-76.

Thursby, J., Jensen, R., Thursby, M. (2001) Objectives, Characteristics and Outcomes of University Licensing: A Survey of Major U.S. Universities, Journal of Technology Transfer, 26: 59-72.

van Vught, F. A. and F. Ziegele (eds.) (2012). Multidimensional Ranking. The Design and Development of U-Multirank. Higher Edu. Berlin: Springer.

Wang, C.F. and Peng, Z. (2009), "An empirical study on the relationship between properties of knowledge, network topology and corporation innovation performance", , Vol. 2, pp. 1230-7.

Wright, M., Clarysse, B., Lockett, A. and Knockaert, M. (2008). Mid-range universities’ linkages with industry: Knowledge types and the role of intermediaries. Research Policy. 37 (8). p.pp. 1205-1223.