Top Banner
DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004 A study prepared pursuant to the Memorandum of Agreement between The Development Bank of Southern Africa Limited and the United Nations University Institute for New Technologies (UNU-INTECH), 2004
79

DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Jun 26, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD

A Study prepared by UNU-INTECH

Maastricht, UNU-INTECH, 31 August 2004

A study prepared pursuant to the Memorandum of Agreement between The Development Bank of Southern Africa Limited and the United Nations University Institute for New Technologies (UNU-INTECH), 2004

Page 2: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004
Page 3: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Acknowledgments The present study was prepared by a team of researchers under the direction of Professor Lynn K. Mytelka, Director UNU-INTECH. Members of the team included Norman Dytianquin (UNU-INTECH), Geoffrey Gachino (Kenya Central Bureau of Statistics/UNU-INTECH), Dr. Micheline Goedhuys, (University of Antwerp/ UNU-INTECH), Prof. Banji Oyeyinka (UNU-INTECH), Dr. Saeed Parto (MERIT/University of Maastricht/UNU-INTECH) and Anthony Arundel (MERIT/University of Maastricht) We would like to thank Peter Loewe (EU, DG Enterprise), Fadzilah Ahmad Din, (MASTIC, Malaysia) Paul Crowley (EU, Eurostat), Renato Baumgratz Viotti (IBGE, Brazil), Mariana Martins Reboucas (IBGE, Brazil), Monica Salazar (Colombian Observatory of Science and Technology and Simon Fraser University), Adi Paterson (DST, South Africa), Simon Mpele (DST, South Africa), William Blankley (HSRC, South Africa), L.A.G. Oerlemans (University of Tilburg & University of Pretoria, South Africa), Ernesto Fernandez Polcuch (UNESCO Institute of Statistic) and John Mugabe (NEPAD S&T Secretariat) whose thoughtful comments, observations, analyses and insights have enriched this study.

Page 4: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004
Page 5: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Table of Contents

Acknowledgements 3 Executive Summary 9 Chapter One The Evolution and Applicability of the Concept of a National System of Innovation to Developing 19 Countries

1.0 Introduction 19 2.0 Understanding Innovation: A framework for analysis 19 3.0 Innovation Policies and Innovation Surveys:

An evolutionary process 23 4.0 Innovation and Development 28

Chapter Two The Evolution, Organization and Conduct of Innovation Surveys 31 1.0 Introduction 31 2.0 Emergence and Evolution of Innovation Surveys 31 2.1 The Output Approach 33 2.2 The Activity Approach 34 2.3 The Innovation Systems Approach 36 3.0 Overview of Innovation Surveys 36 3.1 Western European Countries 36 3.2 Central and Eastern European Countries 37 3.3 Other Non-European OECD Countries 38 3.4 Innovation Surveys in Developing Countries 39 4.0 Comparison of Innovation Surveys 47 4.1 Organization of the Survey 47 4.2 Survey Modalities 50 4.3 Sampling Technique and Sampling Frame 52 4.4 Population, Sample and Response Size 53 4.5 Sectoral Scope 56 4.6 Firm Size 59 5.0 Questionnaire Content 61 5.1 General Information Questions 62 5.2 Questions on Science and Technology and R&D Measures 62 5.3 Innovation Questions 74 6.0 Other Relevant Surveys in Africa 74 6.1 Experience with Industrial Surveys 77 6.2 Experience with RPED Surveys 78

Page 6: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Chapter Three Creating a Policy-Relevant Innovation Survey for Africa 83 1.0 Introduction 83 2.0 Lessons from earlier Innovation Surveys 83

2.1 Creating an Innovation System-Oriented Survey 83 2.2 Complementing Firm-level Innovation Data with 84

Aggregate Science & Technology and R & D Indicators 2.3 Assessing Trends 85 2.4 Providing Feedback for Adaptive Policy-Making 85 2.5 Balancing the Need for Policy-Relevant Information 86

against the Problem of an Overly Long Questionnaire 3.0 Using Innovation Surveys 87 4.0 Choices in the design of a Policy-relevant Innovation

Survey for Africa 91 4.1 Scope of the Sample 92 4.2 Sampling Methodology 92 4.3 Tailoring Questions to the Local Context 93 5.0 Enhancing the Policy Relevance of the Survey 93 Instrument 5.1 Conceptualizing Innovation 94 5.2 Utility for Regional Development Policies 94 5.3 Profiling the Innovative Firm 95 5.4 Enabling Policy-makers to better understand the 96 Innovative Behaviour of Firms and its Relationship to the Performance of Firms 5.5 Contributing to the Design of Innovation-Related 97 Policies 6.0 Proposed Questionnaire 97

Chapter Four Implementing Innovation Surveys in Africa 99 1.0 Introduction 99 2.0 The Interface between S & T and Innovation Surveys 99 3.0 The African Working Group 102 4.0 An Innovation Survey Process in Africa 103 Appendix 1 Innovation Questionnaire for NEPAD 112 Appendix 2 Training Modules: An Illustrative example 127 References 131

Page 7: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Tables

Table 1. Features of Innovation Surveys of Developed and Developing Countries 43 Table 2. Comparison of Innovation Survey Questions 63 Table 3. African Countries in which UNIDO, World Bank RPED and ICA

Surveys have been undertaken and for which S&T statistics were reported to UNESCO (most recent year) 75

Table 4. Overview of RPED Firm Surveys 79 Table 5. Overview of RPED Investment Climate Assessments 80

Figures Figure 1. Innovation Systems 21

Charts

Chart 1. Count of Innovation Surveys: Pre-Oslo Manual Period 32 Chart 2: Count of Countries that Conducted Innovation Surveys in the Post-

Oslo-Manual Period 33 Chart 3. Agency Carrying Out Innovation Surveys 48 Chart 4. Frequency of Survey and Participation Type 49 Chart 5. Reference Years for the Innovation Surveys 50 Chart 6a. Survey Modalities of the European-17 Under the

Community Innovation Surveys 51 Chart 6b. Survey Modalities of the Other OECD and Non-OECD Countries 51 Chart 7. Sampling Frame Used in Innovation Surveys 53 Chart 8a. Population, Sample and Response Sizes of Innovation Surveys By OECD Countries 54 Chart 8b. Population, Sample and Response Sizes of Innovation Surveys By Non-OECD Countries 54 Chart 9. Response Rates in Innovation Surveys 55 Chart 10. Sectoral Coverage of Innovation Surveys in Countries

With Subsequent Rounds 57 Chart 11a. Cut-off Points for Firms Sizes in Non-OECD Countries 60 Chart 11b. Cut-off Points for Firm Size in OECD Countries 60 Chart 12. Timetable and Tasks for the Pilotphase 109

Page 8: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Chapter Two

The Evolution, Organization and Conduct of Innovation Surveys 1.0 Introduction The following chapter provides an overview of the emergence and evolution of innovation surveys in developed and developing countries, their conceptualization, scope, design and content. The latter includes a comparison of innovation survey questionnaires among a number of countries that have done them. Among these, South Africa is presently the only country in Africa to have conducted an innovation survey. To broaden our understanding of surveys in Africa, the final section thus provides an analysis of industrial surveys in African countries including those done through the World Bank’s RPED programme. 2.0 Emergence and Evolution of Innovation Surveys

This section discusses the emergence and evolution of innovation surveys, their scope, conduct and organization, and how these have changed through time by countries that developed subsequent rounds of the survey. Overall, a total of 51 countries have carried out innovation surveys as shown in Table 1. Of these, 21 are developing or non-OECD countries, 5 from Europe (Russia, Bulgaria, Rumania, Slovenia and former Yugoslavia); 4 from Asia (Taiwan, Singapore, Malaysia and Thailand); 1 from Africa (South Africa) and the majority from Latin America.

Most developing countries have patterned their innovation surveys after the innovation

surveys of developed countries, specifically the Community Innovation Survey of European countries, now on its fourth round, which are based on common guidelines stipulated under the Oslo Manual. While the USA was the very first country to develop an innovation survey in the 1960s, the European Community Innovation Survey was the first attempt to harmonize or standardize the innovation survey questions, thus becoming the model for other countries to follow. Latin American countries though developed their own manual called the Bogotá Manual, which tailored some Oslo Manual-type questions asked in the innovation surveys to the peculiarities of innovation in Latin America.

The experiences with innovation surveys by country vary and for those countries that

have carried out subsequent rounds of the innovation survey, the questions and content and scope of the innovation surveys as well as the methodology and sampling procedures have been modified. Three periods can be distinguished in terms of the questionnaire design, scope and conduct of innovation surveys. The first, a pre-Oslo Manual period, consisted of individualized surveys, which in the literature search were carried out by eight countries, namely the USA, Germany, France, Italy, the Netherlands, Norway, Sweden and Uruguay (see Chart 1). In these surveys, innovation was conceptualized in terms of outputs and measured by indicators such as patents and R&D expenditures (see Godin, 2002a). During this time, many countries have collected patent statistics and data on R&D expenditures but both indicators are not a sufficient measurement of innovation. Patents actually measure inventions whereas R&D expenditures are

Page 9: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

only a portion of innovation costs that includes product design, market testing, trial production, and investments in new machinery and equipment.

With the advent of the Oslo Manual in 1992 where harmonized guidelines in innovation

questionnaires were introduced, and following Pavitt’s suggestion in an OECD report (1976), the measurement of innovation was expanded in the second period as an activity (i.e., percentage of a firm’s activity devoted to innovation) and as an output (number of significant new products and processes produced by the firm). This came to be known as the subject (innovating firm) and object (innovations introduced by the firm) approaches to innovation surveys. There was a tendency though to focus more on the results of innovation because of the bias introduced by the output approach. However, an increasing awareness evolved of the need for innovation surveys to address issues of what makes a firm innovate, why they innovate and how they innovate.

The third phase could be characterized as a focus towards innovation as an interactive

process in which a variety of critical actors engage in the exchange of knowledge and information as part of the stimulus to innovation. In this phase, the innovation framework was further developed and applied in policy-making with questions introduced concerning linkages and collaboration, knowledge and information flows, and qualitative performance and impact and the role of government. The scope was also widened beyond the manufacturing industry to cover resource-based and services sectors. Innovation surveys were thus basically carried out to achieve four main goals: (i) to measure inputs and outputs of the innovation process across a wide range of firms and industries; (ii) to acquire an overview of the innovative behaviour of enterprises; (iii) to develop policy and support analysis in the area of innovation; and (iv) to benchmark innovation performance against some best practice standard or reference that could either be a firm, industry, country, or region.

Chart 1. Count of Innovation Surveys: Pre-Oslo Manual Period

1960 1965 1970 1975 1980 1985 1990 1992

Page 10: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Throughout its entire history, a total of 51 countries carried out innovation surveys, of which 30 are OECD or developed countries and 21 are non-OECD or developing countries. Moreover, close to two-thirds or 33 countries have done subsequent rounds of the survey, of which 23 are OECD countries (17 European member states, Hungary, Mexico, Poland, South Korea, Canada, and Australia) and 10 are non-OECD countries, 8 of which are developing nations notably Argentina, Brazil, Chile, Colombia, Uruguay, Malaysia, Thailand, and South Africa. The other two non-OECD countries that have conducted subsequent rounds are Russia and the former Yugoslavia (Chart 2).

Chart 2: Count of Countries that Conducted Innovation Surveys in the Post Oslo-Manual Period

OECD Countries

0 1 2 3 4 5

Canada Belgium

France Germany

Greece Ireland

Italy Luxembourg Netherlands

Norway Poland

Portugal Spain

UK South Korea

Mexico Australia

Austria Denmark

Finland Hungary

Iceland Sweden

Japan Czech Republic

New Zealand Slovak Republic

Switzerland Turkey

USA

Non-OECD Countries

0 1 2 3 4 5

Chile Malaysia

Colombia Russia

South Africa Brazil

Thailand Uruguay

Yugoslavia Argentina

Bulgaria Cuba

Ecuador Panama

Paraguay Peru

Romania Singapore

Slovenia Taiwan

Venezuela

2.1 The Output Approach

Innovation surveys first emerged as a measurement of the outputs of innovation with the use of patents as an indicator of innovation following Schmookler (1950, 1953) and R&D expenditures following Holland and Spraragen (1933). Only three countries conducted surveys based on the output approach: the USA (3 surveys), Canada, and Australia. The first innovation survey using the output approach is attributed by Godin (2002) to the USA when the US National Science Foundation (NSF) attempted to identify, count and characterize firms that commercialized technological innovations following the recommendations of an earlier study conducted by Little and Mansfield (1963) and the TRACES (1968) study on the link between science and technology and the innovative process. Under the supervision of Myers et al (1967, 1969), the first NSF innovation survey was conducted by the US National Planning Association

Page 11: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

11

between the years 1963 and 1967 with results published in 1969. The survey covered 121 firms in five manufacturing industries and analyzed 567 mostly minor or incremental technical innovations and described the characteristics of firms, sources of innovation, nature of innovation (product or process), their costs and impact on firm performance. This was followed by a second survey in 1974 commissioned by Gellman Research Associates (1976) using the same methodology but focusing on radical product innovations that were introduced over the period 1953-1973. The purpose of the second survey, results of which were included in Science Indicators of the NSF (1975 and 1977), was to determine the time lag between invention and innovation, the rate of returns on investments, the radicalism of the innovations and the size and R&D intensity of firms that produced them. The first involvement of the OECD with measuring innovation as an output commenced in 1968 when it published a report Gaps in Technology that underscored innovation performance as distinguishing American from European firms, where innovation was defined as performance in being first to commercialize new products and processes, and also in terms of the level and speed in the use or application (i.e., diffusion) of these new products and processes. Using data covering 140 significant innovations since 1945, the report found that American firms have commercialized about 60 percent of fundamental research and inventions originating from Europe but only few cases haven been found of the reverse. This study was followed through by Pavitt and Wald in an OECD Report (1971) analyzing the country of origin of 110 of the most significant innovations identified in the Gaps report, with the following results: USA being country of origin of 67 percent of the innovations, the United Kingdom and Germany representing 16 percent and 13 percent respectively. Other official efforts to measure innovation under the output approach were the US Small Business Administration’s study of 8,074 innovations that were commercially introduced in the USA in 1982, results of which were published by Edwards and Gordon (1984); Also irregular surveys on technology diffusion were conducted in the USA1, Canada2 and Australia3. From then on, the activities approach for measuring innovation became the standard, owing to the publication of the Steacie Report by the US Department of Commerce (1967). 2.2 The Activity Approach The Steacie Report defined and measured innovation in terms of five categories of activities, namely: R&D, design engineering, tooling and engineering, manufacturing, and marketing. It also discovered that only 5-10 percent of R&D expenditures accounted for innovation costs, which proved that it was not an accurate indicator of innovative activity. The Steacie Report thus influenced subsequent innovation surveys carried out worldwide with Canada credited to be the first country to use this approach in the early 1970s. In 1971 Statistics Canada carried out the regular industrial R&D survey but added questions on how much the 97 firms covered in the survey spent on innovation activities in general in accordance with the classification of the Steacie Report.

1 See various editions of Science and Engineering Indicators by the NSF (1991, 1996, 1998, 2000). 2 See Statistics Canada (1989) and Fortier, Ducharme and Gault (1993). 3 See Pattinson (1992)

Page 12: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

12

Innovation surveys that followed the activity approach have been done even prior to the European Community Innovation Survey by Germany4 as early as 1979, the Netherlands in 19845, Italy6 in 1988, Norway in 19897, Sweden in 19908, and France in 19919. In 1985, the NSF surveyed innovation activities of 620 manufacturing firms using the activity approach for the first time, which was repeated in 1993 by the Census Bureau in a pilot study of 1,000 firms in 1990-1992 as part of the OECD/European Community Innovation Survey I (CIS I).

The OECD’s involvement in innovation surveys was actually spurred by the first regional effort to collect data on innovation activities by the Scandinavian countries, under the sponsorship of the Nordic Fund for Industrial Development. The workshop resulting from this project developed a conceptual framework for formulating innovation indicators, which was revised in Oslo and presented to the OECD Group of National Experts on Science and Technology Indicators (NESTI) in 1989. The resulting manual, which the OECD decided to adopt, thus came to be known as the Oslo Manual. It purports to harmonize national methodologies and collect standardized information on innovation activities of firms, and represents the second10 standardized survey of its kind in the history of science and technology measurement after the international R&D survey (which follows the Frascati Manual) introduced by the OECD in 1963. In 1992, after the Oslo Manual was finalized, the OECD commissioned Eurostat to draft a standard questionnaire that would, based on results of the data collected, allow international comparisons of innovation surveys in Europe. The resulting output came to be known as the Community Innovation Survey because it became a standard questionnaire for members of the EU Community. The design of the Community Innovation Survey was strongly influenced by Kline and Rosenberg’s (1986) chain-link model of innovation as opposed to the pipeline model of innovation. The latter implied a linear progression from research to invention to innovation and diffusion of new technologies and treated R&D as a basic input for innovation in general. The chain-linked model conceptualizes innovation in terms of interaction between market opportunities and the firm’s knowledge base and capabilities, and feedbacks between earlier and later stages of the innovation function. The focus is therefore not only on R&D but also on non-R&D inputs such as design, prototype development and market testing.

4 See Scholz (1992) 5 See Kleinknecht and Reijnen (1992) 6 See Cesaratto and Sirilli (1992) 7 See Smith and Vidvei (1992) 8 See Deiaco (1992) 9 See Auzeby and Francois (1992) 10 A third manual called the Canberra Manual was developed in 1994 to standardize the measurement and development of coherent systems of monitoring the stocks and flows of human resources (scientists and engineers and technological personnel).

Page 13: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

13

2.3 The Innovation Systems Approach Experience with innovation surveys led to the realization that the two building blocks required to understand the innovation process are the firm itself as the creator and manager of knowledge and the national innovation system as the provider of the environment and resources needed for the creation of this knowledge. The first Community Innovation Survey was basically designed to filter out innovating from non-innovating firms in the manufacturing sector and collect information on the types of innovation created, their innovation intensity, costs of innovation beyond R&D, objectives, obstacles and sources of technological knowledge. However, the increased distribution of innovative activity between firms and across networks has led to further refinements in questionnaire design in the CIS to include questions about research cooperation and collaboration. The role of national governments as part of the innovative process has also been recognized in terms of strengthening S&T infrastructure as improving the regulatory framework and institutional conditions affecting innovation. Hence, questions on policy impact of government programs have also been added, particularly by developing countries that have begun to conduct innovation surveys notably Malaysia, Chile, Colombia, Brazil, Argentina and South Africa. The emergence of the knowledge-based and learning economy also made it important to extend the scope and coverage of the survey to the services sector, and recently to resource-based sectors, both of which have evolved to become a heterogeneous highly-innovative segment from the non-innovative homogeneous mass they were before. This has introduced further complications into survey design in terms of comparability of innovation between sectors, calling into question the delimiting definition of innovation itself in the Oslo Manual as purely technological product and process innovation to include organizational and marketing innovation, informal as against formal R&D, etc. The major challenge though for the future design of innovation surveys in the innovation systems approach is to enhance the policy relevance of innovation surveys. This becomes more significant in the light of increasing pressure to remain competitive in the world economy, with a demand-side effect from policymakers asking their advisers and researchers to supply summary measures of their country’s relative national innovation positions.

3.0 Overview of Innovation Surveys 3.1 Western European Countries

The first Community Innovation Survey (CIS)11 covering 40,817 firms of 13 European

countries (Belgium, Denmark, France, Germany, Greece, Ireland, Italy, Luxembourg, Norway, Portugal, Spain, Netherlands and the United Kingdom) was launched in 1991 and carried out in 1992.

A second CIS12 was introduced in 1997 and completed in 1999 with some 11,667

services firms included in addition to 27,102 manufacturing firms13, and more countries participating compared to the first survey (plus Austria, Iceland, Finland and Sweden). 11 See Evangelista et al (1998) for results and methodology of the First Community Innovation Survey. 12 See Foyn (2000) and European Commission and Eurostat (2001) for results and methodology of the Second CIS.

Page 14: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

14

The third CIS14 commenced in 2001 with results delivered in 2002 covering the same countries as CIS-2. Compared to its predecessor, more firms (64,000 enterprises) were sampled, its scope expanded to include manufacturing firms employing 10 people and above compared to 20 people under the second survey, a common core questionnaire used for both manufacturing and services, and a doubling of questions asked for innovators and three times as many questions for non-innovators15. The response rate was very low with results from Greece, Ireland and the United Kingdom still missing at the time the results of the third survey were published. A fourth CIS is to be fielded in 2005.

Other European and non-CIS countries that undertook national innovation surveys,

according to Muzart (1999) were Switzerland and Turkey covering the period 1994-1996, and 1995-1997, respectively. The Swiss innovation survey covered 2,600 manufacturing, 600 construction and 2,200 services enterprises; while Turkey surveyed 4,305 manufacturing and 1,224 services firms. These surveys were done only once. 3.2 Central and Eastern European Countries

Radosevic (1999) describes innovation surveys performed in the countries of Central and Eastern Europe (CEECs) based mostly on secondary material such as statistical reports arising from these surveys. He intends comparing innovation activities of transition economies with the open market-based economies of Western European countries covered by the CIS. He reports two Russian innovation surveys carried out in the period 1992-1994 on a pilot basis and 1995-1996 and cautions the reader on definitional issues of innovation in the Russian context. In Russia, diversification is counted as innovation, as are innovative activities in the initial and interim stages where new equipment is still being assembled but not yet started, possibly leading to overestimation of the number of innovations. Muzart (1999) credits Russia with three innovation surveys of 25,000 manufacturing and electricity firms in 1995, 1996 and 1997.

Other surveys in transition economies included by Radosevic (1999) are Yugoslavia in

1987-1991 and another in 1992-1996, Romania in 1993-1994, Slovenia in 1994-1996, Bulgaria in 1997, Hungary in 1990-1993, and another one according to Muzart (1999) for 1,000 services firms in 1998 and Poland in 1992 on a pilot basis and in 1994-1996 on a regular basis. Niedbalska (2001) describes a third Polish innovation survey carried out in 2000, which included for the first time the services sector (wholesale trade, transport and telecommunications, financial intermediation, computer and related activities, architectural, engineering and other technical activities). The Central Statistical Office of Poland (GUS) conducted the survey regularly almost on an annual basis for the industrial sector (mining, manufacturing, and utilities) but more concise; and every four years starting 2000 for the comprehensive surveys on industry and services patterned after the Community Innovation Survey.

13 Excluding Iceland and Greece at the time the results of CIS-2 were published, as well as excluding Spain and Italy for the services sector 14 See European Commission and Eurostat (2004) for results and methodology of the Third CIS. 15 Based also on presentation of Paul Crowley, DIEP Workshop, UNU-INTECH, Maastricht, 30 May 2004.

Page 15: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

15

In addition to these countries, Muzart (1999) likewise reports the Czech and Slovak Republics to have conducted national innovation surveys: the former covering 1,100 manufacturing firms in the period 1995-1997, and the latter 25 large manufacturing firms on a pilot basis in the period 1994-1997.

3.3 Other Non-European OECD Countries In North America, Canada and not the USA has been the forerunner in conducting

innovation surveys. In 1993, Statistics Canada surveyed manufacturing firms; in 1996 communications, financial services and technical business services sectors; in 1999 manufacturing and selected natural resource industries (for the period 1997-1999); and in 2003 information and communication technology industries, selected professional, scientific and technical services industries, selected natural resource support service industries; and selected transportation industries covering the period 2001-2003. The USA and Japan are two OECD member countries that seldom carry out innovation surveys. According to Muzart (1999), both countries conducted an innovation survey based on the Oslo Manual during 1992-1993 and have since then planned to conduct second rounds.

In Oceania, both Australia and New Zealand have conducted innovation surveys, the

latter one having just introduced it in 2003. New Zealand is thus the OECD country to have just very recently performed its first innovation survey.

Australia developed a national innovation survey on a pilot basis in 1993-1994, which

was initiated by the Australian Bureau of Statistics (ABS)16. The Department of Industry Science and Resources subsequently sponsored a more comprehensive innovation survey of 5,800 firms in the industrial sector in 1996-1997. While the main innovation surveys obtained data from the manufacturing sector, exploratory surveys have also been carried out in the mining, agriculture, construction and telecommunications industries. The ABS planned to undertake a comprehensive innovation survey in 2004 excluding the agricultural sector, with results scheduled for release in end-2004 covering more topics explored in the survey such as general and financial information, innovation activities, outputs, sources of information, technology transfer, business environment, innovation strategy, barriers to innovation, management practices, learning capacity and skills base, information and communication technology upgrades, and innovation expenditures.

In New Zealand17, the Ministry of Research, Science and Technology sponsored the first

innovation survey in 2003 to inform the public about the level of innovative activity and its contribution to economic growth, and to help government in formulating policies to assist growth, and business firms to form their own growth strategies. The population size was 11,557 firms in agriculture, mining, manufacturing, utility, construction, wholesale trade, transport and storage, communication services, finance and insurance, business services, and motion picture, radio and television services. The enterprises belonged to Statistics New Zealand’s Business Frame covering those with turnover of more than NZ$30,000 annually, employing 10 or more

16 See Australian Bureau of Statistics website for information on the different innovation surveys: www.abs.gov.au 17 See Statistics New Zealand webpage for information on Innovation Survey 2003 at www.stats.govt.nz

Page 16: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

16

people, classified according to the Australian and New Zealand Standard Industrial Classification (ANZSIC96) 1996 codes. They belonged to the private sector in accordance with New Zealand Institutional Sector 1996 classification. A total of 2,987 firms responded out of a sample of 3,660 firms. More detailed results were to be published in July 2004. 3.4 Innovation Surveys in Developing Countries

Following the first wave of innovation surveys in OECD countries, developing countries in Latin America, Southeast Asia, and Africa, commenced their own innovation surveys. There are 21 non-OECD developing countries that implemented innovation surveys. By geographical region, 11 are Latin American countries, 5 are European, 4 are from Southeast Asia, and one from Africa (see Chart 2).

3.4.1 Latin America

In Latin America, Anllo (2003) notes that Uruguay was the first Latin American country

to conduct an innovation survey in 1988, but in 1995 Chile was the first country in the region to conduct the survey following harmonized guidelines. This was followed in subsequent years by Colombia (1996), Venezuela, Mexico and Argentina (1997), and Brazil (2000). All surveys were patterned after the Oslo Manual, but shortly after these surveys were conducted, discussions began on how to adapt the Oslo Manual to the peculiar situation of developing economies. Salazar and Holbrook (2003) cite some of the characteristics of Latin American firms that were not adequately addressed by the Oslo standards such as: (i) the informal organizational setting for conducting innovation, (ii) the fewer R&D projects undertaken, (iii) the importance of organizational change in the innovation process; (iv) fragmented information flows within the national systems of innovation; and (v) the fact that innovation is mainly based on the acquisition of technology embodied in capital equipment.

The Colombian Institute for the Development of Science and Technology (Colciencias)

and the Ibero-American Network on Science and Technology Indicators (RICyT) with funding from the Organization of American States, were commissioned to write the Latin American Manual, which came to be known as the Bogotá Manual after its publication in 2000. With the exception of Brazil which still uses the Oslo Manual, the Bogotá Manual and its questionnaire became the basis of innovation surveys carried out now in Latin America with initial testing in Argentina, Uruguay, and Colombia (pilot study in Bogotá) although results from these Bogotá-manual based surveys are still forthcoming at the time of writing this research. 3.4.2 Southeast Asia Five Southeast Asian countries have been active in carrying out innovation surveys. With the exception of South Korea, which is an OECD member country, the other four developing countries are Taiwan, Singapore, Malaysia and Thailand. All of these countries used the Oslo Manual as basis for their innovation surveys.

Page 17: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

17

In Southeast Asia, Malaysia18 was among the first to carry out a national innovation survey in 1995 covering a sample of 815 companies that were a-priori identified as possible innovators from the Malaysian Science and Technology Information Centre (MASTIC) R&D survey, SIRIM (ISO 9000 recipients), MITI (R&D incentives recipients) and the tenants of Technology Park Malaysia. A second national innovation survey for the reference period 1997-1999 was implemented with results published in 2001 encompassing 4,000 firms in the manufacturing sector that were drawn from a list of 20,584 firms provided by the Department of Statistics. Following the recommendations of the Oslo Manual, the second Malaysian innovation survey used a stratified sampling approach. A third innovation survey was completed for the period 2000-2001. Out of 20,359 registered firms a sample was drawn using the same sampling methodology as in the second survey, except that the sample was drawn according to the 5-digit industry classification. In cases where the number of firms in a 5-digit industry classification was less than three, the industry classifications were collapsed until none of the employment size classes in aggregated codes have less than 3 sample firms.

South Korea19, through the Science and Technology Policy Institute (STEPI) introduced

an innovation survey covering 25,596 manufacturing firms over the three-year period 1996 to 1999. A second innovation survey covering 6,233 manufacturing firms was conducted in 2000-2001 based on the Oslo Manual but adapted to suit the Korean situation. The questions from the Community Innovation Survey III were employed with additional information asked on obsolescence periods of knowledge acquired and for strategic positioning in product life cycles.

In 1999, Singapore20 conducted its first national innovation survey through the National

University of Singapore’s Centre for Management of Innovation and Technopreneurship and with the support of the Economic Development Board. The purpose of the survey was to assess the status of innovation among Singapore’s manufacturing firms and benchmark this against OECD countries, and to identify key problems and challenges confronting the firms in innovation processes. Data were collected through a postal survey with questionnaires sent to the CEOs of 1,876 manufacturing firms under four industrial clusters, namely electronics, chemicals, precision and process engineering, and transport engineering sectors, which represent the major manufacturing industries of Singapore.

Patterned after the Third Community Innovation Survey methodology, Taiwan21 also

introduced a Technological Innovation Survey in the period 2001-2002 covering a selected sample of 60,000 firms, of which 10,000 were proper samples and 50,000 reserve samples taken from the population of 153,357 firms under the Directorate General of Budget, Accounting and Statistics’ Industry and Commercial Census. Two methodologies were adopted using the principle of stratified random sampling: 7,689 firms employing 6-499 people were subjected to telephone interviews. Then a face-to-face interview was completed for some 2,478 innovation 18 See MASTIC website: www.mastic.gov.my for downloadable copies of National Survey of Innovation 2000-2001, and 1997-1999, and 1994 National Survey of Innovation in Industry. 19 See STEPI website: www.stepi.re.kr for downloadable copies of Moon-seob Yoon and Jang Jin-Gyu (2000), Technology Innovation Survey in Korea; and Tae Young Shin (2002), Korean Innovation Survey 2002: Manufacturing Sector. 20 See Wong, Finna (2003). The Pattern of Innovation in Singapore’s Manufacturing Sector, Singapore Management Review, January. 21 See Wang et al (2003). A Survey for Technological Innovation in Taiwan, Journal of Data Science 1, pp. 337-360

Page 18: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

18

active firms and 372 innovation inactive firms, or a total of 2,850 firms. For firms employing more than 500 people, a questionnaire was first sent by mail, followed through with a census-style interview of some 506 enterprises, of which 414 were innovation active and 92 innovation inactive.

Finally, Thailand22 carried out two innovation surveys together with its R&D survey in

2000 and 2002, with the latter including the services sector. The surveys relied on the Business On-Line database comprising 50,000 establishments registered with the Commercial Registration Department of the Ministry of Commerce. For the first survey a total of 2,166 firms were selected using stratified random sampling based on the top 13,450 companies by revenues in 1999 consisting of firms generating total revenues of over 12 million bath. Of these 1,019 completed the questionnaires. For the second survey, the same sample as in the first survey was used but extended to cover firms earning at least 12 million baht. Manufacturing firms numbered 14,870 firms inclusive of the 1,019 that responded to the first survey while 6,082 were chosen out of 26,162 service enterprises. 3.4.3 Africa In Africa, only South Africa so far has carried innovation surveys—two in fact: the first, described by Oerlemans et al. (2001) and called the Survey of Innovation in South African Manufacturing Firms, was modelled on the Community Innovation Survey and was jointly undertaken in 1996 by the Directorate for Science and Technology Policy of the Foundation for Research Development (FRD) and the University of Cape Town.

The University of Pretoria, in cooperation with the Eindhoven University of Technology based in the Netherlands, conducted the second South African Innovation Survey (SAIS) in 2001 to address the weaknesses of the first survey which focused too much on outputs and inputs of the innovation process and was biased towards firms that were innovation active thereby neglecting the ways firms managed and used innovative resources (called throughput). The sampling frame was the Reedbase Kompass database containing 16,931 South African firms, sizes of which were based on the number of employees, from which the sample firms were randomly drawn using size classes as strata. Manufacturing and service firms employing up to and more than 50 employees were all included in the sample while those with 10-19 employees and 20-49 employees were proportionally sampled according to their presence in the population. The overall sample size was determined using an allocation formula that incorporated expected response rates, based on which a total number of 7,039 firms ensued. After pre-testing the questionnaire on 15 selected South African companies, a total of 7,000 questionnaires were mailed to managing directors of the surveyed firms, followed by direct surveying through

22 For the second survey, see Intarakumnerd (2003), Thailand’s National Innovation System (NIS) in the Context of East Asian Economies: Initial Findings, National Science and Technology Development Agency, downloadable from www.nstda.or.th/nstc and Brimble (2002), The Results of Thailand’s R&D and Innovation Survey in the Manufacturing and Services Sectors, Asia Policy Research Co. Ltd., slides presented during the Seminar on Thailand R&D and Innovation Survey 2002: Final Results . For the first survey, see Virasa and Brimble (2001), Technological Innovation of Industrial Enterprises in Thailand, slides presented during the Regional Workshop on Innovation in the Manufacturing Sector, Penang, Malaysia, July 20.

Page 19: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

19

telephone and online interviews. A total of 617 firms completed the questionnaires out of the 7,000 dispatched. The results of the second survey are published in Oerlemans et al (2003).

Aside from South Africa, the New Partnership for Africa’s Development (NEPAD) is

contemplating on introducing an innovation survey for 17 African countries. Excluding South Africa, the other sixteen countries are: Algeria, Burkina Faso, Cameroon, Republic of Congo, Ethiopia, Mozambique, Nigeria, Ghana, Kenya, Rwanda, Senegal, Uganda, Gabon, Mauritius, Mali, and Egypt. It is for this purpose why this study was commissioned.

Table 1 summarizes the innovation surveys so far carried out by both developed and developing countries following harmonized or standardized guidelines with details on the reference period, sample size, sectors covered, basis of guidelines or formats, conducting agencies, and sampling methodology and survey methodology. On the basis of information presented in this table, more detailed analysis is undertaken in Section 4.0.

Page 20: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Table 1

Features of Innovation Surveys of Developed and Developing Countries Country

Reference Period

Conducting Agency

Sample Size

Responses

Sectors

Industry Classification

Basis of Guidelines

Sampling Methodology

North America

USA 1992-1993 ? ? ? Manufacturing ? Oslo Manual ? Canada-1 Canada-2 Canada-3 Canada-4

1993 1996 1997-1999 2001-2003

Statistics Canada

? ? 6,618 2,632

? ? 6,037 2,123

Manufacturing Services Manufacturing & Natural Resource industries Services

? ? NAICS 4-digit NAICS 3,4,5-digit

Oslo Manual

Stratified random sampling

Western Europe Pre-CIS Germany 1978-1979 IFO 5,000 ? Manufacturing ? none ? France 1986-1990 SESSI 25,000 ? Manufacturing ? none ? Italy 1981-1985 ISTAT 35,000 8,220 Manufacturing ? none Stratified random

sampling Netherlands 1983-1988 Ministry of

Economic Affairs 10,500 6,194 Manufacturing &

Services ? none Stratified random

sampling Norway 1984-1988 NTNF 279 139 Manufacturing ? none Stratified random

sampling Sweden 1990 Nordic project 200 101 Manufacturing ? none Stratified random

sampling CIS-1 1994-1996 Eurostat 40,817 22,914 Manufacturing NACE Rev 1

2-digit Oslo Manual Stratified random sampling

Belgium 1994-1996 748 284 Denmark 1994-1996 674 344 France 1994-1996 SESSI 3,879 2,909 Germany 1994-1996 ZEW 2,918 642 Greece 1994-1996 399 367 Ireland 1994-1996 999 330 Italy 1994-1996 ISTAT 22,788 14,584 Luxembourg 1994-1996 372 294

Page 21: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

21

Country

Reference Period

Conducting Agency

Sample Size

Responses

Sectors

Industry Classification

Basis of Guidelines

Sampling Methodology

Norway 1994-1996 982 511 Portugal 1994-1996 410 287 Spain 1994-1996 2,372 308 Netherlands 1994-1996 CBS 4,094 2,047 United Kingdom 1994-1996 182 7 CIS-2* 1997-1998 Eurostat 64,000 39.500 Manufacturing

& Services NACE Rev 1 2-digit Oslo Manual

Stratified random sampling

CIS-3* 1998-2000 Eurostat 64,000 35840∗∗∗∗∗∗∗∗ Manufacturing & Services

NACE Rev 1 2 digit Oslo Manual

Stratified random sampling

Switzerland 1994-1996 KOF/ETH 5,400 2,947 Manufacturing, Construction & Services

ISIC Rev 3 2-digit Oslo Manual Stratified random sampling

Central/Eastern Europe

Bulgaria 1997 ? >100 ? ? ? ? ? Czech Republic 1995-1997 CSU 1,100 660 Manufacturing,

Services, Mining, Electricity & Construction

ISIC Rev 3 2-digit Oslo Manual ?

Hungary-1 Hungary-2

1990-1993 1998

IKU

110 1,000

? Manufacturing Services

? ISIC Rev 3 2-digit

? ? ?

Poland-1 Poland-2 Poland-3

1992 1994-1996 1998-2000 1997-1999

GUS

2,430 8,962 9,123

? 7,170 8,211

Manufacturing Manufacturing, mining & utilities Manufacturing, Mining, Utilities & Services

ISIC Rev 3 2-digit Oslo Manual ? Stratified random sampling

Romania 1993-1994 1,276 ? Manufacturing ? ? ? Russia-1 Russia-2 Russia-3

1992-1994 1995-1996 1996-1997

Centre for Science Research & Stat.

25,000 25,000 Manufacturing ISIC Rev 4-digit Oslo Manual

Slovak Rep. 1994-1997 SOSR 25 21 Manufacturing ISIC Rev 3 2-digit Oslo Manual ?

∗∗ Estimated by applying the average response rates, which are available by country based on a chart presented by Paul Crowley at a workshop presentation in INTECH, to the total sample size.

Page 22: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

22

Country

Reference Period

Conducting Agency

Sample Size

Responses

Sectors

Industry Classification

Basis of Guidelines

Sampling Methodology

Slovenia 1994-1996 880 ? Manufacturing ? ? ? Turkey 1995-1997 State Institute of

Statistics 5,529 2,600 Manufacturing &

Services ISIC Rev 3 2-digit Oslo Manual Stratified random

sampling Yugoslavia-1 Yugoslavia-2

1987-1991 1992-1996

48 33

? ?

Metal processing, chemicals, textiles

? ? ?

Oceania Australia-2 Australia-1

1996-1997 1993-1994

ABS DISR

5,800 6,700

? ?

Manufacturing Manufacturing & Services

ANZSIC 2-digit ANZSIC 2-digit

Oslo Manual Oslo Manual

? ?

New Zealand 2003 MoRST SNZ

3,660 2,987 Manufacturing & Services

ANZSIC 2-digit Oslo Manual Stratified two-level sampling

Latin America Argentina-1 Argentina-2

1992-1996 1998-2001

INDEC SECYT

2,333 2,229

1,639 1,688

Manufacturing Manufacturing

ISIC-2-digit

Oslo Manual Bogotá Manual

?

Brazil-1 Brazil-2

1995 1998-2000

GESP/SEAD IBGE

10,658 11,044

6,591 10,968

Manufacturing Manufacturing/ Mining & Quarrying

CNAE 2-digit CNAE-93 2-digit

Oslo Manual ?

Chile-1 Chile-2 Chile-3 Chile-4

1995-1996 1997-1998 2000-2001 1999-2000

INE INE INE INE

546 ? ?

541 521 896

Manufacturing Manufacturing Manufacturing, Mining & energy Agro-forestry

Oslo Manual

?

Colombia-1 Colombia-2 Colombia-3

1993-1996 2000 2002

DNP COLCIENCIAS OcyT, DNP and COLCIENCIAS OcyT, DNP and COLCIENCIAS

4,501 ? ?

885 ? ?

Manufacturing Manufacturing Manufacturing

? ? ?

Oslo Manual Oslo Manual Bogotá Manual

?

Cuba ? ? ? ? ? ? ? ? Ecuador ? ? ? ? Agricultural ? ? ? Mexico-1 Mexico-2

1994-1996 1999-2000

CONACYT CONACYT/ INEGI

1,527 1,610

1,527 1,610

Manufacturing & Services Manufacturing

CIIU Rev 3 CIIU Rev. 3

Oslo Manual Oslo Manual

Stratified random sampling Stratified random sampling

Panama 1999 ? ? ? ? ? Oslo Manual ?

Page 23: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

23

Country

Reference Period

Conducting Agency

Sample Size

Responses

Sectors

Industry Classification

Basis of Guidelines

Sampling Methodology

Paraguay 2000 ? ? ? ? ? ? ? Peru 1997-1999 INEI

CONCYTEC ? ? Manufacturing ? Oslo Manual ?

Uruguay-1 Uruguay-2

1988 1998-2000

CIESU DINACYT

298 3,605

261 762

Manufacturing Manufacturing

? ISIC-Rev 3

None Bogotá Manual

?

Venezuela 1994-1996 CONICIT OCEC

1,523 1,382 Manufacturing ? Oslo Manual ?

Southeast Asia Japan 1992-1993 ? ? ? ? ? Oslo Manual ? South Korea-1 South Korea-2

1996-1999 2001-2002

STEPI STEPI

25,596 6,233

? 3,775

Manufacturing & Services Manufacturing

ISIC Rev 3 SITC-2 digit

? Oslo Manual

?

Taiwan 1998-2000 NSC, National Taiwan Univ.

7,689 2,850 Manufacturing & Services

Oslo Manual Multiple stage sampling & stratified random

Singapore 1999 National Univ. of Singapore and EDB

1,876 371 Electronics, chemicals, precision eng. & transport eng.

? Oslo Manual Cluster sampling

Malaysia-1 Malaysia-2 Malaysia-3

1990-1995 1997-1999 2000-2001

MASTIC MASTIC/Univ. of Malaya MASTIC/Univ. of Malaya

815 4,000 4,000

412 1,044

Manufacturing Manufacturing Manufacturing

? ISIC 3-digit ISIC 5-digit

Oslo Manual Oslo Manual Oslo Manual

? Stratified & proportional random sampling stratified & proportional random sampling

Thailand-1 Thailand-2

2000 2002

NSTDA & Brooker Grp. NSTDA & Asia Policy Research

2,166 6,082

1,019 2,246

Manufacturing Manufacturing & Services

TSIC 2-digit TSIC 2-digit

Oslo Manual Oslo Manual

Stratified random sampling

Africa South Africa-2 South Africa-1

1998-2000 1996-1997

Univ. of Pretoria/ Eindhoven Univ. FRD/ISP & Univ. of Capetown

7,039 ?

617 ?

Manufacturing & Selected Services Manufacturing

SIC 2-digit ?

Oslo Manual Oslo Manual

Stratified random sampling ?

Page 24: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

4.0 Comparison of Innovations Surveys: Organization, Scope, Conduct and Content This section presents a comparative analysis of the organization, survey modalities,

sampling methodologies, sectoral scope, population, sampling and response sizes, threshold points for firm sizes, and the content of the questionnaire. Although every country survey has its strengths and shortcomings, country comparison could highlight best practices that could be picked from among various experiences with the survey and that could impart some lessons for developing a policy-relevant innovation survey for the NEPAD countries.

4.1 Organization of the Survey Innovation surveys are normally conducted by the national statistical agency (NSA)

of the government in collaboration with the government Ministry on Science and Technology (MOST). In some cases, especially when an innovation survey is being carried out for the first time, the government engages the services of a university (UNIV) or even a private consultancy firm (CONSULT) to conduct the survey. The database as well as the analysis of the findings or the report of the results normally resides with the statistical agency.

As can be seen in Chart 3, the national statistical agencies conduct the surveys in a

majority of the cases (35 countries, including the European-17). This is done in collaboration with the government ministry of science and technology (European-17 plus 7 other countries). In one OECD country (Switzerland) and in three non-OECD countries (Malaysia, Singapore, and Taiwan), the government contracted a university to conduct the survey. In one non-OECD country—Thailand, a private consultancy firm was hired by government to manage the survey. In South Africa, a university conducted the survey with some assistance from government in the form of a letter of recommendation from the Minister but without government funding. In Latin America, the Argentinean survey was a tripartite collaboration between the science and technology ministry, the national statistical agency and two universities, while in Colombia, the innovation surveys were conducted by the planning and science and technology ministries without the involvement of the national statistical agency.

Page 25: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

25

Chart 3. Agency Carrying Out Innovation Surveys

0 1 2 3

AustraliaCanada

European-17 (CIS)HungaryMexico

New ZealandPoland

Slovak RepublicSouth KoreaSwitzerland

Turkey

ArgentinaBrazilChile

ColombiaMalaysiaRomania

RussiaSingapore

SloveniaSouth Africa

TaiwanThailandUruguay

Venezuela

NSA MOST/GOVT UNIV/INST CONSULT

In non-OECD or developing countries, the survey was run on a pilot basis first and depending on the demand for the results of the survey, the organizing or conducting agency then decides to hold the survey on a regular basis. OECD countries that are non-CIS participants ran the innovation surveys on a pilot basis as part of the Oslo Manual exercise. Sometimes, in one survey, the questionnaires are pre-tested on a pilot basis and then ran at the national level as in South Africa and Malaysia. There are also a number of cases where the innovation survey is a research project, which makes it occasional and hence it is uncertain whether it would be followed by a subsequent round. Irregular surveys apply to countries that have done more than one round but not periodically. Participation in the survey, in turn, can be mandatory or voluntary. Under the former, a respondent is obliged to respond to the questionnaire when selected as in a census, whereas in voluntary participation, there is the option to respond or not. Participation is probably made mandatory to increase response rates, or to increase representation of the population.

OE

CD

Non

-OE

CD

Page 26: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

26

Chart 4. Frequency of Survey and Participation Type

Aus

tralia

Cana

daCz

ech

Euro

pean

-17

(CIS

)Sw

itzer

land

Pola

ndS.

Kor

eaAr

gent

ina

Braz

ilC

olom

bia

Chi

leM

alay

sia

Thai

land

Sout

h A

frica

Rus

sia

Hung

ary

Peru

New

Zea

land

Sing

apor

eM

exic

oSl

ovak

Tu

rkey

Vene

zuel

aTa

iwan

Slov

enia

Rom

ania

Urug

uay

Yugo

slav

ia

Pilot Occasional Regular Irregular Mandatory Voluntary

Chart 4 plots the participation types and survey frequency. The breakdown on survey

frequency is as follows: pilot basis (8 countries); occasional surveys (8 countries); regular surveys (25 countries, i.e., European-17, Argentina, Brazil, Canada, Chile, Colombia, Malaysia, Mexico, and Russia) and irregular surveys (6 countries, mostly non-OECD countries that have done one round). By participation type, 11 countries outside the EU conducted mandatory surveys and 12 countries conducted voluntary surveys. Within the Community Innovation Survey, the participation rates vary, as France, Portugal and Norway require mandatory participation and the rest are voluntarily participated.

The average reference period for innovation surveys is shown in Chart 5. For

countries that have done several rounds and those that only did one round, a 3-year reference period has become the norm following the Community Innovation Surveys. For countries with several survey rounds, chart 5 shows the average reference period, rounded off to the next non-fraction. Six countries (Australia, Poland, Hungary, South Korea, South Africa, Thailand) were below the norm (with 2 years each) and the former Yugoslavia used the highest reference period (5 years) in two rounds of its survey, followed by Argentina and Colombia with a 4 years reference period. For one-round countries, the Slovak Republic used 4 years, Romania 2 years, and New Zealand and Singapore surveyed only a one-year period.

Page 27: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

27

Chart 5. Reference Years for the Innovation Surveys

0

1

2

3

4

5

Num

ber

of Y

ears

Aust

ralia

Cana

daEU

-17

(CIS

)Po

land

S.Ko

rea

Arge

ntin

aBr

azil

Chile

Colo

mbi

aM

alay

sia

Thai

land

S-Af

rica

Russ

iaHu

ngar

yYu

gosl

avia

Urug

uay

Mex

ico

Czec

hTu

rkey

Slov

ak

Swiz

erla

ndN.

Zea

land

Vene

zuel

aTa

iwan

Slov

enia

Rom

ania

Peru

Sing

apor

e

The main issue with regard to organization of innovation surveys is that of the reference periods chosen. Normally, questionnaires ask questions about whether the firm has done innovations during a typical reference period. While this does not pose a problem for high-technology firms, it does so for low-technology firms who may not have introduced an innovation during the three year period but could have adopted a new technology or, by the time of the survey, it could be in the process of developing a new product or process but not yet having achieved the innovation. Also it could have failed introducing this new process. Most survey questionnaires focus on the product of innovation rather than the process. Salazar and Holbrook (2003) refer to this as “freezing the picture” of innovation processes rather than tracking a moving frame of innovation processes. Another issue is the participation type as normally high innovators would feel that they could provide relevant data and therefore tend to respond while low innovators would have the opposite inclination. Thus, when the participation is voluntary, a smaller pool of low or non-innovators could be expected which introduces an information bias into the survey that could be avoided if the participation were mandatory or obligatory. In terms of frequency of the survey, the issue would concern the lack of panel data to conduct a time series analysis if the survey were irregular and also that of comparability among countries if the surveys were conducted at different intervals and periodicity. 4.2 Survey Modalities The different modalities of conducting the survey are postal surveys, face-to-face interviews, telephone interviews, online questionnaires, follow-ups via postal, telephone, email and fax (PTEF), and computer-assisted telephone interviews (CATI). Based on countries that revealed the conduct of the survey, postal survey is the most common mode of carrying out the survey among the 17 European countries under the Community Innovation Surveys (Chart 6a), and 9 other OECD countries and 4 non-OECD countries (Malaysia, Singapore, Russia and South Africa) as shown in Chart 6b.

Several Rounds One Round

Page 28: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

28

Chart 6a. Survey Modalities of the European-17

Under the Community Innovation Surveys

A

ustr

ia

B

elgi

um

D

enm

ark

Fi

nlan

d

Fr

ance

G

erm

any

G

reec

e

Ic

elan

d

Ir

elan

d

Ita

ly

Lu

xem

bour

g

N

ethe

rlan

ds

N

orw

ay

S

wed

en

P

ortu

gal

S

pain

U

K

Postal Survey PTEF Follow-up Telephone Interview

Personal Interview Online Questionnaire CATI

Chart 6b. Survey Modalities of the Other OECD and Non-OECD Countries

Oth

er O

EC

D

Can

ada

Sw

itzer

land

Cze

ch R

epub

lic

Slo

vak

Rep

ublic

Hun

gary

Sou

th K

orea

Turk

ey

New

Zea

land

Mex

ico

Pol

and

Non

-OE

CD

Arg

entin

a

Bra

zil

Col

ombi

a

Mal

aysi

a

Sin

gapo

re

Rus

sia

Taiw

an

Sou

th A

fric

a

Postal Survey PTEF Follow-up Telephone Interview

Personal Interview Online Questionnaire CATI

Follow-ups via post, telephone, email or fax is the next most used mode, especially in the other OECD countries, and Argentina and South Africa, among non-OECD countries. Five countries resorted to telephone interviews (i.e., Denmark, Iceland and Spain among OECD members and Taiwan and South Africa from non-OECD countries). Personal

Page 29: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

29

interviews, on the other hand, are commonly employed in the Latin American countries, Taiwan, the Czech Republic, and among the member states of the European Community Innovation Survey, only in Luxembourg. Finally, three countries (Belgium, Canada and Brazil) adopted the use of computer-assisted telephone interviews in tandem with postal surveys in the first two and personal interviews in the latter. Only one country (South Africa) used an online questionnaire combined with other modalities. One issue with respect to survey modality is the understanding of the respondents of the key concepts and definitions of innovation, which may be totally different from how it is defined by the surveyors. Thus, a postal questionnaire might give inaccurate responses that otherwise could be clarified under a face-to-face interview, or even telephone interview for that matter. If postal questionnaires are to be the survey modality, there should be some pre-testing on a small subset of the sample to ensure proper or correct understanding and interpretation of concepts and definitions of terms used in the survey. 4.3 Sampling Technique and Sampling Frame

The sampling technique for almost all countries was stratified random sampling. This technique was chosen in order to reduce sampling error by enabling firms to be classified according to size, revenues, and some other criteria such as investment levels, R&D spending, province, etc. In some cases, the sampling procedure was done in stages, that is a multiple stage sampling procedure where a pre-qualification or pre-selection of the sample is done, and then another sampling is taken from the set of those who qualified in the pre-selection. This was done in Malaysia and Thailand, where only innovative firms were further surveyed after the initial sampling. Proportional sampling is another example of multiple sampling techniques where after the total sample size is determined, the number of respondents per strata (employee size, value of turnover or sales, industry classification, province, or even country in the case of the Community Innovation Survey) is apportioned according to their proportion in the entire population. Almost all countries adopting stratified random sampling also employed proportional sampling and to obtain optimal allocation of sampling size, the Neyman formula is used as in the cases of South Africa and South Korea. Meanwhile, only one country—Singapore, based on a cursory description of its innovation survey, used a different approach, that of cluster sampling since the survey identified already four clusters in the manufacturing sector that were innovative.

The sampling frame, which is the list of firms from which the sample is collected,

differs by country. The sampling frame contains the population from which the sample is drawn. This could either be a business register, or census of firms, or previous industrial or production survey, or even an existing database of firms. As illustrated in Chart 7, existing industrial or production surveys or censuses are used as sampling frames by 7 countries. Six countries used business registers (6 countries) while the European-17, Thailand and South Africa source their samples from a database system.

In sampling techniques, one issue is the bias introduced by pre-selecting only

innovative or successful firms. The purpose of the innovation survey is to understand and analyze a process and therefore, non-innovative firms and even failed innovations would impart useful insights to innovation defined as an activity and not as a result (e.g., object versus subject approach).

Page 30: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

30

Chart 7. Sampling Frame Used in Innovation Surveys

Aus

tral

ia

Can

ada

Eur

o-17

(CIS

)

Pol

and

Arg

entin

a

Bra

zil

Mal

aysi

a

Thai

land

S-A

fric

a

S-K

orea

Turk

ey

N. Z

eala

nd

Mex

ico

Taiw

an

Sin

gapo

re

Business Register Ind'l Survey/Census Stat. Database

4.4 Population, Sample, and Response Size Subsequent rounds of innovation surveys also addressed the problems of the size of samples representing the population of firms in an industry, and the degree of responses to the surveys. In many cases, the population is estimated from a census or registry of firms available from the national statistical agency or from an industry survey of the top so-many firms according to some criteria such as revenues or employment issued by a chamber of commerce, or an existing database of firms. While the size of the population is usually not mentioned in the methodological explanation of the innovation surveys, some mention from where the population estimate is obtained. Countries that undertake subsequent rounds also expand the sample size, according to some statistical criteria, to improve ‘representativity’ and reach of the survey. Moreover, initial surveys are only done on a pilot basis so that subsequent rounds implement a wider geographical scope to reflect nationwide results. There is a tendency for response rates to decline as more and more rounds of the survey are implemented.

Charts 8a and 8b depict the population, sample and response sizes of the innovation surveys. The largest population size estimated obviously belongs to the Community Innovation Survey with some 450,000 firms in the third survey compared to 350,000 in the preceding round. For the non-OECD countries, the population size ranges from over 150,000 firms in Taiwan to only a tenth of this in Singapore.

Page 31: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

31

Chart 8a. Population, Sample and Response Sizes of Innovation Surveys By OECD Countries

0

10000

20000

30000

40000

50000

60000

70000

Sam

ple

and

resp

onse

siz

es

0

50000

100000

150000

200000

250000

300000

350000

400000

450000

500000

Pop

ulat

ion

size

Sample Size 4081 6400 6400 5400 2632 6618 6150 1100 8962 9123 5529 3660 1527 1610 5981 6233

Response Size 2291 3950 3584 1798 2123 6037 5412 660 7170 8211 2600 2987 1527 1610 3858 3775

Population Size 4E+0 5E+0 5964 2250 2880 8967 1155 8148 3177

CIS-1

CIS-2

CIS-3

Sw itz-1

Can-4

Can-3

Can-2

Czech-1

Pol-2

Pol-3

Turk-1

NZ-1

Mex-1

Mex-2

SKor-2

SKor-3

Chart 8b. Population, Sample and Response Sizes of Innovation Surveys

By Non-OECD Countries

0

2000

4000

6000

8000

10000

12000

Sam

ple

and

resp

onse

siz

es

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

Pop

ulat

ion

size

Sample Size 233222106110546 120152768371815400400216608703

Response Size 163168659103541521896885138285 76 412104749101224617

Population Size 852 542493 2E+187 205203134410169

Arg-1

Arg-2

Braz-1

Braz-2

Chi-2

Chi-3

Chi-4

Col-1

Ven-1

Tai-1

Sin-1

Mal-1

Mal-2

Mal-3

Thai-1

Thai-2

SAfr-2

The largest sample size is attributed to the European Community among OECD

countries with 64,000 firms, followed by Poland (close to 9,000 firms) and Canada (close to 7,000 firms). For non-OECD countries, the biggest sample sizes belong to Brazil (about 11,000 firms), Taiwan and South Africa (more than 7,000 firms), and Thailand (6000 firms) and the smallest to Singapore (close to 400 firms). Sample sizes increase between the first and second rounds but normally remain the same between the second and third rounds. The CIS-1 broadened the sample size from 40,000 to 64,000 in CIS-2, which was unchanged in CIS-3 due to the addition of services; as did Malaysia from 800 firms in the first survey to

Page 32: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

32

4,000 firms in both second and third surveys. The Canadian fourth survey showed a decrease by half in the sample size of the third survey as it focused solely on services compared to manufacturing, resource-based and services in the previous round to get a snapshot of innovation in the information and communications technology (ICT) sector.

Response rates normally increase between the initial and the second round, as took

place in the CIS and Brazil, although the reverse happened in Malaysia and Thailand. However, the response rates declined in CIS-3 compared to CIS-2 (from 62 percent to 56 percent; in Malaysia-3 compared to Malaysia-2 (from 26 percent to 19 percent) and South Korea-3 compared to South Korea-2 (from 65 percent to 60 percent) even though the sample size hardly changed between both rounds. It is unclear whether this is a trend, although there might be evidence for this if Brazil and Thailand experience the same phenomenon in their third rounds. Chart 9 shows the response rates as percent of the sample size and population size.

Chart 9. Response Rates in Innovation Surveys

CIS-1CIS-2CIS-3

Switz-1

Can-4

Can-3

Can-2

Czech-1

Pol-2

Pol-3

Turk-1

NZ-1

Mex-1Mex-2

SKor-2SKor-3Arg-1Arg-2

Braz-1

Braz-2

Chi-1

Col-1

Ven-1

Tai-1

Sin-1

Mal-1

Mal-2

Mal-3Thai-1

Thai-2SAfr-2

% of Sample % of Population

The average response rate for all innovation surveys was 60 percent, which was also

the average for the European countries included in the Community Innovation Surveys. The highest response rates in the range of 90 percent up occurred in Latin American countries (Mexico, Chile, Venezuela) with Mexico having a 100 percent response rate; 80 percent up in Brazil, Canada, New Zealand and Poland; 70 percent up in Colombia and Argentina. The Asian economies (Taiwan, Thailand, South Korea, and Malaysia), in turn, have low response rates ranging from 20 to 40 percent. Finally, South Africa has the lowest response rate of below 10 percent. As a percent of population, the response rates indicated that they represented about a quarter of the population size in Canada, New Zealand and Turkey; about a tenth of the population size in the European Community, South Korea, and Brazil; and less than a tenth in the rest of the countries.

Page 33: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

33

In the case of the Community Innovation Survey, an analysis of non-responses is taken when the number of non-responses exceeds 30 percent of the ratio between non-responding and responding enterprises in the sample size. In Canada and New Zealand, the non-responding sample is assumed to have the same characteristics as the responding population so that the sample weights of the responding samples are adjusted to account for the contribution of no responses to the estimates. The issue with sampling is for the methodology to ensure that it minimizes sampling error. Estimates based on survey responses become population estimates, which mean that they would represent the percentage of firms in the population that would exhibit a particular characteristic. Aggregating the product of the response variable and the sample weight generates the population estimates. Because the sample selected for the survey is just one of the many possible samples that could have been chosen, the estimates computed from the responses would also be subject to a computed sampling error. Standard errors are used to indicate reliability of results and should hence be determined beforehand. The conductors of the survey would also have to address the issue of imputation of responses in cases of no response to a particular question, and to agree on a ratio of no-response to deem the respondent as a no-response case (e.g., when a respondent does not answer 60 or 70 percent of the questionnaire, it is considered a no response case). A very important consideration also is the respondents themselves as most innovation surveys are sent out to managers or CEOs and not key personnel in the organization who would be knowledgeable about the innovation process although it could be presumed that the responsibility of filling out the questionnaire is delegated by the manager down the company hierarchy to the appropriate personnel. Nevertheless, it is still possible that the outcome might be different if another person would fill out the questionnaire or be interviewed because of different notions and interpretations of innovation. 4.5 Sectoral Scope

In most countries that did subsequent rounds of their innovation surveys, there has

been a trend to expand the scope of sectors covered by the survey. In general, the majority of the innovation surveys initially focused on manufacturing industries in the first round of the surveys, since (i) manufacturing firms have been a source of many innovations historically, and inasmuch as (ii) industrial or manufacturing surveys have existed for quite some time now, whereupon sufficient databases and registers from which representative samples could be taken are readily available. Such has been the case of the Community Innovation Survey, which relied on the existing database of some 40,000 enterprises kept by the Eurostat.

There are two types of sectoral expansion in scope: horizontal and vertical. Horizontal

expansion would entail including in the survey other sectors within the industry group under the national system of accounts, which basically comprise manufacturing, mining, construction, and utilities. Thus, aside from manufacturing where most innovation surveys thrive, horizontal expansion would mean surveying mining, construction and utility firms such as electricity, gas and water supply companies. In some OECD countries such as Canada and Australia, the expansion of the sectoral scope involved precisely the inclusion of mining, utilities and construction firms in the survey. Chile, among non-OECD countries also expanded the scope to mining and electricity sectors in its third round.

Page 34: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

34

Vertical expansion in scope involves widening the scope of the survey to include

other major sectors of the national income accounts, notably either those under agriculture, forestry, fisheries on the one end or services on the other. The Second Community Innovation Survey expanded the industrial scope to include services due to the increasing weight of services in many industrial countries, and the growing trend towards the knowledge economy, which is basically services-based and oriented, spawned by the growth of information and communication technologies and the expansion of trade among countries in services. It is not surprising that the other countries, who have followed the Oslo Manual and patterned their innovation surveys after the European model, also followed suit in expanding the industrial scope of the survey vertically to include services (notably Thailand, Malaysia and South Africa).

Chart 10. Sectoral Coverage of Innovation Surveys

in Countries with Subsequent Rounds

CIS-1CIS-2CIS-3

Canada-1Canada-2Canada-3Canada-4

Australia-1Australia-2Australia-3

Mexico-1Mexico-2Poland-1Poland-2Poland-3Russia-1Russia-2Russia-3

Hungary-1Hungary-2

South Korea-1South Korea-2South Korea-3

Argentina-1Argentina-2

Chile-1Chile-2Chile-3Chile-4

Colombia-1Colombia-2Malaysia-1Malaysia-2Malaysia-3Thailand-1Thailand-2

South Africa-1South Africa-2

Uruguay-2

Manufacturing Mining ConstructionUtilities Services Agro-Forestry

Chart 10 shows the expansion in sectoral scope by countries that had subsequent rounds of innovation surveys. Four countries (South Africa, Thailand, Malaysia, Australia) plus the European Community expanded the sectoral scope vertically by including services in addition to manufacturing while three countries (Australia, Poland and Canada) expanded the

OE

CD

Non

-OE

CD

O

EC

D

N

on-O

EC

D

Page 35: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

35

scope horizontally by including other industrial sectors in addition to manufacturing in their surveys.

It may be observed that in some OECD countries that did subsequent rounds such as

Hungary and Canada, there was really no expansion in sectoral coverage from the preceding round but instead a new survey was undertaken for a sector that was not covered before. For example, manufacturing firms were surveyed in the first round and services firms in the second round. In South Korea, there was an opposite trend, with a concentration on manufacturing in the succeeding round from a much broader scope that covered both manufacturing and services in the first round. What cannot be evident from the illustration is another type of horizontal expansion where subsequent rounds of an innovation survey could widen the industry digit-classification to cover more subsectors, and to capture industrial clustering, or innovations in upstream and downstream activities within the a sector. The fourth Canadian innovation survey expanded the industrial classification to 5 digits from 3 in earlier rounds. The fourth Community Innovation Survey also plans likewise by expanding the scope of services to include retail trade, among others23. Finally, it must be noted that from the literature search only one country surveyed innovation for the resource-based industries such as the agro-forestry sector —Chile, but it is not known whether this was followed by a subsequent survey.

An issue in sectoral coverage is the fact that innovation surveys have tended to focus on manufacturing enterprises, thus neglecting the role of innovation in the resource and services sectors. This is attributed to the definition of innovation as technological so that the emphasis of innovation surveys is on hard technologies rather than soft technologies (such as organizational or managerial innovations, and marketing innovations), which often thrive in the resource-based and services sectors. With regard to resource-based industries, this stemmed partly from the belief that innovation was less intensive in traditional sectors, and partly to the emergence of what were identified as ‘high tech’ or research-intensive sectors (see Pavitt’s typology) such as ICTs and these were classified as manufacturing. With regard to services, the international standard industry classification (ISIC) categories had not been developed to include knowledge-based services such as software until recently and earlier services categories such as retail trading, etc. were again viewed as non-innovative. More recently, banking, software and health services (e.g., biotech) have been recognized from an innovation system perspective as innovative and their relationship to the innovative performance of manufacturing firms has been acknowledged. While recent innovation surveys such as that of the European-17 and Canada include services, the tendency was to just replace the word ‘product’ with ‘services’ in the questionnaire. There is the belief that innovation in services is entirely distinct from innovation in manufacturing so that new instruments should be developed to capture these types of innovations. Tomlinson (2000) cites the distinction between knowledge-intensive business services or KIBS (such as marketing firms, financial services, designers, etc.) and technological-KIBS (like software firms, IT support, engineering consultants, etc.), which are both highly innovative in providing business solutions but would have different working definitions of innovation. The concerns of resources-based industries where a lot of organizational innovation (e.g., farm marketing systems) is occurring, have also been neglected and need to be addressed in future innovation surveys.

23 Based on Paul Crowley’s presentation at the DIEP Workshop in Maastricht, 30 May 2004.

Page 36: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

36

4.6 Firm Size

Firms are classified as small, medium or large by either the number of people employed or by revenues generated, depending on country practice. The former is prevalent in the majority of cases. Only in Argentina, Thailand, Malaysia and New Zealand is firm size defined in terms of some cut-off for revenues, although in its third round Malaysia shifted to firm size by employee count as well, to make results internationally compatible and comparable. The reverse case applies to Argentina who shifted to revenue size in its second round (see Chart 11a). As shown in Chart 11b, a noticeable trend is the different cut-off points of employees for manufacturing and other sectors as evident in the Polish and Community Innovation Surveys. In the former, the third survey design was a combination of census (where the cut-off point was 50 employees) and sampling (where the cut-off point was lowered to 10). The census was applied for the subpopulation of firms in manufacturing, mining, and utilities and the sampling for the rest of manufacturing firms including those employing more than 10. In the case of the Community Innovation Survey, the threshold was 10 employees for services and 20 employees for manufacturing in the second round, but standardized the cut-off point to 10 in both sectors under the third round.

The overall picture depicts that 9 countries (Argentina, Chile, South Africa, Brazil,

South Korea, Hungary and Turkey) and the 17 countries under the Community Innovation Survey used 10 employees as the cut-off point in the latest round of their surveys. Two countries (Singapore and Slovenia) used 20 employees as the hurdle for their first rounds just as the European-17, Australia, Canada and South Korea did in previous rounds. Three countries (Mexico, Poland and Russia) used 50 as the cut-off point, with the latter two maintaining these in all rounds except for the last round of Poland where a combination of cut-off points depending on sectors was employed. Finally, Malaysia used the lowest cut-off point of 1 employee and the Czech Republic the highest at 100 employees.

Page 37: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

37

Chart 11a. C

ut-off Points for Firms Sizes in N

on-OE

CD

Countries

Argentina-1

Argentina-2

Chile-1

Chile-2

Chile-3

Colombia-1

Colombia-2

Venezuela-1

Uruguay-2

Taiwan-1

Singapore-1

Malaysia-2

Malaysia-3

Thailand-1

Thailand-2

SAfrica-2

Brazil-2

Slovenia-1

Russia-1

Russia-2

Russia-3

1010

1010

20

50

55

5

20

110

1020

5050

50

Chart 11b. C

ut-off Points for Firm Size in O

EC

D C

ountries

Poland-1

Poland-2

Poland-3

Canada-3

Canada-4

NZealand-1

CIS-3

CIS-2

Australia-2

SKorea-1

Skorea-2

Hungary-2

Czech-1

Mexico-1

Mexico-2

Switzerland

Turkey50

5050

2015

1010

2020

2010

10

100

5050

510

20 10

10

NZ$250M

12MM Baht

12MM Baht

RM 250 M

25MM Pesos

Page 38: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

38

The issue with firm size dwells more on taking the firm as the unit of measurement. Salazar and Holbrook (2003) and Tomlinson (2000) acknowledge that the use of the firm as the unit of measurement of innovation is becoming less relevant with the increasing trend towards outsourcing and collaboration among firms. Thus, innovation takes place among networks of firms although from the statistical point of view, it would be difficult to adopt an alternative unit of analysis. Thus, questionnaires should be able to tackle these network relationships and adapt relevant economic mapping from sociometric techniques. As for firm size, the underlying difference between manufacturing and services sectors is that the latter, with the widespread use of information and communication technologies could be innovative even with an employee size of less than 5. Thus, firm size in future innovation surveys will need to address the size and consistency issues between manufacturing and services firms.

A second concern about firm size is the under-representation of the public sector as a

source of innovation, especially when studies confirm that the government, being a major service provider, is an early adopter of new technologies in the services industry. Two surveys carried out by Statistics Canada of the services sector in 1996 and 1999 covered the unregulated computer industry sector and the heavily regulated banking and financial sector. In the Survey of Electronic Commerce and Industry, the findings of Statistics Canada (2002b) showed an equal rate of adoption in existing technologies that were new to the organization between the public and private sectors but the development of new technologies was higher in the former. 5.0 Questionnaire Content

The innovation surveys that were compared in this study shared several common features or a standard core set of questions. These relate to:

• Basic information on the firm: turnover, employment, activity, linkage with foreign firms

• Science and technology and R&D measures • Types of innovation (product/process; invented/adopted by firm)

introduced • Objectives, goals or reasons for innovating • Sources of information for innovation • Cooperation or collaboration for innovation (with competitors,

customers, universities, government) • Impact of innovations on firm performance • Obstacles to innovation • Costs, financing and expenditures of innovation • Government policy or incentives affecting innovation

Table 2 compares the innovation survey questionnaires representative of good

practice in each of the major world regions. These are: Europe (Community Innovation Survey), North America (Canada), Asia (Malaysia), Latin America (Brazil and Colombia) and Africa (South Africa). As innovation surveys have thus far been conducted in only one African country, two rounds of the industrial surveys conducted by the World Bank and European donors with the collaboration of the African governments were included for comparison. The questions were grouped according to the following typologies: General Information, Science and Technology and R&D Measures, and Innovation Questions as more

Page 39: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

39

or less the questions revolved around these general themes. The actual questionnaires, however, do not show such typologies.

5.1 General Information Questions

General information questions focus on the profiling of firms according to ownership background, location, activity, size in terms of number of employees and turnover, international linkages, and the competitive environment. In Table 2, three distinguishing questions stand out: (i) questions on value of imports and investments in machinery and equipment by Latin American countries; (ii) questions on the competitive environment by Canada; and (iii) questions on industrial product classifications by the European-17 and the Latin American countries. The first one was one of the novelties introduced in the Bogotá Manual, to take account of the unique situation of the Latin American region in terms of acquiring technology embodied in capital equipment; the second gives a picture of how competition shapes innovation behaviour of firms; and the third was one way of capturing innovation done in industrial clusters. One novelty that could be introduced is the use of postal codes or area codes in order to incorporate locational or regional clustering of innovation. 5.2 Questions on Science and Technology and R&D Measures The second category of questions concerns standard S&T and R&D metrics24 normally available under S&T indicators or R&D surveys. One reason for including these questions is to compare R&D expenditure and innovation expenditure data coming from the innovation survey and from the R&D or S&T survey. Godin (2000b) accounts for discrepancies between data on R&D expenditures found in the R&D survey and innovation survey, especially for small and medium enterprises (SMEs) where formal R&D sectors are non-existent. The R&D survey only looks at the formal R&D sector following the guidelines of the Frascati Manual25.

One noticeable difference among the countries being compared in Table 2 is that the

Community Innovation Survey and the Malaysian survey focus only on the results (e.g., the number of scientists and engineers and R&D personnel and patent applications and grants). More questions are included in the Canadian and Latin American surveys on the level of education of employees, whether the firm has an employee-training program and how much time is spent on it, and even the gender aspect in the case of Latin America. The Canadian survey further asks questions on (i) Internet connectivity and usage, (ii) R&D collaboration and alliances with other firms; and (iii) ways by which the firm protects intellectual property. The Latin American questionnaires pose questions on the sources of financing R&D while the South African survey added a question on technology transfer and types of technology supplied.

24Metrics is the study of measurement as applied in different fields such as econometrics, bibliometrics, etc. 25 The Frascati Manual is the OECD manual for the measurement of resources devoted to research and experimental development, and was written by and for the national experts in OECD countries who collect and issue national R&D data and who submit responses to the OECD international R&D surveys.

Page 40: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004
Page 41: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

Table 2. Comparison of Innovation Survey Questionnaires

European Community Innovation Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate Assessment c

I General Information Questions

1 Name, Address X X X X X X X X

2 Geographic/ Regional Location of the firm (city, suburb)

X X X X

3 Owner Background (Educational level: high school, university, graduate school, etc.)

X X X

4 Main activity of the firm (sector)

Manufacturing, services, import-export, etc.

ISIC/NACE classification

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

5 Date of establishment X X X X X X X X

6 Ownership structure (Local, foreign, single proprietorship, partnership, company, conglomerate)

X X

7 Form of organization (joint venture, subsidiary, franchise, chain, license, affiliate, etc.)

X X X X X X X X

8 Size of the firm (by number of employees, sales turnover, operating revenue, exports, etc.)

X X X X X X X X

9 Value of imports and breakdown into machinery and equipment and others

X X

10 Value of investments and breakdown into machinery and equipment

X X X X

Page 42: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

42

European Community Innovation Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate Assessment c

11 Value of profitable or productive assets and utilization of installed capacity

X X X X

12 Measuring changes between years :

If firm experienced merger, sale, change of ownership and when

Did turnover increase due to merger with another firm or decrease due to closure of part of the firm

X

X

X

X

X

X

X

X

X

X X

13 Questions on competitive Environment: X X X

13a Rating whether consumer demand is predictable, product is substitutable, competitor behaviour is predictable, product becomes easily obsolete, technology in the industry changes rapidly, competitors easily substitute among suppliers, arrival of competitors is a constant threat

X

13b Rating intensity of competition in the industry in terms of price, flexibility to meet customer needs, quality, customer service, customisation of products, wide range of products offered, frequency of introducing new products

X

13c Rating importance of firm success due to technology and R&D, management, production, marketing, financing, human resources

X

Page 43: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

43

European Community Innovation Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate Assessment c

II Science and Technology and R&D Measures

15 Number of employees by level of education (graduate, university/college, secondary, elementary)

X X X X X X

16 Does the firm have formal employee training and how much spent on education and training?

X X X X

17 Does the firm use the Internet?

Percentage of employees with Internet access

Does the firm have a home page on world wide web?

X

X

X

X

X

18 Number of scientists and engineers X X X X

19 Number of R&D personnel

And breakdown by sex

X X

X

20 Number of patent applications and grants X X

21 Amount spent on R&D X X

22 Sources of financing R&D (government grants, venture capital, parent company, etc.

23 Type of research done (develop new products/ processes, improve existing products/ processes)

X

24 Results of R&D (new products, prototypes, new processes, pilot plants, patents, publications)

X

Page 44: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

44

European Community Innovation Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate Assessment c

25 Cooperation and alliances in R&D (clients/customers, suppliers, competitors, joint ventures, consultants, other firms in another industry, government laboratories, universities, others)

X X

26 How intellectual property is protected by the firm (copyrights, patents, industrial designs, trade secrets, trademarks, integrated circuit designs (semiconductor chips), plant breeders rights (plant varieties), others (specify)

X

27 Rating effectiveness of each type of protection for IP listed in 26

X

28 Does firm sell or transfer technology to other firms and types of technology the firm supplies (marketing/ distribution, sales technology, production/ processing technology, product technology, technical know-how and techniques)

X

III Innovation Questions

30 Introduction of innovations X X X X X X X X

30a Did the firm introduce any technologically new or improved products during a period of time

X X X X X (X) X X

30b Did the firm introduce any technologically new or improved processes during a period of time

X X X X X (X) X X

Page 45: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

45

European Community Innovation

Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate

Assessment c

30c Did the firm introduce any improvement in organization structure or internal business routines

X X X X

31 Profile of innovation activity (R&D, technology acquisition with and without capital, capital investments, market testing, modernization, design, commercialisation, , etc.)

X (X) X

32 Novelty of innovation (use of new materials, use of new intermediate products, new functional parts, use of radical new technology, fundamental new functions, new production techniques, new organizational innovations, new professional software, others (specify)

X X

33 Objective of innovations X X X X X (X) X

33a Replace products being phased out X X X X

33b Improve product quality X X X X X

33c Extend product range X X X X X

33d Open up new markets or increase market share (and where by region)

X X X X X

33e Fulfil regulations/ standards X X X X X

33f Improve production flexibility X X X X X

33g Reduce labour costs X X X X X

33h Reduce materials consumption X X X X X

Page 46: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

46

European Community Innovation

Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate

Assessment c

33i Reduce energy consumption X X X X

33j Reduce environmental damage X X X X X

33k Reduce production lead time X X X X X

33l Take advantage of government support/ incentives X X

33m Develop environmental-friendly products X X

33n Others (specify) X X X X X

34 Sources of information for innovation X X X X X X X

34a Sources internal to the firm (in-house R&D, marketing, production, management, others)

X X X X X X

34b Sources external to the firm (competitors, acquisition of embodied technology in equipment, customers, suppliers of materials/ equipment/ components, other firms not related to the line of business, consultancy firms, strategic allies)

X X X X X X

34c Generally available information (government information programs, fairs and exhibitions, professional conferences/ meetings/ publications, social gatherings, patent literature, electronic information)

X X X X X X

34d Education and research institutes (universities, government research institutes, private research institutes)

X X X X X X

Page 47: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

47

European Community Innovation

Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate

Assessment c

35 Collaboration and cooperation in innovationd X X X X X X X

35a Other firms within family X X X X X

35b Competitors X X X X X

35c Clients/ customers X X X X X X

35d Consultancy firms X X X X X X

35e Suppliers of equipment, components, software X X X X X X X

35f Private universities and educational institutes X X X X X X

35g Government or private non-profit research institutes X X X X X

35h Strategic allies X

35i Head office or parent company X X X X

35j Technology centers X

35k Other partners X X X

36 Impact of innovations X

36b Quantitative improvements

Percentage of operating revenue and exports coming from new products (goods and services), improved products (goods and services) unchanged products (goods and services) and in terms of cash flow

X

Page 48: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

48

European Community Innovation

Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate

Assessment c

36a Qualitative improvements

In productivity (of employees, customers), product or market expansion (range of goods and services provided, geographic scope), quality of service (flexibility in adapting to customer’s requirements, speed in supplying/ delivering services, accessibility of clients to products/ services in number of hours), environmental impact (compliance with safety requirements, meeting ecological/ medical/ ergonomic requirements, durability and recyclability of products)

X X X (X) (X)

37 Obstacles to innovatione X X X X X X

37a Number or percentage or frequency of innovation projects that were delayed, abolished or not started

X X X

37b Excessive perceived economic risks (cost-benefit analysis show doubtful results)

X X X X

37c Innovation costs too high (exceeding budget) X X X X

37d Lack of appropriate financing X X X X

37e Organizational rigidities (internal resistance, long administrative and approval process)

X X X X X

37f Lack of information on technology (knowledge gap) X X X X X

37g Time to market (could not meet required market introduction time)

X

Page 49: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

49

European Community Innovation

Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate

Assessment c

37h Restrictive public or other government regulations X X X X

37i Lack of customer responsiveness to new products X X

37j Lack of skilled personnel X X X

37k Recent economic developments (e.g., recession) X

37l Other bottlenecks (specify) X X X X

38 Financing of innovation X X (X) (X)

38a Sources of financing innovation (own resources, firm relationships, in-house, other firms, government, commercial banks, international cooperation, venture capital)

X (X) (X)

38b Identification of different innovation funding programs if they exist

X

39 Expenditures on innovation X X X X X

39a Breakdown of expenditures on innovation (intramural R&D, extramural R&D, tooling-up/ start-up, industrial engineering, training linked to innovation, marketing of new or improved products, gross investment directly linked to introduction of new products)

X X X X X

39 Government assistance for innovation X (X)

39a Technical consultancy services (assistance through technology transfer)

X

Page 50: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

50

European Community Innovation

Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate

Assessment c

39b Technical support services (evaluation of equipment, implementation of productivity improvements, registration of patents)

X

39c Duty free importation of machinery or equipment X X

39d Funding for commercialisation of R&D results X

39e Subsidies and grants for R&D X

39f Tax incentives X X

39g General question on suggestions on how government can assist, support or encourage innovation

X

40 Innovation management tools

Did the firm manage the innovation process through

X

40a Technology monitoring and scanning X

40b Technology forecasting and foresighting X

40c Competitive technological intelligence X

40d Competitor analysis X

40e Industry analysis X

40f Market analysis X

40g Technology innovation self-audits X

40h Core competence assessments X

Page 51: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

51

European Community Innovation

Survey

Canada

Malaysia

Colombia

South

Africa

RPED

Round 1 a

RPED

Round 3 b

RPED

Investment Climate

Assessment c

40i Intellectual property self-audits X

40j Project portfolio management X

40k Cross functional teams in innovation projects X a Based on analysis of Burundi, Côte d’Ivoire and Tanzania Round I questionnaire b Based on analysis of Tanzania Round III questionnaire c Based on analysis of Uganda ICA 2002 questionnaire d In the CIS, the location of the partner or collaborator is also asked while in Canada, the question on cooperation and collaboration is confined to R&D activities only. In Colombia, the cooperation is evaluated or rated by degrees of satisfaction. e The Colombian survey categorizes obstacles at the micro-, macro- and meso- levels.

Note: An (X) indicates that some partial information could be derived or deduced from other questions but is not directly asked in the questionnaire.

Page 52: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

52

5.3 Innovation Questions

All countries ask a core set questions. These deal with the objectives of innovation, sources of information, collaboration and cooperation in innovation26, obstacles to innovation, financing of innovation expenditures. The differences between the countries compared in Table 2 lie in: (i) questions on the novelties of the innovation by Canada; (ii) questions on the quantitative and qualitative impacts of innovation by Canada and Latin American countries; (iii) questions on government assistance to innovation by Malaysia; and (iv) questions on innovation management tools by South Africa. The question on the novelty of innovation, or in other words, what was the new element that the firm introduced, for example, use of new materials, new functional parts, new intermediate products, new organizational improvement, etc. helps to establish a broader scope of innovation and is able to capture non-technological innovations. The impact question is also noteworthy insofar as the qualitative aspects are concerned, since it is difficult to quantify and directly attribute the contribution of an innovation to sales or revenues or even cash flows. Quantification would increase the propensity for no response or delay the submission of the questionnaire without adding value to the outcome or results, except for academic users who would probably use the data in econometric exercises. The government assistance question is very relevant in terms of policy while the innovation management tools questions on technology forecasting and foresight as well as other technology monitoring and scanning tools, which are not even asked in any of the Community Innovation Survey rounds, are felt to be highly irrelevant and inapplicable in the African context. 6.0 Other Relevant Surveys in Africa Earlier, it was observed that most OECD countries had systematically undertaken industrial or production surveys and censuses that provided the sampling frame for innovation surveys. Few of the NEPAD countries have this kind of basic information. Table 3 provides data on the occasional surveys that have covered industrial notably manufacturing activities. A number of African countries have undertaken industrial surveys. In addition, in several African countries, UNIDO and the World Bank have also carried out occasional industrial manufacturing surveys. Under the World Bank’s Regional Program on Enterprise Development (RPED) in Africa, some countries participated with one or several rounds while a few more countries participate in the Investment Climate Assessments, also coordinated by the World Bank. The UNESCO has periodically attempted to collect data on S&T indicators but the data are often scanty and for many countries do not even exist. This section discusses the experience with industrial surveys taking Kenya as the case and describes the lessons learned from conducting the RPED surveys that could be applied when constructing an innovation survey for NEPAD countries.

26 The Canadian questionnaire includes questions on R&D collaboration in the R&D metrics section.

Page 53: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

53

Table 3. African Countries in which UNIDO, World Bank RPED and ICA Surveys have been undertaken and for which S&T statistics were reported to UNESCO

(most recent year)

Country

UNIDO

WB/RPED

WB/ICA

UNESCO S&T statistics

Algeria

Angola

Benin

Botswana

Burkina Faso 1997

Burundi 1993

Cameroon 1995 1996

Cape Verde

Central African Rep.

Chad

Comores

Congo, Rep.of 2000

Congo, Dem. Rep. of

Côte d’Ivoire X 1996

Djibouti

Egypt 2000

Equatorial Guinea

Eritrea 2002

Ethiopia X 2003

Gabon

Gambia

Ghana X 1994

Guinea

Guinea-Bissau

Kenya X 1995 2003

Lesotho

Liberia

Page 54: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

54

Libya

Madagascar 2000

Malawi

Mali 2004

Mauritania

Mauritius 1997

Morocco

Mozambique 1998 2002

Namibia

Niger

Nigeria X 2001

Reunion

Rwanda

Sao Tome & Principe

Senegal 2004 1997

Seychelles

Sierra Leone

Somalia

South Africa 2004 2001*

Sudan

Swaziland

Tanzania X 1996 2003

Togo

Tunesia 2000

Uganda 1998 2002 2000

Zambia X 1995 2003 1999

Zimbabwe X 1995

Note: For the UNIDO surveys, X indicates a survey has been done. For the World Bank surveys, the year of the most recent survey is reported. The last column of the UNESCO S&T Statistics, covering the period 1996-2002, mentions the most recent year for which data are reported.

* South Africa has conducted a full-fledged S&T survey

Page 55: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

55

Page 56: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

56

6.1 Experience with Industrial Surveys In most African countries public data collection is generally done by a specific government department often falling within a ministry such as the Ministry of Planning and National Development, whose functions comprise the data collection, data analysis and production of official statistics, dissemination of results to users and producers, and archiving of survey and census results. In what follows, Kenya is taken as an example. In Kenya, the industry division within the Central Bureau of Statistics (CBS) undertakes the industrial surveys. The CBS maintains a master file of all establishments in the country, which provides a framework of collecting establishment-based data. The surveys are the Survey of Industrial Production (SIP) conducted annually and the Census of Industrial Production (CIP), conducted every five years. Both surveys are mandatory in that firms (and establishments) must respond to them. The two surveys are broad and are supposed to cover all establishments in the manufacturing industry. The other industrial survey is known as the Monthly Survey of Industrial Production (MSIP), done in collaboration with the RPED survey.

Another department, the Registrar of Industries within the Ministry of Trade and

Industry, undertakes an annual mandatory survey of registered industries. The information captured in this survey includes: • General firm information (date of establishment, location, type of organization,

owners etc). • Details regarding factory operations (operational time, capital data, loans, capital by

nationality). • Employment and labour costs • Production and installed capacity. • Reasons for underutilization of capacity. • Changes in stocks. • Consumption of utilities. • Raw material consumption (locally available raw materials, imported raw materials). • Sales and other revenues (domestic sales and export sales). • Plant and technology in use. • Research and development (areas and expenditure). • Fixed assets. • Planned expansion and diversification.

In all the industrial surveys undertaken, questionnaires are used in the data collection. In the preparation and design of the questionnaires, input is sought from key stakeholders that are expected to use the resulting data. After preparation of definitive questionnaires, a pilot survey is done to provide valuable insights for the survey organizers and this experience gathered is used to modify the survey instruments accordingly. The questionnaires are mailed and then follow-ups done by phone or visits to try and improve the response rates. The sampling design depends on the nature of the survey although stratified random sampling using employment size as strata, and simple random sampling are often employed. Although an effort is made to include new entrants in the master file and subsequently in the industrial surveys, this is however not

Page 57: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

57

effectively done. The updating is weak to the extent that at a point in time one obtains several key firms existing on the ground yet missing in the master file, leading to under-representation of new firms in the surveys or under-coverage of some sectors.

In some of the well supported/facilitated surveys (e.g. RPED), data capture begins when the trained personnel in the field collect the required data. In the field, the data are reviewed and corrections are made. Once the data have been captured, the survey questionnaires are sent for editing, data entry and processing by a team of well-trained data entry clerks who enter the data into the computer using appropriate software. Coding in advance has the advantage that it saves time as data entry can be commenced almost immediately after the survey has been launched.

There are many factors that affect the quality of industrial data collected. Lack of

finance and other resources disrupts consistency and frequency of collecting data in industrial surveys. As a result of this most of the industrial data provided are estimated based on previous surveys, which were discontinued more than a decade ago, thus greatly affecting the quality of data and making the results obtained after the analysis dubious. The implication is that industrial policies based on such kind of data set are likely to be flawed. The lack of resources such as vehicles and telephone facilities disable follow-ups of non-responding firms, and again some date imputation or estimation is resorted to.

Other factors that could affect the quality of data from industrial surveys include: • Lack of statistical law enforcement to enforce and sanction non-response • Lack of continuous revision to modify the questionnaires and instruments used. • Lack of maintained periodicity and timeliness in some surveys resulting to use of

estimates. • Lack of well kept databases, which are often not available electronically • Lack of full exploitation of the links established with other stakeholders.

Due to the seriousness of data quality, a socio-demographic Statistics Project for

Anglophone Africa has been launched by the World Bank-IMF to strengthen statistical capacity in 14 developing countries in sub-Saharan Africa. The countries include: Botswana, Eritrea, Ethiopia, Kenya, Lesotho, Liberia, Malawi, Namibia, Nigeria, Sierra Leone, Sudan, Swaziland, Zambia and Zimbabwe. The project aims at helping these countries to improve their statistics by participating in the general data dissemination System (GDDS) developed by the IMF. GDDS encourages countries to improve the quality of their official statistics; provides a framework for evaluating the needs for data improvement and for setting priorities for statistical development; and guides participating countries in the public dissemination of comprehensive, timely, accessible and reliable economic, financial, and socio-demographic statistics.

6.2 Experience with RPED Surveys

In the beginning of the 1990s, a research programme ‘Regional Program on Enterprise Development’ was initiated by a group of European donors and the World

Page 58: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

58

Bank, who coordinated the program. The origin of the initiative was the observation that the debate on the role of the industrial sector and the discussions on economic policies in Africa suffered from weak data. Detailed firm-level surveys were rare, which made the policy debate superficial. Therefore, the principal objectives of the program were to collect firm-level data in order to:

� Develop a more precise understanding of how various elements of the African business environment influence private enterprise behaviour and performance;

� Translate the research results into recommendations for more effective policies and assistance programs for private enterprise development;

� Strengthen the analytical capacity of African institutions; and � Create the capability for both donor agencies and African institutions in the

private enterprise development field. The intention was to survey a panel of about 200 firms in four major industries, over a period of three consecutive years. Table 4 presents the countries, the year in which the survey rounds were carried out and the size of the initial sample.

Table 4. Overview of RPED Firm Surveys

27 Data and questionnaires for Burundi, Cameroon , Côte d’Ivoire, Ghana, Kenya and Zimbabwe can be downloaded from http://www.worldbank.org/research/projects/facs/africa.htm. The questionnaires for Zambia, Uganda and Mozambique could not be found, hence they are not mentioned in most of the following discussion. 28 In Ghana, in total five survey rounds were organised. The first three rounds were organised as part of the RPED program. The last two rounds were organised as the ‘Ghanaian Manufacturing Enterprise Survey’ and conducted by the Centre for the Study of African Economies, University of Oxford and the Ghana Statistics Office. These last two survey rounds were done in 1996 and 1998. They covered each a two-year period, 1994-1995 and 1996-1997 respectively and used a questionnaire that is very similar to the RPED questionnaire. However, the technology section was very small in round 4 and absent in round 5. Information and survey data can be downloaded from http://www.csae.ox.ac.uk/datasets/Ghana-rped/Ghmain.html 29 The data sets questionnaires and publications can be downloaded from http://www.csae.ox.ac.uk/datasets/Tanzania-RPED-TMES/Tanz-rped-tmes-main.html

Country Round 1 Round 2 Round 3 No. of firms surveyed in round 1

Burundi27 1993 - - 120 Cameroon 1993 1994 1995 208 Côte d’Ivoire 1995 1996 - 234 Ghana28 1992 1993 1994 200 Kenya 1993 1994 1995 224 Tanzania29 1993 1994 1996 217 Zambia 1993 1994 1995 215 Zimbabwe 1993 1994 1995 201 Mozambique 1998 146 Uganda 1998 ?

Page 59: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

59

Using elaborate questionnaires in a face-to-face interview, the surveys gathered detailed information in order to screen firm’s conduct and performance in relation to their business environment. As such, questions were asked on the firm’s history, employment, technology acquisition, investment, profitability, experience, education and skills of entrepreneurs, relations with clients and suppliers, and the effects of government regulations, business support services and infrastructure. As part of the survey, up to 10 workers per firm were interviewed and data were collected on their education, training, wages and work history.

The survey was restricted to four subsectors of manufacturing— food processing, textiles/garments, woodworking and metalworking. The samples included both registered formal sector enterprises and unregistered informal ones. Samples were stratified on the basis of sector, size and location. Most surveys suffered from serious dropouts of sample firms between the interview rounds (over 20%). Replacement firms were therefore included in subsequent rounds, and selected so as to match the initial sample in terms of size, sector and geographical distribution.

More recently, to mainstream the data collection and analysis into the World

Bank’s operational and research agenda, Investment Climate Assessments (ICA) were conducted in a number of African (and other) countries. ICAs have become the core product of the RPED program, and more ICAs are planned to be carried out. The RPED Business plan states that countries to be surveyed (or re-surveyed) in the next few years include Madagascar, Malawi, Cameroon, Côte d’Ivoire, Ghana, Niger, Benin, Lesotho and Sri Lanka. Table 5 gives an overview of the ICAs conducted in Sub-Saharan Africa, for which reports have been found citing the corresponding sample sizes.

Table 5. Overview of RPED Investment Climate Assessments

Country Round 1 No. of firms surveyed in round 1

Eritrea 2002 224 Ethiopia 2003 ? Kenya 2003 285 Mali 2004 ? Mozambique 2002 193 Nigeria 2001 232 S. Africa 2004 ? Senegal 2004 ? Tanzania 2003 276 Uganda 2002 300 Zambia 2003 334

A typical Investment Climate survey in Africa uses a stratified random sample of

200-400 firms active in manufacturing, construction, commercial agriculture or tourism.

Page 60: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

60

The ICA questionnaire is different from the original firm questionnaire of the RPED with its inclusion of a sub-questionnaire on capacity, innovation and learning.

The RPED questionnaires for round 1 included a section on technology, further

subdivided into four parts:

a) Technology acquisition; asking whether firms have foreign licenses or assistance contracts; employ expatriates; and the experience and educational level of the general and production manager.

b) Initial investment in plant and equipment, asking the value of the initial

investment in equipment, and the origin of the equipment (new versus used, local versus imported)

c) Most recent addition or change in plant and equipment, asking the value and date

of the most recent investment in plant and equipment, as well as the origin of the equipment (new versus used, local versus imported), the purpose of the investment (improve process, add capacity, new product, replace old equipment); the results for unit cost and quality of production; whether adaptations of the equipment were done and who did it.

d) Technological efforts, assessing the number of scientists and engineers, the

proportion of them active in R&D30, the amount of R&D expenditures and the incidence of collaboration and satisfaction with technology institutions.

After the first wave, the technology section was completely restructured and

partly absorbed into the other questionnaires. For instance, in Côte d’Ivoire round 2, there is no more technology questionnaire whereas the Tanzanian questionnaire for round 3 additionally included a new section on ‘technological innovation’ in the firm. With respect to innovation, the questions included the number of

� new products or designs introduced in the preceding 3 years � major improvements or changes in products or designs in the preceding 3 years � improvements or changes in production methods /processes in the preceding 3

years � organisational or marketing improvements or changes in the preceding 3 years

Subsequently questions targeted the objectives of the innovation, origin of the ideas, collaborations with other firms or institutions. For the non-innovative firms, the obstacles to innovation were assessed.

In Round 3, a separate section on training was included, asking the number of employees trained last year, the type of training (company training, in-house training by

30 R&D is defined as any creative systematic activity undertaken to increase the stock of knowledge, and the use of knowledge to devise new applications. It includes fundamental research, applied research and experimental development work leading to new devices, products or processes. It excludes routine activities of quality control and maintenance even if scientists and engineers are employed in their execution.

Page 61: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

61

external trainers, training outside company, informal in-plant training), cost and purposes of the training. There was also a general section on the structure of the labour force, with number of men and women active in the different functional categories in the firm, allowances paid to them, unionization, etc.

A few other sections of the questionnaire cover aspects related to technology and innovation in a crosscutting manner. These include:

� Questions on infrastructure: to what extent power supply, water, freight transport,

roads, telephone, ports, waste disposal and security are a problem to the firm.

� Questions on business support services: asks firms if they ever used assistance from national business support institutions, such as Chambers of commerce, Investment promotion centers, Universities, Export promotion centers, Management training centers, Ministry of industry and commerce, International financial institutions, Employers Associations, National office of industrial property, Bureau of standards and to rate satisfaction with the assistance.

� Questions on competition: For the first, second and third most important product,

the number, type (public/private, foreign/local), size and competitive advantage of competitors is asked; the importance of main client; the pricing strategy of the firm; the number, type, size and diversification of new entrants and exiting firms and its impact on own firm.

The RPED surveys are particularly useful for the NEPAD innovation surveys in

that they make possible an assessment of the way African firms have responded to certain types of questions related to technology and innovation in the past, so that improvements could be made in the technical aspects of the survey questionnaire developed here (see chapter 3). They also show that concepts need to be defined clearly (e.g., innovation), and questions formulated simply. The response experience has been good, in the sense that there are hardly any missing values except when quantification (numbers) is asked for, or when there are open questions. While numerical variables contain critical information, the extraction of the information of the firms’ records (output, raw materials, wage bill) was time consuming, costly and still problematic with respect to the quality of the data. Open questions also take time, not only during the interview, but also during the data entry process, and data processing (post coding). The easiest and well-responded questions are usually of: (i) the binary types: e.g., yes/no, male/female, in-house/outside; (ii) Likert scales, preferably restricted to 3 options, as there is a tendency to answer at the middle and two extremes of the scale; (iii) pre-coded questions where a gradation is provided for in case more than one response is allowed.

A second application is that the data can be used, however imperfect, to construct

a time series on innovative behaviour of firms or capital stock and human capital endowments. There are various proxy indicators for innovation that RPED surveys use. These are capital investments in new machinery, changes in the skills profile or the

Page 62: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

62

increased use of in licensing. By looking at the changes in each round of these variables, one obtains an indicative measure of innovative performance.

Finally, there are questions in the RPED surveys questions that may be considered as important from a policy perspective: These include the following:

• In the regulation questionnaire: a question on experience with regulations like investment and ownership laws, and licensing systems;

• In the business support services questionnaire: a question on assistance from support institutions, the type of assistance sought, and the benefits of such assistance to the firm;

• In the entrepreneurship questionnaire: a question on training received from government, and the title of the course.

The next chapter discusses the policy relevance of innovation surveys and what

could be done to enhance the policy significance of the innovation survey to be formulated for the NEPAD countries based on experiences with innovation surveys in general, and the experiences with industrial and RPED surveys in the African countries, in particular.

Page 63: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

63

Chapter 3

Creating a Policy-Relevant Innovation Survey for Africa

1.0 Introduction

A number of problems, identified in connection with earlier innovation surveys, have a bearing on the policy-relevance of such surveys. These include the scope and quality of the survey samples, the lack of panel data, the content of the questionnaires and their length, the quality of the data emerging from these surveys and their utility for policy-makers. Section two discusses these problems and the various ways around them. It is based on interviews with individuals who have participated in the design and implementation of innovation surveys in their own countries or at the EU level. Section three looks at the way in which innovation surveys have been used by researchers, including academics and policymakers. Section four draws upon these earlier experiences in the conduct, content and use of innovation surveys, to design an innovation survey for Africa. The survey instrument itself is contained in Appendix One to this report.

2.0 Lessons from earlier Innovation Surveys 2.1 Creating an Innovation System-Oriented Survey Innovation surveys do not cover the full range of actors in an innovation system. In the past they focused mainly on the industrial sector and within it on manufacturing firms. Recent innovation surveys have been shaped more by ‘innovation system’ thinking and the scope of the survey was thus widened to include service sector enterprises, notably utilities (electricity, gas and water supply), transportation, banking and in some instances wholesale trade and transportation. Few countries, however, have included mining and construction (Poland, Australia and Canada) and only one (Ecuador) has extended the range of respondents to include farms. No innovation survey currently includes educational or research organizations, NGOs or policymakers and only a few partially cover intermediates such as productivity centres, business associations and other service providers potentially relevant to the innovation process.

The focus on manufacturing and/or manufacturing and service sector firms tended

to bias questionnaires towards those factors internal to the firm that shape choices about whether to innovate, the kind of innovation (product, process, organizational or marketing) and through what means (licensing in, arms length purchase of new generation machinery and equipment, in-house design and product development, in-house R&D, collaborative RTD). From the firm’s perspective, how the process of innovation is managed within the firm thus became a relevant area for investigation and some of the questionnaires, South Africa for example, supplemented the basic CIS approach with questions designed to deal with this issue. The bias towards factors internal to the firm as opposed to across the system more broadly was also reflected in the

Page 64: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

64

emphasis these questionnaires placed on the enterprise’s objectives and the obstacles to innovation as the enterprise sees them.

There are a number of ways to partially overcome the bias inherent in a focus on

the firm. The Canadian survey, for examples, enables some innovation systems–related linkages among actors to be captured indirectly by asking the respondent firms to provide data on such linkages, their purpose and importance. Similarly, the impact of innovation-related policies has been addressed through questions that ask the firm to assess the frequency and importance of access to various services, programmes or financial instruments designed to stimulate and/or support innovation-related activities. These are generally added to the core questions in the survey and tailored to the specific policies/ programmes/ services/ resources available in that particular country.

Obviously, the best way to realize a more comprehensive innovation system

survey would be to widen the scope to include other actors. But this requires the use of multiple questionnaires addressing the specific concerns and activities of a more diverse range of actors31. Issues of manageability thus impose the logic of reducing the scope of the survey, the quality of the data available to determine population size and enable the drawing of a quality stratified sample for other actors than firms and the funding and administrative capability needed to undertake a larger survey. In Section 4 we therefore recommend adoption of a focus on the enterprise sector, complemented by questions that provide indicators of the broader set of linkages and knowledge flow that are needed for a dynamic innovation system. 2.2 Complementing Firm-level Innovation Data with Aggregate Science & Technology and R&D Indicators Many OECD countries and some countries in the developing world have undertaken science and technology (S&T) or R&D surveys. Data at the aggregate (national) level on engineers and scientists, researchers and research expenditures, patents and other S&T or R&D indicators are thus available. Where these have not been systematically carried out, as is the case in most of sub-Saharan Africa, appropriate S&T indicators at the aggregate level will have to be developed and a separate survey instrument will be needed for this purpose. These data are frequently available within Ministries of higher education, Ministries of Science and Technology, Ministries of Research and the national patent office and their collection can be undertaken in parallel with an innovation survey focused at the micro-level of the firm. In addition, to probe more deeply into the performance of other actors in the system and their links to innovation in the enterprise sector, selected case studies can be carried out based on a subset of actors – universities or specific departments or centres

31 Though not for reasons related to the conceptualisation of the survey in innovation systems terms, RPED surveys, have adopted a sampling approach that includes both manufacturing and service sector firms (defined to include trade, financial, and transport services and tourism) as well as interviews with individual workers. However, only registered firms are included in the survey.

Page 65: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

65

within them, publicly funded research institutes and NGOs, which through S&T or R&D surveys have demonstrated a strong S&T base, carry out research and development activities and/or engage in knowledge diffusion activities.

2.3 Assessing Trends Innovation surveys are not constructed to provide panel data, which is needed for a dynamic perspective on the innovation process overtime. From a policy perspective, the lack of panel data, that is a core set of respondents who are surveyed across several ‘rounds’ of innovation surveys, is an especially weak point in existing innovation surveys and reduces opportunities for a dynamic analysis of change in the behaviour, choices and innovative performance of these firms. An example of the potential use to which panel data can be put is found in a recent study of changes in the type of innovative activities undertaken by firms in Argentina under differing economic conditions (Chudnovsky: 2004). The analysis builds a set of panel data for 718 firms on the basis of two Argentine innovation surveys, one covering a high growth period 1992-1996 and the second covering a period of recession, 1998-2001. The analysis shows that in the first period, firms increased expenditures on embodied and disembodied technologies, while in the second they drastically cut their expenditures on technology acquisition but maintained R&D activities. The econometric32 results indicate that having linkages to other agents, especially suppliers, undertaking R&D and technology acquisition expenditures had a positive payoff in terms of the probability that a firm would introduce new products and processes onto the market. Innovative firms, such as these, moreover, attained higher productivity levels than non-innovators, though small firms had a lower probability of engaging in innovative activities and of becoming innovators and this was even more pronounced in a period of recession. The policy implications of this analysis are evident. 2.4 Providing Feedback for Adaptive Policy-Making

Panel data are also useful when the survey instrument contains questions

expressly designed to provide feedback on the impact of particular government policies intended to stimulate innovation. The current norm, however, is to use a 2-3 year reference period and to undertake innovation surveys at regular intervals. This does provide some time series data but does not provide the analytical possibilities that would be available through the use of panel data. Moreover, the length of time needed to carry out and analyze the data from innovation surveys, reduces their ability to provide information needed for adaptive policy making.

32 The study employed a conceptual framework set by Crepon, Duguet and Mairesse (1998) and developed further in Mairesse & Mohnen (2003, 2004). The framework considers innovation as a process involving specific inputs, such as R&D activities or the acquisition of embodied and disembodied technologies and interactions with other firms and institutions.

Page 66: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

66

Currently there is no easy way around this problem, but a number of initiatives are underway that might be applied in the NEPAD context. One is design a set of policy-relevant indicators that make use of the raw data provided by innovation surveys ahead of time. Anthony Arundel (2004) describes three types of indicators that could serve this purpose.

• Simple indicators that are based on answers to a single survey question such as whether a firm applied for a patent during the reference period can give rise to a patent propensity indicator that measures the percentage of all firms that applied for a patent;

• Complex or synthetic indicators which combine two or more survey responses that yield an indicator such as the percentage of firms that are high growth innovators combining survey responses on firm size, foreign linkages of the firm, export values, etc.

• Composite indicators that combine two or more simple or complex indicators based on diverse measurement scales into a weighted average (e.g., a competitiveness index) and present them as an index.

By converting innovation survey responses into simple, complex or composite indicators of relevance to specific policy areas and analyzing the changes in these indicators over time as revealed by succeeding rounds, policymakers would have some basis upon which to adapt innovation policies to unexpected impacts and changing conditions. A second opportunity to introduce a more dynamic perspective on the innovation process and on the impact of innovation policies exists in countries where industrial surveys are undertaken annually. By assigning codes to all firms covered by the innovation and the industrial surveys and adding a small number of innovation-oriented questions to industrial surveys, a set of panel data can be created. It might also be possible to extract a small-stratified sample from the innovation survey and, in alternate years, carry out a mailed questionnaire based on a small number of core questions, with telephone follow-up to ensure a high response rate. Alternatively, it may be possible to introduce, a shorter mailed questionnaire, as the European Union plans to do. 2.5 Balancing the Need for Policy-Relevant Information against the Problem of an Overly Long Questionnaire Questionnaires are generally designed by academics to maximize information and thus are overly long. There is also a tendency for questionnaires to become longer overtime. EU mailed questionnaires, for example, have been growing and response rates falling. The response rate in mail and computer based surveys alone is too low in most developing countries to be useful in Africa. In a small number of cases, telephone

Page 67: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

67

interviewing has been used but it takes much longer when complemented by computerized questionnaires. Face-to-face interviews seem to deal with these problems best in developing countries. In Brazil, for example, it takes approximately 70 minutes to complete the survey through face-to-face interviews (Viotti Interview). There is currently no consensus on the length of an innovation survey questionnaire, though there is an opportunity to reduce their length if other survey instruments, industrial and S&T surveys, provide some of the descriptive and/or background data needed. This can take place by alternating innovation surveys with other surveys in different years. Policy makers, moreover, do not currently use much of the information collected in innovation surveys. Learning more about what has been useful to policy-makers and how to make the wealth of data generated by innovation surveys more useful can provide guidance in restructuring innovation surveys overtime to reduce their size and to better tailor questions to the local context. 3.0 Using Innovation Surveys There are basically two main groups of users for the data generated by innovation surveys: researchers, including academics in universities and in public sector research institutes, and policymakers. Innovation surveys provide innovation statistics potentially useful in analyzing economic performance and innovative patterns across industries. In OECD countries, results of innovation surveys have been used by academics in, among others, the following ways. (i) To develop models that identify determinants of decisions to innovate or not among manufacturing firms [see Pamukcu and de Boer (2000) who applied a binomial logit model to the Turkish innovation survey data]; (ii) To study the relevance of particular constraints to innovation in industry [see Canepa and Stoneman (2002) who used results of the second Community Innovation Survey to investigate the relative importance of finance as a constraint to innovation among European firms]; (iii) To compare innovative performance of countries [see Mohnen and Therrien (2001) who employed the results of the Canadian 1999 Innovation Survey and the Second Community Innovation Survey to benchmark Canadian innovative performance with four Western European countries]; (iv) To analyze innovation patterns of firms in an industrial sector by comparing results of successive innovation surveys [see Niedbalska (2001) who analyzed and compared results of the GUS 2001 and 1997 innovation surveys]; and

Page 68: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

68

(v) To develop innovation indicators from innovation survey data and compare country statistics [see Mairesse and Mohnen (2001) who used the CIS-1 results to develop innovation indicators and compare results of seven European countries]. In developing countries, a study by Chudnovsky et al. (2004) used the results of two innovation survey rounds of Argentina to analyze the impact of innovation inputs and outputs on the productivity of Argentine manufacturing firms.

Innovation surveys have also been used to provide information to policymakers and politicians. This information generally relates to government policies designed to create an environment conducive to innovation. Chapter one provided a discussion of the way in which policies interact to set the parameters within which actors make choices about learning, linkages and investment –three key elements in an innovation process. Among these, Kim and Dahlman (1992) have identified four broad types of policy instruments that governments use to foster and manage the process and direction of acquiring innovative capabilities, including, for example,

• Financial incentives to firms that undertake research and development; • Policies designed to strengthen the supply side relating to enhancing S&T

capabilities; • Policies designed to strengthen the demand side relating to creating market

needs for technology and • Policies designed to provide effective linkages between supply and

demand sides by making innovation activities technically and commercially successful.

In general, innovation surveys have tended to focus more on innovation inputs and outputs and there is thus a need to stimulate uses of innovation survey results in designing demand-side policies and in developing policies that promote linkages, networks and collaboration among the different actors of the national innovation system. Nonetheless, whether focused on inputs and output or more broadly on innovation system processes, policy makers in Europe, despite the attention they have paid to innovation policies, have been slow to make use of innovation survey data.

Pianta and Sirilli (1997, 1998) criticize the Oslo Manual for not providing

guidelines to measure the use and impact of government policies on firm’s strategy and performance thus reducing the utility of innovation surveys to policy-makers. Some countries (Italy and Malaysia), however, have included questions asking how government policies have assisted the firms in introducing innovations into their own questionnaires.

In the case of the European Union, therefore, three other factors seem to have

contributed more to the limited use of innovation survey data for policy-making. First, innovation survey-based data are only now becoming widely available through, for example, the innovation scoreboard. Second is the long learning curve in developing indicators based on the CIS, as many policy themes require configurations of survey questions that could be developed into relevant indicators. Third is the tradition of using

Page 69: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

69

case studies by the policymaking community33 within the EU. These tend to be limited to a smaller set of firms that may not be representative of the population. A policy-study on the commercialization of public research undertaken by Tekes, Finland’s innovation policy agency, for example, hardly relied on or cited survey data that could have been highly relevant in its discussion of government innovation support (Arundel: 2004).

Similarly, the European Commission study on “The role of universities in the

Europe of knowledge” (Commission: 2003) gives ‘the licensing of university intellectual property and spin-off and start-up companies’ as the two main mechanisms for the commercialization of university research. As Arundel notes, however, data from innovation surveys that have covered this issue34 “…would quickly show that this statement is wrong. Instead, this statement is probably based on interviews with managers of spin-off firms or university technology transfer offices, who will tend to stress licensing and new firm formation and ignore long-standing mechanisms for knowledge transfer such as publications, conference presentations, and hiring newly trained scientists.”(Arundel: 2004, 5). Recent research on university-enterprise relations in France confirms the importance of such linkages (Mytelka: 2004b).

Despite the limited use of data from the Community Innovation Surveys in policy documents, a number of studies based on CIS data have shown their utility for policy-making. Pianta and Sirilli (1997, 1998), for example, using innovation survey results argue that government innovation policies basically serve the twin goals of enhancing the technological capabilities of firms that are already innovative (most larger firms) and of stimulating other (normally smaller) firms to introduce innovations. From an analysis of the firm’s responses to questions dealing with the relevance of different public support initiatives35 for innovation, their study also showed that the largest firms in high-technology sectors were the major recipients of most public support and funding, while a large number of smaller innovating firms rated these government policy tools as inadequate to support their innovation needs. Moreover, these smaller firms mainly used the indirect incentives for new investment embodying innovation, which given their labour-saving technology orientation, imply negative consequences on employment levels. In another policy study Evangelista et al (1997) used the results from the first Community Innovation Survey to analyze the level and structure of innovation expenditures, i.e., R&D, non-R&D (product design, trial production, training and tooling-up, acquisition of products and licenses, market analysis) and capital investments (acquisition of new technology through investments in new machinery and equipment). With respect to levels, the study attempted to identify the extent of industry-specific innovation expenditures across countries in Europe and analyzed cross-country differences. With regard to composition, the study aimed to determine whether

33 The policymaking community includes policymakers and their advisors from among researchers in universities and public sector research institutes and consultants. 34 The CIS surveys do not cover the commercialisation of public research. 35 These ranged from funding programs to the provision of services and public procurement.

Page 70: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

70

components of innovation expenditures vary across industries in Europe and whether such variation is consistent across countries. The study revealed that:

• R&D expenditures comprised a small proportion of total innovation expenditures (10 to 25 percent) and most of innovation expenditures are devoted to capital investments (40 to 60 percent);

• Levels of innovation expenditures reflect industry-specific rather than country-specific characteristics; and

• Composition of innovation expenditures varies by industry, and these differences in composition are similar across countries.

The policy implications of this study relate to the balance of effort between the provision of R&D and non-R&D innovation support as well as to the appropriate level of policy action, i.e., European Commission level vis-à-vis member states depending on country-specificity or industry-specificity of the innovation structure. If innovations at the industry level possess strong country-specific features, then policies directed to specific industries are best relegated to the member state level; otherwise commonalities across countries merit action at the European level. The analysis showed a predominance of industry-level over country-level effects thus implying that European industries share strong commonalities in innovation expenditures. Further analytical studies should be encouraged by the European Union. For the moment, however, the main use of innovation surveys in the European Union and in a number of developing countries has been for benchmarking innovation performance. This includes (i) Broadly mapping innovation in countries with weak national statistical systems36, (ii) Analyzing innovation performance in a new sector or technology of interest, such as information technology or biotechnology, (iii) Analyzing specific characteristics of interest, such as firm size or sector, that might serve to identify strengths and weaknesses at the firm or industry level,

(iv) Comparing national performance relative to other countries as in a race or competition, to motivate governments to introduce policies that will improve performance. The European Commission’s European Innovation Scoreboard now uses CIS-based indicators. The latter serves three policy needs: (i) acting as a warning device for

36 The Brazilian survey (PINTEC) for example, has mapped innovation activities across sectors, sizes of firms and in terms of the type of innovative activities undertaken and whether these have been done in-house or through collaboration with other actors in the system.

Page 71: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

71

potential problems at the national level; (ii) tracking changes over time in national strengths and weaknesses; and (iii) building a consensus in introducing policy action to improve innovation capability (Arundel: 2004; Crowley: 2004). Thus far, the following five indicators are available for member states:

• Percent of SMEs with in-house innovative activities • Percent of SMEs that collaborate on innovation • Total innovation expenditures as percent of total sales • Percent of total sales from new-to-market products • Percent of total sales from new-to-firm products.

The following complex indicators are now being developed and will be available in late-2004 by member state:

• Four levels of firm innovativeness • Percent of high-growth innovators • Percent of firms that use non-patent intellectual property • Percent of firms that are organizational innovators only • Percent of firms that collaborate internationally • Percent of firms that source knowledge from public sector research

institutions • Index for openness to external knowledge sources • Percent of firms that find collaboration of high value to their

innovation activities • Financial factors as a barrier to innovation • Percent of firms that are environmental innovators.

In Brazil, data on innovation surveys were used to profile innovative and non-innovative Brazilian firms based on four criteria – export orientation, firm size, foreign capital origin and industrial sector effect. Results of the analysis served as inputs for the Brazilian government to identify policy measures relating to financial and other forms of incentives for exporting firms to innovate; and to formulate new laws that increase forms of collaboration and cooperation with universities and research centres. Little is known about how innovation surveys in the other developing countries have been used from the policy perspective. In Malaysia, the S&T Policy II does not make direct reference to the innovation survey results but suggests some improvements to innovation activities under the concept of the national innovations system. The Ministry of International Trade and Industry (MITI) is also in the process of reviewing the Industry Master Plan 2 and may possibly use the results of the innovation survey, but to what extent is not known. In Thailand, Intarakumnerd (2003) used some results of the innovation survey to describe the state of Thailand’s national innovation system, particularly the responses on performance of government science and technology institutions under the chapter on science, technology and innovation (STI) Policy. 4.0 Choices in the design of a Policy-Relevant Innovation Survey for Africa

Page 72: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

72

The problem of how to design an innovation system-oriented and policy-relevant innovation survey is complex and the choices are not easy. Where countries have a tradition of conducting both industrial surveys and science and technology surveys, the sources which serve as the basis upon which a sample is drawn are readily available, surveying techniques are well established and there is abundant time series data dealing with industry characteristics and performance and research and development inputs and outputs 37. An innovation survey then goes beyond such data to look at innovation in terms of the crises, competition, and technological changes that trigger it, the interactive processes that stimulate it, that problems that impede it and the policies that support it. Even then, as the critique of existing innovation surveys in section two illustrates, designing a policy-relevant innovation-system oriented survey remains a difficult task. In what follows a set of choices are proposed and the reasoning and trade-offs that such choices represent are discussed. Section four presents a draft questionnaire that takes these choices as its base.

37 The latter provide base-line data on the outputs of university training programmes in terms of numbers of scientists and engineers, the performance of research institutes, universities and enterprises measured in terms of expenditures on research and development, patents granted, numbers of scientific publications and often, their ranking in terms of citations. As discussed in section 2.3 of this chapter, where such data are not available a separate effort will be needed to build up these databases as a complement to an innovation survey.

Page 73: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

73

4.1 Scope of the Sample

From a policy perspective, the utility of a survey, is related to the quality and coherence of the survey sample and to the congruence between questions and respondents. In the African context a number of parameters need to be specified in order to assure relative coherence of the population as a target for policy-making, quality of the survey sample for analytical purposes and relevance of the questions to the respondents. Above all, choosing a modest survey size is critical to assuring quality and a high response rate. Although the agricultural sector remains an important component of economic activity in Africa and current conditions have made innovation in this sector increasingly important for competitiveness, a significantly different set of questions would be required to deal with the issues of innovation in this sector and the sample would have to be considerably broadened. Agricultural services would also have to be included. This runs the risk of an overly long questionnaire, a very expensive process of carrying out the survey as well as substantial problems in drawing a stratified sample that would likely reduce the quality of the overall survey results. A similar set of problems emerges in the inclusion of much of the service sector –retail trade and transport, for example. The alternative is to reduce comprehensiveness in the interest of quality, coherence and congruence within the sample survey and in the relationship of the respondents to the questions, and to use the first round of surveys as an opportunity for learning and experimentation.

Size of the enterprise is another relevant parameter in dealing with the issue of congruence between questions and respondents and contributing to the utility of the data for policy-makers. The economic importance from a welfare perspective of the informal sector, of small farmers and traders notwithstanding, we would recommend that defining a manageable target population for innovation policies in this first survey be given priority. A broader set of questions that survey the characteristics, competences, resources in human and financial terms, access to information flows and habits and practices with respect to risk-taking, investment, learning and innovation in the informal sector should be the objective of a separate survey.

The target population for this first survey would thus consist of firms in selected

resource-based sectors, all manufacturing sectors and selected service sectors, with ten or more employees, thus excluding informal sector firms. It would use the three digit ISIC codes, revision 3.1 (2002) contained in an appendix attached to the questionnaire at the end of this Chapter, as the basis for creating the sample. 4.2 Sampling Methodology

The quality of the survey sample depends critically on the sources from which

the survey sample is drawn. These have varied across surveys. Probably the most comprehensive source in most countries is government industrial surveys, especially

Page 74: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

74

when these are conducted annually. Comprehensiveness in terms of both the data available per respondent and the number and types of respondents is key to the preparation of a well-designed stratified sample. The least comprehensive in both senses are private, association- based membership lists, though these can be useful for crosschecking and may become a primary source for drawing the innovation system survey in the absence of current industrial survey data.

If the objective of an innovation survey is not comprehensiveness but rather to draw a sample of firms that are most likely to be innovative, other types of databases have been used. These include awards to innovative firms or when these exist, the registry of firms that have applied for and/or received tax credits or grants from publicly funded R&D or innovation-related promotional schemes. Such data sources can also be used to crosscheck a stratified sample drawn from a more comprehensive base that generally provides indicators for size and sector but not for ‘innovativeness’. We would opt for the norm in sampling technique, which is a stratified random sample where firms are stratified by size as measured by employment. A full discussion of sampling techniques can be found in Chapter 2. We would also recommend that every effort should be made to develop a sampling methodology that provides the basis for panel data in subsequent rounds of the innovation survey. 4.3 Tailoring Questions to the Local Context Current weaknesses in innovation surveys also stem from the disparate understanding of concepts such as ‘innovation’ and of the value of Likert-type scales where rankings are subjectively based on different meanings assigned to rankings of ‘importance’ or ‘significance’. These problems are being dealt with by the use of pre-coded responses that provide examples to the uninitiated. Care has also been taken to tailor questions and definitions to the context in Africa. Thus the questionnaire in Annex 1 eliminates nearly all open-ended questions in favour of response forms that are both easier to understand and to code. These include the use of Likert-type scales, yes/no answers and ticking of appropriate choices. 5.0 Enhancing the Policy Relevance of the Survey Instrument

This section addresses the problem of creating a more policy-relevant, systems-oriented questionnaire for a NEPAD innovation survey in five ways. These include

• A conceptualization of innovation that better corresponds to the realities of

innovation in a developing country context. • Indicators that make it possible to use data from the survey in the creation of

regional development policies,

Page 75: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

75

• Indicators that enable policy-makers to profile firms with a higher propensity to innovate

• A focus on questions that have been shown to provide data of relevance to policy makers,

• The addition of questions that provide feedback on existing innovation-related policies and programme.

5.1 Conceptualizing Innovation In a developing country context, innovation is understood to involve the process by which firms master and implement the design and production of goods and services that are new to them, irrespective of whether they are new to their competitors, their countries or the world. Many small improvements in product design and quality, changes in the way production is organized and knowledge managed, the introduction of new maintenance routines, creativity in marketing and modifications in production processes and techniques will collectively bring costs down, increase efficiency and flexibility to respond to changes in competitive conditions and ensure environmental sustainability. In this manner innovation will not be an end in itself but contribute to long-term economic and social benefits as well as to competitiveness. Most developing countries introduce process innovations through the purchase of machinery and equipment and through the licensing-in of technology. With this as its point of departure, the questionnaire broadens the definition of innovation to include a wider set of changes in products, processes, organization and marketing than in earlier surveys, accepts as ‘innovation’ changes that are new to the firm even if they are not new to the country or the world, regards the purchase of new machinery and equipment and recent licensing-in of technology as innovation and examines the extent to which licensing gives rise to learning and innovation in the future. These differences are reflected in questions (Q) about innovation practices that include

• The introduction of new waste management, maintenance and quality control routines, new ways of organizing production and marketing, including through sub-contracting relationships (Q30)

• The purchase of new machinery and equipment from within the country or abroad

over the previous three years (Q27, Q28)

• Whether the firm has a licence contract for product or process technology, the year in which it was obtained and whether it was obtained from a local or foreign firm or research institute (Q21, Q22, Q23).

• The impact of licensing on learning (Q24).

Page 76: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

76

5.2 Utility for Regional Development Policies Clusters are geographical agglomerations of firms and ancillary services. Historically firms have spontaneously tended to cluster near markets, power sources as in the emergence of towns specialized in textiles where mills could take advantage of hydro-power, or where the necessary inputs, including in the case of natural resource based industry, the right climatic and soil conditions are available. Clustering is also believed to offer unique opportunities, especially for SMEs to engage in the wide array of domestic linkages between users and producers and between the knowledge producing sector (universities and R&D institutes) and the goods and services producing sectors of an economy that stimulate the learning and innovation needed to transform traditional industries. Within a cluster, stable vertical relationships between users and producers, for example, can reduce the costs related to information and communication, the risks associated with the introduction of new products and the time needed to move an innovation from the laboratory or design table to market. Horizontal collaboration between same- sector small and medium-sized enterprises can also yield ‘collective efficiencies‘ (Schmitz: 1997) in the form of reduced transaction costs, accelerated innovation through more rapid problem- solving and greater market access. Still other studies have pointed to the positive externalities generated by agglomerations in the availability of skilled labour, of certain kinds of infrastructure, of innovation-generating informal exchanges and learning made possible through the adoption of a variety of embedded institutions (Storper: 1995; Maskell: 1996; Maskell & Malmberg: 1999). Yet not all ‘clusters’ are innovation systems. The potential for interactions and externalities that physical proximity affords may not be realized. Policies, however, might be designed to provide the necessary stimulus and support for innovation to firms located in such agglomerations. The first step in doing so would be to identify agglomerations of firms located in proximity to each other, map their sectors as a proxy for possible common interests and needs and analyze their innovative performance. The questionnaire provides data that can be used to create a set of simple and complex indicators of relevance for the development of regional development policies and support services. These include

• The identification of firms that are in physical proximity through their postal codes, (Q4)

• The firm’s industrial sector as a proxy for possible common interests and needs,

(Q14)

• The firm’s innovative performance as reflected in whether it has innovated and the type(s) of innovation it has undertaken (Q30), as a first step in analyzing the current impact of ‘clustering’ on economic (Q16, Q17, Q19) and innovative (Q35) performance,

Page 77: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

77

• The firm’s past linkage behaviour as a proxy for the potential to stimulate intra-cluster linkages (Q34)

5.3 Profiling the Innovative Firm In designing innovation policies there is a need to better understand the habits and practices of actors with respect to learning, linkages, investment and their performance with regard to innovation. It is not always possible to survey all firms and thus the ability to profile the potentially innovative firm, as a target for policy and support programmes is useful. A set of questions included in this survey instrument collectively provides an indicator of the firm’s propensity to innovate. The profiling technique draws upon established relationships in the literature on innovation, notably in developing countries. The propensity to innovate is related to

1. Location of the firm within or close to a major urban area (two such urban areas can be pre-designated as an explanatory note) and thus in greater proximity to sources of new knowledge and ease in participating in knowledge flows. (Q3, Q4)

2. Educational level of the Owner/CEO/Manager, especially a degree from

a technical university or engineering programme that stimulates and facilitates problem solving. (Q8)

3. Global exposure through training, work or study abroad which opens

opportunities for networking for knowledge flows and collaboration and creates an awareness of the utility to do so. (Q9)

4. Ownership structure of the firm, which influences the choice of

products and processes as well as their subsequent modification or change. (Q11, Q12, Q13)

5. The firm’s sector, which provides a measure of the stimulus to

innovation resulting from the higher R&D intensity of the sector and nature of competition within the sector. (Q14)

6. The size of the firm, which is related to its access to resources to and

opportunities for knowledge scanning to support a process of innovation. (Q15, Q17)

7. Exports (as a percentage of sales) and whether this is rising as an

indicator of the firm’s competitive interests and abilities. (Q19, Q20)

8. Habits and practices of innovation as reflected in having innovated, in at least one of the ways noted in Q30.

Page 78: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

78

5.4 Enabling Policy-Makers to better understand the Innovative Behaviour of Firms and its Relationship to the Performance of Firms. The questionnaire provides two ways of doing this. A first set of questions enables analysts to map the innovative behaviour of the firm in terms of the various types of innovative activities undertaken by the firm. These may include licensing and learning through licensing (Q21, Q22, Q23, Q24), the acquisition of new machinery and equipment (Q27, Q28, Q29) and activities giving rise to the introduction of new products, processes, forms of organization and marketing (Q30). Additional questions examine the various sources of knowledge and information accessed by the firm (Q25, Q26, Q32, Q33, Q34, Q38, and Q39) and the various stimuli to innovation (Q32) and obstacles that the firm encounters in undertaking innovation (Q36). A second set of questions enables the analyst to relate the innovative behaviour of the firm to its overall performance. These questions ask respondents to provide information on the performance trend of the firm with respect to employment, sales, exports and patenting over the previous two years and to assess the impact of innovation on their performance. (Q16, Q18, Q20, Q35, Q40) 5.5 Contributing to the Design of Innovation-Related Policies The innovation survey will provide input into the design of innovation-related policies in two ways. First is by asking firms that have innovated to identify those factors that have motivated them to engage in these innovative activities. This provides feedback on the macro- policies that shape innovation processes. Second is by opening space in the questionnaire for the addition of questions that provide feedback on specific innovation-related policies and programmes in the national context. These questions would be developed in collaboration with the national agency that will administer the innovation survey. Although the results of innovation surveys are based on data that are often two or three years old, this represents one of the rare opportunities for countries in Africa to obtain feedback on the impact of the broader macro-economic environment and specific innovation-related policies and programmes on innovative behaviour. These data might thus encourage further efforts at policy/programme monitoring and evaluation and stimulate a process of policy dialogue, policy learning and adaptive policy-making at the national level as well as opportunities for harmonization across countries through the African Union. 6.0 Proposed Questionnaire

Page 79: DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR … · DESIGNING A POLICY-RELEVANT INNOVATION SURVEY FOR NEPAD A Study prepared by UNU-INTECH Maastricht, UNU-INTECH, 31 August 2004

79

The proposed questionnaire is found in Appendix 1. It contains four broad sets of questions.

1. General information on the firm:

• Name of the enterprise (Q1), • Year established (Q2) • Address and postal code (Q3, Q4, Q5) • Name & position of respondent (Q6, Q7) • Educational level of the owner, CEO or top decision maker in the firm (Q8) • Global exposure of the owner, CEO or top decision maker in the firm through

training, work or study abroad (Q9) • Number of scientists and engineers employed in the firm (Q10) • Ownership structure of the firm (Q11, Q12, Q13) • Main activity of the firm (sector classification) (Q14) • Firm size (number of employees, total exports, total sales and their evolution)

(Q15, Q16, Q17, Q18, Q19, Q20) 2. Innovation indicators:

• Licensing and Learning (Q21, Q22, Q23, Q24) • linkages (Q25, Q26) • New machinery and equipment purchases (local/imported) and expenditures for

them (Q27, Q28, Q29) • Innovative activities (Q30) • Reasons for innovating (Q31) • Sources of information for innovation (Q32, Q33) • Innovation collaboration and cooperation (Q34) • Impact of innovation on firm performance (Q35) • Obstacles to innovation (Q36)

3. Science and technology indicators:

• Number of persons engaged in research and development (R&D) (Q38) • R&D activity and expenditures (Q37, Q39) • Patents granted over the last two years (Q40) • Use of the internet (Q41, 42)

4. Policy Impact

• Importance and Impact of innovation-related policies (Q43, Q44, Q45) • Obstacles to innovation (Q36)