Click here to load reader
Click here to load reader
May 24, 2018
EVALUATING THE INDUSTRIAL INDIRECT EFFECTS OF TECHNOLOGYPROGRAMMES: THE CASE OF THE EUROPEAN SPACE AGENCY (ESA)
B.E.T.A, Universit Louis Pasteur, Strasbourg, France
The aim of this paper is to present and discuss the experience in policy evaluation methods andpractices related to large-scale technological programmes. The presentation will rely on theevaluation of the European space programmes, as an archetype of technological (mission-oriented)programmes focused on a highly specific and narrowly predetermined end product (such as a launcheror a satellite).
Various methods have been used by major space agencies in the United States and Europe tomeasure the economic returns to space-related research and development. A number of approacheshave been taken, including microeconomic analysis of specific technologies as well asmacroeconomic modelling of long-term productivity gains. Most of these approaches have estimatedvery positive returns to investment in space. Since the 1960s, economists have tried to measure theeconomic impact of space programmes with a variety of tools. The levels of expenditure involved inthese programmes are so high that public opinion is increasingly calling for an assessment of thetangible benefits accruing to the economy in return for the considerable sums invested. To this end,macroeconomic analysis combined with econometric tools has been used to assess the global impactof space expenditures (macroeconomic modelling, influence of R&D expenditures on amacroeconomic production function, etc.). A separate approach was used to evaluate the economicactivity and employment directly induced by space programmes in the space industry and its suppliers(input-output analysis, use of economic multipliers). Other studies have focused on the impact of theuse of meteorological or communication satellites on weather forecasting or activities related totelecommunications, as well as on the evaluation of space technology transfer policy (through theanalysis of some of the markets created around or fertilised by space technologies). However, thewide variety of simplifying assumptions behind these models means that no single model can provideconclusive results. Moreover, many controversies remain as to the interpretation of the results. Forinstance, trying to justify large-scale programmes through the existence of some successfultechnological spinoffs (such as the Teflon case) was strongly criticised, and this has cast doubt onthe evaluation procedures adopted.
This paper is one element in the methodological debate on the economic effects of large spaceprogrammes. It aims first to present the methodology designed by the Bureau dconomie Thoriqueet Applique of the University of Strasbourg (B.E.T.A.) to evaluate what we consider to be the mostspecific economic effects of those programmes: the indirect industrial effects, also called the spinoffeffects. As a practical example of the use of the methodology, we will describe the measurement ofthe economic impacts of projects implemented by the European Space Agency (ESA). Thisevaluation has two objectives: first, to obtain distinct quantitative figures that can be used to test theeffectiveness of a particular programme in order to justify or not public-sector financial commitmentsby providing a minimum approximation of the indirect industrial effects of ESA contracts; thesecond objective is informative and prescriptive. It does not call into question the established statusof the programme, but rather attempts to improve its effectiveness by analysing its economic,scientific and organisational impacts on those involved in the project and on their corporateenvironments. In other words, it depicts the behaviour and requirements of industry in relation to themanagement of the diffusion of technology and know-how.
In addition to the presentation of the B.E.T.A methodology, this paper will discuss some of themain issues and recommendations relating to the evaluation of government programmes designed tostimulate the economy, based on the lessons learned from evaluations of large-scale technologicalprogrammes. The main issues arise from the two main types of evaluation of large-scaletechnological programmes.
The first type, the evaluation of the social effects, addresses the direct use of the projects endproduct (does the use of a meteorological satellite really improve meteorological forecasts and howcan we evaluate the economic impact of these potential improvements?). From this perspective,methodologies which follow more or less closely the classical cost-benefit analysis approach wouldseem to be adapted because it is generally possible to identify the activities (agriculture,transportation, etc.) affected by the programme. Then, for each type of activity, one could definevariations of the demand curve that would lead to estimations of consumers surpluses using aclassical static comparative analysis. Use of this general framework raises a number of difficulties(what is the real nature of impact for each activity, how can the impact be accurately quantified, howcan the relevant costs be assessed, how can alternative scenarios be assessed, etc.), but most of thetechniques used have the same analytical perspective.
The second type, evaluation of the industrial effects, addresses the problem of evaluating thespread of knowledge arising from the programme and its diffusion throughout the economy. Theseeffects stem from the contractual relationships between the space agencies and the contracting bodies(firms and laboratories) that carry out the project. Evaluating these effects through conventionalstatic comparative methods does not lead to satisfactory results: how can one identify the markets oractivities which might have been fertilised by the industrial knowledge gained from the projectwhen the routes and forms taken by the diffusion of knowledge are a priori totally unpredictable?The risk is that one would tend to select with a strong bias those markets where people know that theimpact is positive. The risk of bias is too strong but, even if such analysis were feasible, it would notcapture the essence of the dynamics of the diffusion of knowledge. This is why we propose todevelop, validate and improve methodologies based on direct interviews with the contracting bodies,which is where the process of dynamic diffusion of knowledge originates.
The industrial effects are two-fold:
First, the direct industrial effects comprise the effects which are directly related to theobjectives of the project as defined in the contractual relationship between the agency andthe group of contractors. These effects arise from the establishment and operation of anindustrial infrastructure (launcher, satellite, etc.), mainly on account of the stimulation ofactivity (measured in terms of level of production and net job creation).
Second, the indirect industrial effects correspond to the effects in terms of creation of newknowledge, transfer of technology, building up of new competences, qualityimprovements, acquisition of new processes, development of new markets, etc., that thecontracting bodies derive from their participation in space programmes and that they areable to use elsewhere. The process expands beyond the frontiers of these contractingcompanies, spreading throughout the economy.
The objectives of the evaluation of industrial effects can be manifold:
First, the evaluation can focus on the measurement, for a given contractor, of the outputsof the knowledge process arising from his participation in a space project. According tothe typology suggested by Schumpeter, these outputs can be classified in terms of new (orimproved) markets, products, technologies, processes, patents, publications, etc. Themethodology adopted by B.E.T.A is partly derived from this classification.
Second, the evaluation can focus on the measurement of the learning effects within a givencontracting firm. These effects include the building or reinforcement of corporatecompetences, the constitution of a critical mass of highly qualified employees,improvements in the acquisition, treatment or diffusion of new knowledge.
Third, the evaluation can focus on the measurement of the learning process betweencontracting bodies. This aspect is becoming increasingly important in terms of theefficiency of the network of contractants in a programme. It is particularly relevant in thecase of international programmes, such as the European space programmes, which arebased on co-operation between firms from different countries. It is also important withinthe group of contractors the performances and problems specific to, say, SMEs or researchlaboratories.
Fourth, the evaluation can focus on the diffusion of the knowledge gained by thecontracting bodies to other sectors. This raises the critical issue of technological transfersfrom the space sector to other sectors, and to space from other sectors.
In order to measure all these effects, we propose a very accurate methodology of directinterviews. The main features of the methodology are described in this paper (samplingprocedures, identification and quantification of effects, etc.). However, two extremelyimportant points relating to the methodology need to be emphasized:
First, for various reasons (forgetfulness, confidentiality, human or material impossibility,etc.), many of the effects cannot be evaluated. This is why we propose to evaluate theminimum results each time there is a doubt or where a range of values has to be takeninto account, we will systematically record the lower limit.
Second, the survey must be carried out in confidence between investigators andcontracting