Using Social Networks Using Social Networks Methodology to Evaluate Methodology to Evaluate Research & Development Research & Development Programs Programs Franco Malerba Franco Malerba CESPRI CESPRI Luigi Bocconi University Luigi Bocconi University & Nicholas S. Vonortas Nicholas S. Vonortas Center for International Science and Technology Policy Center for International Science and Technology Policy and Department of Economics and Department of Economics The George Washington University The George Washington University Joint AEA/CES Meetings Joint AEA/CES Meetings Toronto, Canada Toronto, Canada October 27, 2005 October 27, 2005
48
Embed
Using Social Networks Methodology to Evaluate Research & Development Programs
Using Social Networks Methodology to Evaluate Research & Development Programs. Franco Malerba CESPRI Luigi Bocconi University & Nicholas S. Vonortas Center for International Science and Technology Policy and Department of Economics The George Washington University Joint AEA/CES Meetings - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Using Social Networks Methodology Using Social Networks Methodology to Evaluateto Evaluate
Research & Development ProgramsResearch & Development Programs
Franco MalerbaFranco MalerbaCESPRICESPRI
Luigi Bocconi UniversityLuigi Bocconi University&&
Nicholas S. VonortasNicholas S. VonortasCenter for International Science and Technology PolicyCenter for International Science and Technology Policy
and Department of Economicsand Department of EconomicsThe George Washington UniversityThe George Washington University
PRIOR RESULTS ON IST-RTD PRIOR RESULTS ON IST-RTD NETWORKSNETWORKS
Prior Results Prior Results European IST RTD NetworksEuropean IST RTD Networks
The network of research collaborations has:The network of research collaborations has:
A self-organizing structure, dominated by A self-organizing structure, dominated by “hubs”, which are also key nodes in National “hubs”, which are also key nodes in National research networksresearch networks
A scale-free architecture at the thematic levelsA scale-free architecture at the thematic levels
Prior ResultsPrior ResultsEuropean IST RTD NetworksEuropean IST RTD Networks
European research is characterized by “small world” European research is characterized by “small world” connectivityconnectivity
Strong tendency of scientists to cluster around national Strong tendency of scientists to cluster around national communitiescommunities
Strong tendency to cluster with research disciplines and Strong tendency to cluster with research disciplines and within industrial sectorswithin industrial sectors
The funding structure has a strong influence on research The funding structure has a strong influence on research co-operationsco-operations
Prior ResultsPrior ResultsEuropean IST RTD NetworksEuropean IST RTD Networks
As a result of the new As a result of the new Integrated ProjectsIntegrated Projects and and Networks Networks of Excellenceof Excellence::
The density of links is higherThe density of links is higher The share of participants in the principal component The share of participants in the principal component
is higheris higher The average path length is lowerThe average path length is lower Large firms and research institutes are more dominant Large firms and research institutes are more dominant
as gate-keepers of collaborationas gate-keepers of collaboration Small companies are “crowded out” relative to FP5Small companies are “crowded out” relative to FP5
Prior ResultsPrior ResultsEuropean IST RTD NetworksEuropean IST RTD Networks
The IST RTD network as a whole has “small world” The IST RTD network as a whole has “small world” characteristics - but this is not true for each and every characteristics - but this is not true for each and every one of its programmesone of its programmes
FP6 is more likely than other research collaboration FP6 is more likely than other research collaboration frameworks to:frameworks to:
Connect universities and industryConnect universities and industry Connect different research themesConnect different research themes Include new Member StatesInclude new Member States Include key patent-holdersInclude key patent-holders Include SMEsInclude SMEs
CURRENT STUDY ON IST-RTD CURRENT STUDY ON IST-RTD NETWORKSNETWORKS
Towards an ERA for IST:Towards an ERA for IST:Overall ObjectivesOverall Objectives
Develop and apply a quantitative analytical framework Develop and apply a quantitative analytical framework for the assessment of the characteristics and for the assessment of the characteristics and performance of networks supported by IST RTD in performance of networks supported by IST RTD in FP5 and FP6.FP5 and FP6.
Analyze knowledge and partnership networks in Analyze knowledge and partnership networks in selected IST RTD domains, concentrating on network selected IST RTD domains, concentrating on network nature, topology, time evolution and effectiveness.nature, topology, time evolution and effectiveness.
Supplement quantitative information with some Supplement quantitative information with some qualitative information, and inter-organizational qualitative information, and inter-organizational networks with inter-personal networksnetworks with inter-personal networks
Towards an ERA for IST:Towards an ERA for IST:Evaluation QuestionsEvaluation Questions
How do the characteristics of the How do the characteristics of the IST-RTD IST-RTD partnership and knowledge networkspartnership and knowledge networks compare with compare with the characteristics of the the characteristics of the global partnership and global partnership and knowledge networksknowledge networks of IST-RTD companies and of IST-RTD companies and with the characteristics of the related global with the characteristics of the related global networks?networks?
How well are the companies participating in IST How well are the companies participating in IST RTD programs positioned in the global partnership RTD programs positioned in the global partnership and knowledge networks?and knowledge networks?
Towards an ERA for IST:Towards an ERA for IST:Evaluation QuestionsEvaluation Questions
How effective are IST-RTD networks as How effective are IST-RTD networks as mechanisms for transmitting knowledge?mechanisms for transmitting knowledge?
Are the Integrated Projects (IPs) and the Are the Integrated Projects (IPs) and the Networks of Excellence (NoEs) creating Networks of Excellence (NoEs) creating leading “knowledge hubs”?leading “knowledge hubs”?
What makes these “knowledge hubs” What makes these “knowledge hubs” effective?effective?
Towards an ERA for IST:Towards an ERA for IST:Evaluation QuestionsEvaluation Questions
To what extent does the prominent network status of To what extent does the prominent network status of certain IST RTD companies of clusters match the EU certain IST RTD companies of clusters match the EU technological leadership in certain areas?technological leadership in certain areas?
Are the global networks of selected “hub” companies Are the global networks of selected “hub” companies with extensive ICT supply chains represented in the with extensive ICT supply chains represented in the FP6 IST RTD?FP6 IST RTD?
Are the perceived national IST “knowledge hubs” Are the perceived national IST “knowledge hubs” well integrated into the FP6 network?well integrated into the FP6 network?
Selection of IST
technology domains
IST-RTDFramework Programme 6
Matching of IPC codes with technological domains
Patent examiners
Field experts
Matching of SIC codes with technological domains
INNET alliances
EP-CESPRI patents/citations
KNOWLEDGE NETWORKS (Ib, IIb, IIIb)
PARTNERSHIP NETWORKS
(IIa, IIIa)
PARTNERSHIP NETWORK Ia
Towards an ERA for IST:Towards an ERA for IST:Network TypesNetwork Types
SIC Codes: AlliancesSIC Codes: AlliancesClassClass descriptiondescription ClassClass descriptiondescription
13111311 Crude Petroleum and Natural GasCrude Petroleum and Natural Gas 48414841 Cable and Other Pay Television ServicesCable and Other Pay Television Services
28212821 Plastics Materials, Synthetic Resins, and Plastics Materials, Synthetic Resins, and Nonvulcanizable ElastomersNonvulcanizable Elastomers 48994899 Communications Services, Not Elsewhere Communications Services, Not Elsewhere
35773577 Computer Peripheral Equipment, Not Computer Peripheral Equipment, Not Elsewhere ClassifiedElsewhere Classified 50455045 Computers and Computer Peripheral Computers and Computer Peripheral
Equipment and SoftwareEquipment and Software
35713571 Electronic ComputersElectronic Computers 50655065 Electronic Parts and Equipment, Not Electronic Parts and Equipment, Not Elsewhere ClassifiedElsewhere Classified
36513651 Household Audio and Video EquipmentHousehold Audio and Video Equipment 67946794 Patent Owners and LessorsPatent Owners and Lessors
36613661 Telephone and Telegraph ApparatusTelephone and Telegraph Apparatus 73727372 Prepackaged SoftwarePrepackaged Software
36633663 Radio and Television Broadcasting and Radio and Television Broadcasting and Communications EquipmentCommunications Equipment 73757375 Information Retrieval ServicesInformation Retrieval Services
36743674 Semiconductors and Related DevicesSemiconductors and Related Devices 73737373 Computer Integrated Systems DesignComputer Integrated Systems Design
36793679 Electronic Components, Not Elsewhere Electronic Components, Not Elsewhere ClassifiedClassified 73717371 Computer Programming ServicesComputer Programming Services
37143714 Motor Vehicle Parts and AccessoriesMotor Vehicle Parts and Accessories 73797379 Computer Related Services, Not Elsewhere Computer Related Services, Not Elsewhere ClassifiedClassified
37113711 Motor Vehicles and Passenger Car BodiesMotor Vehicles and Passenger Car Bodies 87318731 Commercial Physical and Biological Commercial Physical and Biological ResearchResearch
45124512 Air Transportation, ScheduledAir Transportation, Scheduled 87328732 Commercial Economic, Sociological, and Commercial Economic, Sociological, and Educational ResearchEducational Research
Effectiveness of Knowledge HubsEffectiveness of Knowledge Hubs
Hubs as knowledge depositories
Hubs at the cross-road of information and ideas
• Number of Patents • Number of Citations Received• Number of Highly Cited Patents
• Degree Centrality • Betweeness Centrality
Effectiveness of Knowledge Hubs: Effectiveness of Knowledge Hubs: Hypothetical ExampleHypothetical Example
patent citation/patent
betweenness IIIb
degree IIIb
highly cited patent
0
0,02
0,04
0,06
0,08
0,1
0,12
KH IIIa
KH Ia
OTHER Ia
Effectiveness of Knowledge Hubs:Effectiveness of Knowledge Hubs:Hypothetical ExampleHypothetical Example
• closely matches that of global KHs in terms of three variables closely matches that of global KHs in terms of three variables (number of patents, network centralities);(number of patents, network centralities);
• lags seriously behind in terms of the remaining two variables lags seriously behind in terms of the remaining two variables that approximate the quality and the importance of their patent that approximate the quality and the importance of their patent portfolios;portfolios;
the FP KHs seem to perform better in diffusing knowledge the FP KHs seem to perform better in diffusing knowledge through their centrality roles in the networks than in creating through their centrality roles in the networks than in creating powerful and influential portfolios of new ideas.powerful and influential portfolios of new ideas.
Subject 3: LeadershipSubject 3: Leadership
Leadership
Two different definitions of Leadership:
• Technology Leadership:Technology Leadership: the role played by each the role played by each organisation in the innovative process organisation in the innovative process
• Market leadership:Market leadership: the share of revenues in ICT the share of revenues in ICT among EU25 among EU25
Technology Leadership
Technology leadership is defined in terms of two Technology leadership is defined in terms of two concepts:concepts:
• Niche overlapNiche overlap concerns the crowdedness of the concerns the crowdedness of the technological area explored by organisations. Its technological area explored by organisations. Its measure is based on similarity of technological measure is based on similarity of technological antecedents (i.e. co-citation).antecedents (i.e. co-citation).
• PrestigePrestige deriving from the direct technological ties deriving from the direct technological ties between actors (i.e. direct patent citations)between actors (i.e. direct patent citations)
Technology Leadership
Four different kinds of actors:Four different kinds of actors:
• Technology Leaders:Technology Leaders: a key source of knowledge a key source of knowledge spillovers for many other organizations in the industry. spillovers for many other organizations in the industry. Their research activity is focused on the exploitation of Their research activity is focused on the exploitation of opportunities in relatively mature and therefore highly opportunities in relatively mature and therefore highly crowded fieldscrowded fields
• Technology Brokers:Technology Brokers: sources of knowledge in sources of knowledge in relatively new and relatively new and unexploredunexplored fields fields
Technology Leadership
• Technology Followers:Technology Followers: they they do not contribute do not contribute significant spillovers to other organizations and engage significant spillovers to other organizations and engage into relatively mature and crowded technological into relatively mature and crowded technological subfieldssubfields
• Isolate Organisations:Isolate Organisations: they they do not receive direct do not receive direct citations from many other organizations and are citations from many other organizations and are exploring relatively untapped technological subfields.exploring relatively untapped technological subfields.
Technology Leadership:Hypothetical Example
-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
0 1 2 3 4 5 6 7 8 9 10
Crowding
Pre
stig
e
EU-Non FP
EU-FP KH
EU-FP Non KH
Global KH
Average prestige
Average alfa
Technology leaders
Technology isolates
Technology Leadership:Hypothetical Example
This analysis might suggest:
• The number of identified leaders and brokers that participate in the Framework Programme
• The number (and identity) of those who not only participate but they can also be characterized as Partnership HUBs in the Framework Programme.