Evidence Based Policy & Practice in Education Working group on Networking Governance Project Results - Overview September 2010 Marc Van Praet - Cecilia.

Evidence Based Policy & Evidence Based Policy & Practice in EducationPractice in Education

Working group on Networking Governance

Project Results - Overview September 2010Marc Van Praet - Cecilia Göranson

04/20/23 Marc Van Praet 3

Our overall project goalsOur overall project goals

“In order to realize a more evidence based local education and training policy

to develop a network in which both knowledge and experience related to evidence-based policy can be exchanged

to improve the exchange of data between researchers, policy-makers and practitioners “

Local education policy makers in future should play a

brokerage role to facilitate knowledge sharing

04/20/23 Marc Van Praet 4

3 work packages3 work packages

1. Policy Networking on local education policy

- describes PN: social infrastructure in which

brokerage takes place & key factors for success of PN

2. Cipo model and knowledge brokerage - describes used models for brokerage & key factors

for successful brokerage

3. Practical peer review- review method for city visits (networking & brokerage)

04/20/23 Marc Van Praet 5

Work package 1 - NetworkingWork package 1 - Networking

Networking– a solid consortium brings together

research centers, local education policy makers and education practitioners from different EU countries

Search for common denominators – for successful local evidence based

policy networking (PNP : policy network performance)

04/20/23 Marc Van Praet 6

Work package 1: deliverablesWork package 1: deliverables

Suggestions for standardized assessment of policy network performance in local education policy networks

• Report will include:– Literature study– Model for peer-reviewing local education policy networks – Resulting in

• suggestions for common denominators of successful evidence based local education policy networking

• suggestions for further research & development

• Reporters: Marc van Praet & Cecilia Göranson

04/20/23 Marc Van Praet 7

Applied approach / definition of Applied approach / definition of local policy network level:local policy network level: Policy networks are set(s) of relationships

• with stability• endurance• and structure

between multiple actors• who are relatively autonomous

• and resource dependent using processes of resource

exchange by bargaining and negotiating

to achieve public purpose (Voets et.al. , 2008)

04/20/23 Marc Van Praet 8

Why performance assessment of Why performance assessment of local Policy Networks?local Policy Networks? (Local) Policy networks

– have to deliver results (product performance) in a more complex structure than a single organization

– have to focus on process performance to ensure democratic nature of PN

– have to be resilient and robust (regime performance)

(Voets, Van Dooren, De Rynck; 2008)

04/20/23 Marc Van Praet 9

Performance assessment Performance assessment

Research on performance traditionally – on macro level (e.g. country: use of

statistical series) – and on micro level (e.g. organisation: use

of score cards)For policy networks situated on meso

(city) level, traditional approaches for performance assessment do not work.

04/20/23 Marc Van Praet 10

3 levels of assessment in PN 3 levels of assessment in PN PerformancePerformance How does the PN perform towards

participants / organizations within the Policy Network

How does the PN perform towards the network as a whole

How does the PN perform towards the community level: outcome - results of the Policy Network in taming “wicked” problems

(Provan & Milward; 2001)

04/20/23 Marc Van Praet 11

3 dimensions of PN Performance3 dimensions of PN Performance

Product performance• Goal attainment is key criterion: builds trust. Without

goal attainment PN loses trust and ultimately becomes irrelevant (Huxham & Vangen, 2005)

Process performance• Ensuring democratic quality (legitimacy,

accountability, accordance)

Regime performance• PN as capacity builder to tackle “wicked issues”

The role of government is to ensure a balance of performance on the different levels & dimensions (Voets, Van Dooren, De Rynck, 2008)

Performance assessment of local Performance assessment of local policy network(ing)policy network(ing)

Product performance

Process performance Regime performance

Towards its members

(How) does the PN deliver a surplus to its members to attain their goals?

(How) does the PN ensure to its members democratic decision making & accountability?

(How) does the PN include / exclude members and their views?

Towards the network

(How) does the PN create efficiency in attaining its goals?

(How) does the PN create accordance within the PN?

(How) does the PN institutionalize?

Towards public

(How) does the PN “in the eye of the public” attain its goals in “taming the wicked problem”?

(How) does the PN legitimize itself towards the public, how does it give account & create consent?

(How) does the PN ensure capacity building: improve its relationships with government & the public?

04/20/23 Marc Van Praet12

Performance assessment of local policy network(ing) overviewPerformance assessment of local policy network(ing) overview

Product performance

Process performance Regime performance

Towards its members

S: By coordinating resources the individual partners can use the best resources to reach optimum results.The sum of the different perspectives prevents “home blindness”. W: Loss of autonomy to decide on the “own product”.Working in PN means also extra workload: loss of capacity to work on other aspects.O: Learning from others is a gain in its own right.T: Individual partners can “hide” behind the network and avoid responsibility.

Operationally: decisions are reached by voting but mainly by consensus.S: Strategically: process performance of its members is improved by creating awareness about different positions, different mandates and power relations.W: lack of transparency if not all members are aware of what is going on in the networkO: Being linked to a network makes the partner more accountable / credible, improves status.T: Being partner in a network can imply loss of individuality / loss of right to own standpoint or decision.

S: Growing awareness that working together pays off. W: Power balance can change when new members enter or members leave.Some partners are “more equal than others”…O: 1 + 1 > 2 and getting the support from the group improves the impact of the individual member.T: Growing awareness that in networking partners have to give up some of their independence: their views can become less clear, more blurry. Their link with / support by their own backing group can become less strong.

04/20/23 Marc Van Praet13

Performance assessment of local policy network(ing) overviewPerformance assessment of local policy network(ing) overview

Product performance

Process performance Regime performance

Towards the network

S: Keeping the work focused on the main goals.The product of network(ing) is joint commitment towards reaching these end goal(s).W: Not sharing realistic ambitions. Getting over- zealeous.O: Finding common win-win situations is a key factor for getting to the end goal.T: If the winning is only for the happy few it threatens the joint commitment and goal attainment.

S: Accordance through clear agenda setting and by regular meetings of people with a mandate to decide within the network. Preferably by consensus because commitment and trust are vital. W: Too many or too little members can influence the credibility of the PN and its impact. There is an “optimum” to be reached. If PN is done imbalanced this can lead to “explosion or implosion” of the network. Individual member can manipulate (with good or bad intentions) group decisions.O: If the right instruments / structures are used PN can lead to more external acceptance and support.T: Too much external influence (politics, other governance level, media …) can threaten the network as a whole.

S: Structural problems get structural attention & means.W: The network as a structure can become the goal in itself: risk of “belly-button gazing”. O: The network can attain more responsibility (and power / means) from (local) government.T: The network can become a “scape goat” for lack of effective governance.

04/20/23 Marc Van Praet14

Performance assessment of local policy network(ing) overviewPerformance assessment of local policy network(ing) overview

Product performance

Process performance Regime performance

Towards the public / political level / media

S: giving proof that the network (and its partners) are getting hold on the challenge and are reaching at least positive output, measured as objectively as possible by use of pre-defined and bench marked indicators will create public trust.W: If the objectives are set too high, the partners can get the feeling that all efforts are not leading to success and they can give up their efforts. O: PN can influence public opinion by delivering the expected / asked “good product” which no single partner could deliver. T: Wicked problems cannot be solved, so “taming” them is maximum what is aimed at and PN can create awareness with the public that the problem will always stay there... Which also can threaten the existence of the network. Unless: the public creates new opportunities for the PN to reach its goals… and that again is a strength.If the PN is not recognized by the public opinion, it cannot answer to the challenge.

S: The life cycle of a network is different from the life cycle of projects. Group dynamics and multi-linkage to external structures and stakeholders make it a more organic living organism with longer life expectancy than the individual projects.W: Network if invisible / not transparent it does not have to be / cannot be held accountable. (“Sauna networking”)O: If all partners in the PN use the same data, they can create consent about the challenge and the possible solutions. T: Lobbyists can be a threat to correct networking.

S: All kinds of public reporting to all the members and sponsor(s) of the PN supports their focus on the goals. W: In networking in the end no-one can be held individually responsible for the results.O: Although in many cases the network is not a public body, if it can show good results it can – as a pressure group - keep its goals as a public / political priority. T: “Sauna networking”, “golf club or old boys club”,… informal networking can be a threat to official governance (and even to public officials if involved: corruption).

04/20/23 Marc Van Praet 16

Possible review instruments:Possible review instruments: Document analysis Intervision / peer review

• Standardized questionnaires in the form of a presentation to the visiting peers

• Graphic analysis of relations between actors • Consensus validation (Hycner, 1985)• Journal of interpretations (Guba, 1981)

Interviews• Participating observation (difficult in city – visits)• In depth interviews (on the basis of standardized presentations)

SWOT

The PN performance presentation The PN performance presentation and peer – review tooland peer – review tool

For details: see separate presentation

Questions we put to our peers:– Identification of PN– Why network & who ?– Structure of the PN ?– Roles of the partners in PN ?– Procedures & routines ?

04/20/23 Marc Van Praet 17

The PN performance presentation The PN performance presentation and peer – review tooland peer – review tool– Relations & balances in PN– Phases in building the PN– Information brokerage within PN– Evaluation procedures– Critical factors & future plans-> Self S.W.O.T. evaluation vs S.W.O.T.

evaluation by visiting peers-> Comparative analysis of local PN in education

04/20/23 Marc Van Praet 18

Results September 2010Results September 2010 Literature: see brochure 12 standardized ‘practical’ peer review

presentations– graphic presentation of relationships

– key questions on • product performance• process performance• regime performance

Try outs of standardized in depth peer interviews

04/20/23 Marc Van Praet 19

The last 3 months of the project:The last 3 months of the project: Extra peer – presentations & practical peer

reviews (totaling +/- 20 presentations & reviews)

Analysis of first results:– Quality of the tool(s) (PPT, matrix & Swot)– Key factors (denominators) for PN performance ?

End report Suggestions for further development

– of tools and research– of international network on local policy networking

04/20/23 Marc Van Praet 20