Our space epart2014

Co-Funded project by Competitiveness and Innovation Framework Programme – ICT Policy Support Programme (ICT PSP)Co-Funded project by Competitiveness and Innovation Framework Programme – ICT Policy Support Programme (ICT PSP)

Lessons learned from the OurSpace project

Peter Parycek, Michael Sachs, Florian Sedy, Judith Schoßböck

Danube University Krems (Austria)

Co-Funded project by Competitiveness and Innovation Framework Programme – ICT Policy Support Programme (ICT PSP)

Agenda

oAbout OurSpaceoProject Results: OverviewoEvaluation MethodologyoDetails on the Evaluation MethodologyoProject Results: DetailsoLessons Learned


About OurSpace


Main objective(s)

oCross-boarder eParticipation platform for youth deliberation• Creating a community: focus political topics• Include decision makers and institutions• Learning process: make young people familiar with• decision makers, • governmental/political institutions and • the decision making process.


Challenges

o Involvement of • Young people• Decision makers• Public institutions / non government organisations

oOutcome oriented discussion • 4 Phases deliberation model

oDiscussions with 4 languages• Moderation and automatic translation option

o Integration of Social Media and Networks• Facebook connect, social media marketing


4 Pilots in 4 Countries

o4 different approaches to participation and user engagement• AT (Danube University Krems): Workshops in schools, youth

events, internet-communities

• CZ (DUHA): Political youth organisation that mainly used a combination of political offline events with the web platform, stands at youth events, TV and radio

• GR (Cafe Babel and NTUA): Mass media, political media, workshops at universities and schools

• UK (BYC): Political youth organisations that mainly used their existing network to reach the youth via the internet


Deliberation Process in 4 Phases

1. Suggest and rate topics2. Discuss a topic and provide comments and proposals

(solutions); rate comments and proposals3. Rate the pre-selected best proposals4. Show results and get feedback from relevant bodies and

persons. (up to 10 different organisations)


Sustainability

oA self moderated platform • Theme can be suggested by users• Users shall take over moderation

oSoftware• Open Source• Modular Tools for quick adaptation • Low costs


Project Results: Overview


Schedule

1 March 2011

1 January 2012

1 April 2012

1 September 2012

31 December 2013


Results Overview: Platform activityoPlatform data

31 Dec 2013 11 Feb 2014

Users 3630 4113

Posts 4749 4831

Thumbs 6035 6068

Threads 543 566

Finished debates 45

Engaged Decision Makers 20

o Project end 31 December 2013o Engagement campaign ended mid February 2014o Platform activities ended March 2014


Results Overview: Progress

Mar.1

2

Apr.12

Mai.1

2Jun.12

Jul.12

Aug.12

Sep.12

Okt.12

Nov.12

Dez.12Jän.13

Feb.13

Mär.1

3

Apr.13

Mai.1

3Jun.13

Jul.13

Aug.13

Sep.13

Okt.13

Nov.13

Dez.13

0

1000

2000

3000

4000

5000

6000

Platform Activity

Threads Users Thumbs Posts


Platform data per country

Until 1 Feb 2014 Users Threads Posts Thumbs

EU - 168 543 606

AT 863 44 210 471

CZ 594 127 1.504 2.444

GR 1.600 163 2.228 2.269

UK 1.027 56 298 271

Total 4.084 390 4.240 5.455


Progress per month per country

385 810

712 375 1035


Results Overview: Google Analytics

o In February 2014, the platform had since the beginning • 52.000+ visitors,• 29.000+ unique visitors,• 338.000+ page visits and

oThe average user looked at • 6+ pages and stayed for• 6+ minutes.


Evaluation Methodology


Developed on the basis of …

o Macintosh, A. & Whyte, A. (2008). Towards an Evaluation Framework for eParticipation. Transforming Government: People, Process and Policy, 2(1), 16-30. (GRAPHIC)

o Lippa, B. (ed.) (2008): D13.3 DEMO-net booklet eParticipation Evaluation and Impact, Available at: ics.leeds.ac.uk/.../DEMOnet_booklet_13.3_eParticipation_evaluation.pdf

o Aichholzer, G., Westholm, H., 2009, Evaluating eParticipation Projects: Practical Examples and Outline of an Evaluation Framework, European Journal of ePractice, No. 7, March, 27-44.


4 levels, 11 indicator categories

oPolitical level• Relevancy and popularity of selected

deliberation themes• Effectiveness of communicating the

trial results to decision makers and relevant public bodies

• Degree of influence on decision-making process and political actions

o Technical level• Platform and tools usability• Platform Purpose suitability

o Social level• Effectiveness of integrating multiple

communication tools • Digital connections created between

users • Quality of discussion and deliberation

process

oProject / Methodological level• Effectiveness of the deliberation model

(Method) • Effectiveness of dissemination

activities (Engagement)• Effectiveness of user engagement

tactics (Engagement)


Methods o Short Questionnaire

• 7 questions for registered users only• Permanently on the platform • 76 responses in LimeSurvey

o Long Questionnaire• 20 (+4 demographic) questions • Final 2 month of the project• 420 responses in GoogleForm

o Interviews• 12 users • 6 decision Impact on users groups• 3 e-part experts

oDiscourse and media analysis• Discussion and language analysis• Relevance and popularity of themes

oMonitoring Tool• Planning and coordination of

engagement actions • Regular assessment of engagement

strategies and impact

oData• Platform data, Google Analytics• Data from newsletters, social media,

etc.

o Focus Group• OurSpace consortium• Final project month• Analysis of entire data and project


Application of the methodologyo A total of 11 indicator-categories were measured with 51 indicators.

o Each indicator was measured with at least one tool.o Benchmarks were mostly defined before the evaluation (quantitative).


Details on the Evaluation

Methodology


Rela

tion

of

Ind

icato

r C

ate

gori

es

to r

esu

lts a

nd

exp

ecte

d

ou

tcom

es

O1

O2

O3

O4

O5

O6

O7

R1

R2

R3

R4

R5

R6

R7

R8

R9

PC1

PC1

PC3

TC1

TC2

SC1

SC2

SC3

MC1

MC2

MC3

Objectives – Indicator-Categories – Expected Results


List of objectives# List of objectives of the OurSpace project

O1Establishing the relevant OurSpace Network comprising of the members of the OurSpace consortium, including also, key players outside of the consortium, a fact that will bring forward new policy proposals and will provide relevant feedback to the public.

O2Successfully Deploy an Innovative communication area for democratic participation: Setting up a web ‐based platform which will allow the youngsters from different countries to discuss and share common issues and create online debates

O3Strengthening and enhancing transparency and accessibility of dialogue: Formalising the online deliberation process with a particular emphasis on cross border deliberation and their inherent cultural, ‐lingual and other challenges

O4Engage a broad range of young EU citizens to participate by using the tool to deliver true value through collaborative participation. Organizing events & communication channels in order to strengthen and further explore OurSpace network to a wider audience‐

O5Feed the results of the debates to the relevant National & EU governmental organisations: Creating a feedback framework that will deliver results to the decision makers and will encourage them to respond ‐back to youngster via the platform

O6 Evaluate the levels of success of the project trials: Conducting an overall evaluation of the pilots’ efficiency and added value

O7 Create a sustainable strategy for project achievements and outcomes: Using the outcomes to understand the best practices for achieving positive impact


List of expected results

# List of expected results of the OurSpace project

R1 Tool to reduce the complexity of EU decision making processes is in place

R2 Citizens are more informed through the use of ICT

R3 Capacity is enhanced on the existing ICT framework

R4 Opinions

R5 Educate young people in the role of the European parliament and associated democratic actors

R6 Improved trust in politics and balance expectations

R7 Improve the role and function of the Youth Parliament and its democratic actors(actually youth organisations)

R8 Sustained involvement of young people in the decision making process across EU

R9 Improved access to youth opinion for across EU decision makers


List of indicator categories# List of Indicator categories

PC1 Relevancy and popularity of selected Deliberation themes

PC2 Effectiveness of communicating the trial results to the relevant public administration Bodies

PC3 Degree of influence on the decision-making and policy formation process

TC1 Platform and tools Usability

TC2 Platform Purpose suitability

SC1 Effectiveness of integrating multiple communication channels (web, mobile, social media) in a single, community-oriented platform

SC2 Digital (or close-up) connections created between OurSpace Users / young citizens that are politically active

SC3 Quality of discussion and deliberation process

MC1 Effectiveness of the deliberation model

MC2 Effectiveness of dissemination activities

MC3 Effectiveness of user engagement tactics / Role of Youth organizations


Ind

icato

r C

ate

gori

es &

In

docato

rs


Example: Details on Indicator Category TC1 and the respective Indicators TC1.1-TC1.3


Project Results:Details


Feedback loop for improvements

oBased on an continuous analysis the project work OurSpace was improved throughout the project:• Platform: e.g. Design and navigation.

• Engagement: e.g. Landing page designed to make registration easier. Pilot operators discussed best and worst practices and adopted their strategies. Promotion with iPad contest.

• Inclusion of decision makers: e.g. Make the tool useful to them. Create their own topic and ask the youth to comment (instead of decision makers commenting on youth opinions on random topics).


Result Overview: Political Level (I) o Experts, users and decision makers agree:

• Topics on the platform are relevant

• The platform has potential for political engagement

but opinions are indifferent about the impact on political work.

o It is hard to get decision makers feedback

oUsers want more participation of decision makers.

Activity level is very different in participating countries:

• Austria: one third of MEPs, 1 Secretary of State and 1 Regional Politician

• Greece: 0


Example of improvements:Decision makers featured on front page


Result Overview: Political Level (II)

Long Questionnaire: Does OurSpace provide content that interests you?

Yes Rather yes Neutral Rather no No Responses

AT 48 % 21 % 19 % 5 % 8 % 166

CZ 42 % 36 % 1 % 16 % 5 % 98

GR 44 % 33 % 14 % 7 % 2 % 88

UK 30 % 48 % 16 % 3 % 3 % 61

ALL 43 % 31 % 13 % 8 % 5 % 413


Result Overview: Political Level (III)

Long Questionnaire: Do you think that platforms like OurSpace are good to get involved or more interested in politics?


AT 46 % 27 % 14 % 7 % 7 % 169

CZ 58 % 23 % 7 % 2 % 9 % 43

GR 25 % 42 % 25 % 5 % 4 % 85

UK 42 % 47 % 6 % 2 % 3 % 64

ALL 40 % 35 % 13 % 6 % 7 % 357


Result Overview: Technical Level

oUsers were satisfied with technical featuresoUsers liked the improved design oMobile access was irrelevant as functionality was o Suggestions for improvement:

• Enhance visibility, interconnectivity & network aspect„Add option that allows members to create groups, like political parties, etc.” (GR, Questionnaire)

• More options in personal profile (similar to upcoming social networks)

• Improve guidance

• Modern design


Example of improvments: Page guide implemented


Result Overview: Social Level oRegistration:

• One click registration through Facebook is popular.

• Entering personal data is a hurdle.

o Limited face2face meetings• takes a lot time from all parties.

oVarious and mass media promotion channels generate a diversity of users

o Tone of discussion: friendly and on an equal level esp. with Decision Makers

oCross-country deliberation: language as major hurdle


Example of improvments: Reaching various diverse users

TV-Spot

Workshops and Youth EventsUpdated Flyers


Result Overview: Social Level (II)

Long Questionnaire: Are you satisfied with OurSpace as a platform for political debate?


AT 34 % 34 % 20 % 4 % 8 % 166

CZ 47 % 30 % 16 % 5 % 2 % 43

GR 24 % 43 % 24 % 5 % 5 % 88

UK 32 % 38 % 17 % 8 % 5 % 63

ALL 33 % 36 % 21 % 6 % 5 % 363


Result Overview: Social Level (III)

Long Questionnaire: Did you get more interested in the work of a politician that you met/that posted on OurSpace?


AT 18 % 25 % 30 % 10 % 17 % 167

CZ 17 % 28 % 16 % 19 % 21 % 43

GR 6 % 14 % 42 % 15 % 22 % 85

UK 21 % 19 % 37 % 6 % 16 % 62

ALL 15 % 22 % 32 % 12 % 18 % 357


Result Overview: Methodological Level o The 4-phases deliberation model was understood • but guidance by moderators was necessary.

o Engagement actions were continuously improved

There was steady growth of users and activity.

oVariety of dissemination strategies/methods and holistic, country and user-group specific approaches are necessary:• Combination of on- and offline actions. Offline events needed an internet

connection as transition of offline to online media is otherwise difficult.

• Political youth organisations quickly bring active users but hardly attracted users beyond their direct outreach.

• Marketing and promotion take a lot of effort and must fit the target group (70 % of users were between 16 and 24 years old).


Example of improvments: Online youth sites instead of newspaper


Platform Activity & Engagement

Test StageOpening Events (esp. CZ)

TV-spot in GR, offline events in CZ, referendum in AT

Yo!-Fest in Brussels (mainly UK), TV spot in GR

iPad-Campaign, school workshops and decision makers involvement in AT


Lessons Learned


Discussions

o Different levels of activity in pilot countries:• Concerning discussions, users from CZ and GR were most active:

• Controversial discussions in GR.• Active political youth organisation in CZ.

• Active participation of decision makers AT, CZ and UK. • Not in GR because they didn‘t want to expose themselves to

difficult and heated discussions.o Controversial and heavily promoted topics were most successful on

the activity level. Especially in GR discussions were long and intense. o Moderation of topics:

• Online (in particular reminders for MEPs, changing comments into proposals, sometimes to delete a reported post)

• Offline (e.g. workshops – guidance on „how it works“)


Inclusiveness and transparency

o Variety of promotion activities leads to a diverse group• brings also those users to the platform that are not initially interested in

politics. Reaching only out to an interested audience shows limits in user growths.

o Language is the major obstacle in cross-country deliberation.• Automatic translation tools have limits. International discussions were

usually held in English and only a few posts replied in another language.o The EU-discussions were only active when promoted intensely.

• Users were more interested in their language version of the platform.

o Registration via Facebook & Co is important:• Registration was mandatory for all activities, but typing your data is a

major hurdle. Registration through Facebook-Connect was more appreciated.

• However, not all activities in an e-participation should require user registration.


Deliberation model

o The principles of the 4-stages model was understood.o The potential of the structured and outcome-oriented

deliberation was recognised by most users.o Some detailed aspects of the process where not noticed by many

users: e.g. the difference between a comment and a proposal in phase 2 (discussion phase).

oModeration is important in such a process.oGuidance and design must support the usage of the features.• Continues minor changes of the platform improved the

understanding of the users. • Collaboration with users from the very beginning will

increase the success of a platform.


Barriers

oUsers think that the impact of the platform on decision makers is only inspirational.

oDecision makers commented that such platforms are important to get feedback from the people.

oUsers were sceptical if such a platform could increase trust in politics.

oUsers are interested in discussing general standpoints, but complex issues and concrete legislation cannot be discussed in details.

oOnly those users that are highly interested in politics continued to engage in discussions.

oUsers wonder why they should discuss on OurSpace while they could do that directly in their social networks.


Thank you !

www.joinourspace.eu

http://www.joinourspace.eu/

Our space epart2014

Science

decision making

decision makers

social level

social media

political

lessons learned

activity level

ict psp