imaginary line between level 3 and level 4 where the non monetary value is translated into money isolating the effect of the meeting is what is needed of multiple influences besides the meeting are taking place (advertising, incentives etc) Event ROI Foundation Programme Day 1 Key learning points To change your behavior What behaviour do you want to change? ROI is as much a measurement methodology as a planning methodology How do I define when the results are good? Critieria Objectives Theory Part 1 The essential principles of creating value from events we are in the BEHAVIORAL CHANGE BUSINESS Part 2 The Event ROI Pyramid Level 5 Value -/- Change = Profit calculated in money ROI is profit translated into a % Level 4 Impact Business impact = the change in behaviour that create the VALUE organisational effectiveness more effective less conflicts more quickly more enjoyment Impact of changes = translates into lower costs Level 3 Application How do you apply the learning to change the BEHAVIOUR note : the behaviour doesn't change the value. Level 2 Learning 4 types 1 Information 2 skills learning 3 values and attitudes cognitive change process example of 6 years olds in school with 43 nationalities 4 relationship learning "lernen kennen" learning about another person trust collaborate checklist when developing objectives Level 1 Reaction & Satisfaction Satisfaction & planned actions Only needs to be good enough setting the criteria in level 1 objectives Hospitality & logistics satisfaction Part 3 ROI methodology The chain of impact = > what happens when participants transition through the 5 levels How do you design a meeting with the Plan from the top down What impact are we looking for What do attendees need to doe How do Level 5 down to 1 Measure from the bottom up From level 1 to Level 5 You can then story tell why the event result was good or bad and you demonstrate what was the maximum you could get based on the objectives set MUST -> measure all levels to be able to identify if the ROI is too low and really WHY it went wrong OPTIONAL level that can be added Level 0 do you have the right people coming to the event Who is invited and in the room Then from level 1 do's and dont's Part 4 Exercise Matching Evaluation Levels with Objectives individual work to put in what level the objective is Worksheet Group to agree in 10 minutes on what level the objectives are Definition Stakeholder The person who has something at stake Meeting Owner You cannot evaluate the value of an event You can only evaluate ROI for a stakeholder Different data and objectives for different stakeholders Isolation Methods of isolation 1 Control Group (Placebo) = best method Example of Mobile Telecom Shop trainers Not frequently used because Quite cumbersome Inconvenient not informing the control group 2 Estimation method Easier & quicker Example Car manufacturer sales team trainings Isolation issues In Level 4 you always have to isolate the business impact of the meeting In Level 3 you may choose to isolate the application Design Post event survey for Risk Mngt course Questions Level 1 How will you use what you learned? Discuss with my colleagues Review risk management procedures in my organisation Conduct formal Risk assessments before major events Review how we check on suppliers safety procedures Will you meet with your insurance broker to review current policies Others (specify) No , probably not, Maybe, Probably Yes, Definitely Level 1 Level 1 can be ques- tioned to only a couple of the people Level 2 In the next session I would like you to answer the Q's twice. How well did you know it before the seminar How well did you know it after the seminar Tell me how well do you feel that you know the basic principles of Risk management Before the seminar After the seminar Do you feel comfortable with leading a risk assessment exercise for a major event? Before the seminar After the seminar Could you draw a matrix for classifica- tion for different types of risks? Before the seminar After the seminar Could you list the four different risk management alternatives Before the seminar After the seminar In the next session I would like you to answer the Q's twice. A)-> How well did you know it before the seminar B)-> How well did you know it after the seminar Level 3 Repeat level 2 Questions 3 months later ask for if they applied of the Level 1 questions Format After the event day 3 months later ask for commitment to do level Planned action (level 1) and Level 3 Application is pretty much the same thing but then verified later in time Level 3 Application Ask all the speakers before the event: Level 2 & 3 objectives How to do you apply this at the sessions? This is what you will learn This is how your organisation will benefit what you have learned Is everything you talk about relevant for the change of that behaviour What you measure improves Ask a small sample group as evaluation committee Ask a couple of respected individuals some extra information Ask a small group of participants some extra feedback after the event. (10 minute instead of 3 minute survey The way the question is asked What do you think Day 2 Case study Planning Start the measurement planning when you start the planning Setting the objectives Making the assumptions Check assumptions Data collection Find out how to get 80% response without too much effort Make discussion focused on planned actions Get data around what planned actions do you have when you get back to your daily routine don't focus on Types Surveys Online give reward reminder 4-7 days If you want 100% response Add a text on the unsubscribe -> If you think you cannot find the time to respond to these 5 questions I will assume I can send you a message every day until you respond... Paper in person calibrating the scale call and listen to the low scorer call and listen to the high scorer call 10% or 10 people Timeliness + Metrics to be pre-determined Third Party sources Interviews on the floor video voice interviewer Mystery shopper check skills application ask a participant to Data Data from CRM Number of downloads presentations Data from Social Media Self reporting Test Quiz 19 ways to get high response rates for surveys online Easy to respond to - SIMPLE - Unambigious questions Clearly relevant Questions get commitment to respond Reciprocity - deliver something better and earlier than expected Cover letter signed by person of authority share results anonymous (option to be anonymous) say how long it will take say how long it takes to complete return by date reminders with option to opt out Make it compulsory Must complete to get certificate You will learn something by completing it Professional Design Elements of Humour Method Planning Level 4 What is overarching type of event Opportunity Revenue driving Cost reduction Improve efficiency in use of Information event Learning event to try something new Level 3 What categories are there for the level 3 objectives? First elaborate on the categories then try to cluster if they have overlapping level 4 objectives Take one of the categories Question Ask the client? In a dream / ideal world what would you like your stakeholder to do differently after they attend your event Use the Action Learning Matrix Objectives on Level 3 on top level Horizontally Horizontally write the Objectives we want participants to take 1. Teach Doctors on new diagnotics procedures sothat medics can start treatment earlier and save more patients 2. Key opinion leaders to share their insights to influence other medics 3. Believe that the new diagnostic procedure and provide them materials to change their procedures in their practice Vertically Actions 3 learning objectives A skill Learn the diagnostic skill An attitude change the preference of diagnostic options An information objective LEarn where on the website the case studies are available Make a MATRIX and then grade how important (on a scale of 5 the actions (vertical) is to the objective Handover this document to the Meeting Architect Meeting Architect will develop the level 2 objectives based on level 3 objective He makes the level 2 matrix So example Level 3 = Believe in changing the procedure is the best way Level 2 Do they learn this from dinner? Mention in Speakers message Will they learn this from Satelite symposium session Website template + download requirement How can I make the learning happen? Meeting Architect needs to know what behavior changes need to happen on level 3 and then design the action Level 2 Learning Objectives What Skills need to be learned? Learning Is there some information needed? Learning to know (relationship learning) Do they need to be introduced to someone? Do they need a way Connected to the level 3 objective od that category of participant Level 1 Logistics & good learning environment creating the right learning environment Example Post Work Steps 31 August 2010 ROI certificate of attendance Milestone 1 by 7 September email to Elling with plan Event name First I have to do = Alternative stakeholders Decide which stakeholder I will set objectives & collect data for I will by a realistic date send you my draft at making a complete set of objectives 4 3 2 1 in Matrix format Make survey questions sample Milestone 2 event date + 3 weeks Make the report Write a report Provide a case study Milestone 3 Alumni community become a member share list of objectives Pre-work Reading Show me the Money Beyond Learning Objectives ROI Meetings & Events Itinerary Participants Mireia Iglesias MSI Stakeholders Attendees Exhibitors María José Tiffon Dermatology events organizer SBC Mónica Sala Ultramar Ruud Janssen ©TNOC | The New Objective Objective Collective Case study Participants Mireia Iglesias MSI Meeting Support Institute Meeting Content Conference Stakeholders Participants Meeting Planners & Designers Apply in their job the specific knowledge acquired Meeting formats Objective is that 70% of participants within 3 months after the session will apply the newly acquired Formats in their next event How many new meeting formats would like to use in the next 3 months Motivational Sciences Come back as a repeat visitor Contribute information to the MSI website Spread the news to MSI interested clients through new media MSI organisation Objective Because we see a clear increase in events using MSI member's services MSI to have a pannel (250 members) survey which takes 15 minutes and ask the same Q's every year what percentage of total event budgets are spent on meeting support technology For participants to know that MSI is the professional useful organisation Increase the business for MSI members Invite 20 new potential members What would the chairman write in his opening statement members Exhibitors Maria Dermatology events organizer Dermatology Workshop Shareholders Spanish Dermatology Academy To increase future sales The Sponsor Loreal Attendees Trainees Receive the sales call of Loreal sales people favourably in the future Speakers Monica Ultramar Internal Ultramar Conference Ideas Stakeholders Management of companies Owners of company Employees of Accommodation & Destination Services Ultramar Event Management Destination Speakers Increase sales Level 2 Learning Know where to keep upto date on dev't in the hybrid technology & social media IDentify the multilingual problems of involving hybrid audiences Attitude Using Lickert scale questions Test attitude statements Internal Managers always have the best interest of the employees in mind This event has really changed the importance on client needs and what I can do to change this Client We fully understand the client need before we propose solutions to them Strongly agree - Strongly disagree Relationship Learning Skills Deliver a Pecha Kucha format and see the impact on remote audiences Know how to contribute effectively to the backchannel using microblogging Improve internal communication problem Ruud Janssen ©TNOC Event Camp Twin Cities Level 4 Objective Opportunity // Organizational Effectiveness Learning/Experience event if they apply the new technology learned at this event to their own meeting to the own meetings to actively consider the use of hybrid event elements in the upcoming meeting and start contributing to the #eventprofs community as well as know who can help in as a service provider in hybrid events Level 2 Learning Know where to keep upto date on dev't in the hybrid technology & social media IDentify the multilingual problems of involving hybrid audiences Attitude Using Lickert scale questions Test attitude statements Internal CLient Strongly agree - Storngly dissagree Relationship Learning Skills Deliver a Pecha Kucha format and see the impact on remote audiences Know how to contribute effectively to the backchannel using microblogging Participant Level 3 Objectives Categories of stakeholders? Stakeholders Participants Level 3 question In our minds, Event Camp will have succeeded if you can go home and say the following: “ Yes, I have now seen how hybrid event technology, social media and new collaborative formats can transform events. Yes, we could try ________ at our event. We may not be able to do it without help – but yes we could do it. AND, we could do it 10 times better than those crazy people at Event Camp Twin Cities.” This event is going to be a lot of fun! Level 2 Learning Know where to keep upto date on dev't in the hybrid technology & social media IDentify the multilingual problems of involving hybrid audiences Attitude Using Lickert scale questions Test attitude statements Internal CLient Strongly agree - Storngly dissagree Relationship Learning Skills Deliver a Pecha Kucha format and see the impact on remote audiences Know how to contribute effectively to the backchannel using microblogging To seek the support of profession- als if I can't do it myself Organisers check Minneapolis Interactive Meetings Technology LLC Sam Smith Consultancy Speaking Appevision Ray Hansen App devt Basel ©TNOC Ruud Janssen Online collaboration tools for communi- ties in a collective Collective People Learning Make a list of people you met who within the next 6 months can help you setup an online community Consultancy Speaking Publishing Case study ROI Case study Marketing of Services provided by TNOC Test the waters for Event Camp Europe event Increase Titter following Increase reach Kahlweiss Video Productions Magnus Kahlweiss Video productions Wordcamp experience When they need help they come to the specialist consultant What can I he;p them with How to contact me I now know the I know that Inxpo is the leading supplier in putting Dallas Press MIC magazine Headquarters Magazine One+ ©TNOC | Blog Article Drive traffic Increase Titter following Venue Cafe Tacuba Basel Get to know Cafe Tacuba as a new venue in Basel Action Come back to Cafe Tacuba for meals Use Cafe Tacuba for small meetings Speakers Pechu Kucha Elling Level 2 learning There is awareness of the existence of the ROI methodology Recognized in multiple industries Easily learn how to do it Read the book Attend the certification course Level 3 Share the message of ROI + know where to learn more on ROI methodology Ruud Level 2 Awareness of workings of collective Opportuntiy to use Experience how to present hybrid & Pecha Kucha 10 changes in working in the next 10 years Level 3 Become active participants and become part of the TNOC collective Become contributor to #eventprofs community #Eventprofs community Ideas digital footprint What to measure (Who) Would you recommend attending Event Camp to your industry colleagues in the next 2 weeks Measuring ROI for Events & Meetings Dr Elling Hamso Managing Partner European Event ROI institute Jack Philips President ROi Insititute Elevator pitch The ROI Methodology enables meeting owners to create events that deliver maximum value and prove the value created Event is developed on the basis on evaluation objectives Starting with the impact the event will have on the bottom line 4 levels of meeting Measure success from good learning, environment Problem solved Enables you to ensure you get the maximum value and impact and prove that you did achieve it