imaginary line between level 3 and level 4 where the non monetary value is translated into money
isolating the effect of the meeting is what is needed of multiple influences besides the meeting are taking place (advertising, incentives etc)
Event ROI Foundation
Programme
Day 1
Key learning points
To change your behavior What behaviour do you want to change?
ROI is as much a measurement methodology as a planning methodology
How do I define when the results are good?
Critieria
Objectives
Theory
Part 1 The essential principles of creating value from events
we are in the BEHAVIORAL CHANGE BUSINESS
Part 2
The Event ROI Pyramid
Level 5Value -/- Change = Profit calculated in money
ROI is profit translated into a %
Level 4 ImpactBusiness impact = the change in behaviour that create the VALUE
organisational effectiveness
more effective
less conflicts
more quickly
more enjoyment
Impact of changes =translates into lower costs
Level 3 Application How do you apply the learning to change the BEHAVIOUR
note : the behaviour doesn't change the value.
Level 2 Learning 4 types
1 Information
2 skills learning
3values and attitudes cognitive change process
example of 6 years olds in school with 43 nationalities
4 relationship learning
"lernen kennen"
learning about another persontrust
collaborate
checklist when developing objectives
Level 1 Reaction & Satisfaction Satisfaction & planned actionsOnly needs to be good enough setting the criteria in level 1 objectives
Hospitality & logistics satisfaction
Part 3 ROI methodology
The chain of impact = > what happens when participants transition through the 5 levels
How do you design a meeting with the
Plan from the top down
What impact are we looking for
What do attendees need to doe
How do
Level 5 down to 1
Measure from the bottom up From level 1 to Level 5
You can then story tell why the event result was good or bad and you demonstrate what was the maximum you could get based on the objectives set
MUST -> measure all levels to be able to identify if the ROI is too low and really WHY it went wrong
OPTIONAL level that can be added Level 0
do you have the right people coming to the event
Who is invited and in the room
Then from level 1
do's and dont's
Part 4
Exercise Matching Evaluation Levels with Objectives
individual work to put in what level the objective is
Worksheet
Group to agree in 10 minutes on what level the objectives are
Definition Stakeholder
The person who has something at stake Meeting Owner
You cannot evaluate the value of an eventYou can only evaluate ROI for a stakeholder
Different data and objectives for different stakeholders
Isolation
Methods of isolation
1 Control Group (Placebo)
= best method
Example of Mobile Telecom Shop trainers
Not frequently used becauseQuite cumbersome
Inconvenient not informing the control group
2 Estimation methodEasier & quicker
Example Car manufacturer sales team trainings
Isolation issues
In Level 4 you always have to isolate the business impact of the meeting
In Level 3 you may choose to isolate the application
Design
Post event survey for Risk Mngt course
Questions
Level 1 How will you use what you learned?
Discuss with my colleagues
Review risk management procedures in my organisation
Conduct formal Risk assessments before major events
Review how we check on suppliers safety procedures
Will you meet with your insurance broker to review current policies
Others (specify)
No , probably not, Maybe, Probably Yes, Definitely
Level 1
Level 1 can be ques-tioned to only a couple of the people
Level 2
In the next session I would like you to answer the Q's twice.
How well did you know it before the seminar
How well did you know it after the seminar
Tell me how well do you feel that you know the basic principles of Risk management
Before the seminar
After the seminar
Do you feel comfortable with leading a risk assessment exercise for a major event?
Before the seminar
After the seminar
Could you draw a matrix for classifica-tion for different types of risks?
Before the seminar
After the seminar
Could you list the four different risk management alternatives
Before the seminar
After the seminar
In the next session I would like you to answer the Q's twice. A)-> How well did you know it before the seminarB)-> How well did you know it after the seminar
Level 3Repeat level 2 Questions 3 months later ask for if they applied of the Level 1 questions
Format
After the event day
3 months later
ask for commitment to do level
Planned action (level 1) and Level 3 Application is pretty much the same thing but then verified later in time
Level 3 Application
Ask all the speakers before the event: Level 2 & 3 objectives How to do you apply this at the sessions?
This is what you will learn
This is how your organisation will benefit what you have learned
Is everything you talk about relevant for the change of that behaviour
What you measure improves
Ask a small sample group as evaluation committee
Ask a couple of respected individuals some extra information
Ask a small group of participants some extra feedback after the event. (10 minute instead of 3 minute survey
The way the question is asked What do you think
Day 2 Case study
PlanningStart the measurement planning when you start the planning
Setting the objectives
Making the assumptions Check assumptions
Data collection
Find out how to get 80% response without too much effort
Make discussion focused on planned actions
Get data around what planned actions do you have when you get back to your daily routine
don't focus on
Types
Surveys
Online
give reward
reminder 4-7 days
If you want 100% response
Add a text on the unsubscribe -> If you think you cannot find the time to respond to these 5 questions I will assume I can send you a message every day until you respond...
Paper
in person
calibrating the scalecall and listen to the low scorer
call and listen to the high scorer
call 10% or 10 people
Timeliness + Metrics to be pre-determined
Third Party sources
Interviews on the floor
video
voice
interviewer
Mystery shoppercheck skills application
ask a participant to
Data
Data from CRM
Number of downloads presentations
Data from Social Media
Self reporting
Test Quiz
19 ways to get high response rates for surveys online
Easy to respond to - SIMPLE - Unambigious questions
Clearly relevant Questions
get commitment to respond
Reciprocity - deliver something better and earlier than expected
Cover letter signed by person of authority
share results
anonymous (option to be anonymous)
say how long it will take
say how long it takes to complete
return by date
reminders with option to opt out
Make it compulsory
Must complete to get certificate
You will learn something by completing it
Professional Design
Elements of Humour
Method Planning
Level 4 What is overarching type of event
Opportunity
Revenue driving
Cost reduction Improve efficiency in use of
Information event Learning event to try something new
Level 3
What categories are there for the level 3 objectives?
First elaborate on the categories then try to cluster if they have overlapping level 4 objectives
Take one of the categories
Question Ask the client? In a dream / ideal world what would you like your stakeholder to do differently after they attend your event
Use the Action Learning Matrix
Objectives on Level 3 on top level
HorizontallyHorizontally write the Objectives we want participants to take
1. Teach Doctors on new diagnotics procedures sothat medics can start treatment earlier and save more patients
2. Key opinion leaders to share their insights to influence other medics3. Believe that the new diagnostic procedure and provide them materials to change their procedures in their practice
Vertically Actions 3 learning objectives
A skill Learn the diagnostic skill
An attitudechange the preference of diagnostic options
An information objective
LEarn where on the website the case studies are available
Make a MATRIX and then grade how important (on a scale of 5 the actions (vertical) is to the objective
Handover this document to the Meeting Architect
Meeting Architect will develop the level 2 objectives based on level 3 objective
He makes the level 2 matrixSo example
Level 3 = Believe in changing the procedure is the best way
Level 2
Do they learn this from dinner? Mention in Speakers message
Will they learn this from Satelite symposium sessionWebsite template + download requirement
How can I make the learning happen?
Meeting Architect needs to know what behavior changes need to happen on level 3 and then design the action
Level 2Learning Objectives
What Skills need to be learned?
Learning Is there some information needed?
Learning to know (relationship learning)Do they need to be introduced to someone?
Do they need a way
Connected to the level 3 objective od that category of participant
Level 1 Logistics & good learning environment creating the right learning environment
Example
Post WorkSteps
31 August 2010ROI certificate of attendance
Milestone 1by 7 September email to Elling with plan
Event name
First I have to do =
Alternative stakeholders
Decide which stakeholder I will set objectives & collect data for
I will by a realistic date send you my draft at making a complete set of objectives
4
3
2
1
in Matrix format
Make survey questions sample
Milestone 2event date + 3 weeks
Make the report
Write a report
Provide a case study
Milestone 3Alumni communitybecome a membershare list of objectives
Pre-work
Reading
Show me the Money
Beyond Learning Objectives
ROI Meetings & Events
Itinerary
Participants
Mireia Iglesias MSIStakeholdersAttendees
Exhibitors
María José TiffonDermatology events organizer
SBC
Mónica SalaUltramar
Ruud Janssen©TNOC | The New Objective Objective Collective
Case studyParticipants
Mireia IglesiasMSI Meeting Support InstituteMeeting Content ConferenceStakeholders
ParticipantsMeeting Planners & Designers
Apply in their job the specific knowledge acquired
Meeting formats
Objective is that 70% of participants within 3 months after the session will apply the newly acquired Formats in their next event
How many new meeting formats would like to use in the next 3 months
Motivational Sciences
Come back as a repeat visitor
Contribute information to the MSI website
Spread the news to MSI interested clients through new media
MSI organisationObjective
Because we see a clear increase in events using MSI member's services
MSI to have a pannel (250 members) survey which takes 15 minutes and ask the same Q's every year what percentage of total event budgets are spent on meeting support technology
For participants to know that MSI is the professional useful organisation
Increase the business for MSI members Invite 20 new potential members
What would the chairman write in his opening statement
members
Exhibitors
Maria
Dermatology events organizerDermatology Workshop
Shareholders
Spanish Dermatology AcademyTo increase future sales
The
SponsorLoreal
AttendeesTrainees
Receive the sales call of Loreal sales people favourably in the future
Speakers
Monica
Ultramar
Internal Ultramar Conference
Ideas
Stakeholders
Management of companies
Owners of company
Employees of Accommodation & Destination Services Ultramar Event Management
Destination
Speakers
Increase salesLevel 2
Learning
Know where to keep upto date on dev't in the hybrid technology & social media
IDentify the multilingual problems of involving hybrid audiences
AttitudeUsing Lickert scale questionsTest attitude statements
Internal
Managers always have the best interest of the employees in mind
This event has really changed the importance on client needs and what I can do to change this
ClientWe fully understand the client need before we propose solutions to them
Strongly agree - Strongly disagree
Relationship Learning
Skills
Deliver a Pecha Kucha format and see the impact on remote audiences
Know how to contribute effectively to the backchannel using microblogging
Improve internal communication problem
Ruud Janssen
©TNOC
Event Camp Twin Cities
Level 4 ObjectiveOpportunity // Organizational Effectiveness
Learning/Experience event
if they apply the new technology learned at this event to their own meeting to the own meetings
to actively consider the use of hybrid event elements in the upcoming meeting and start contributing to the #eventprofs community as well as know who can help in as a service provider in hybrid events
Level 2
Learning
Know where to keep upto date on dev't in the hybrid technology & social media
IDentify the multilingual problems of involving hybrid audiences
AttitudeUsing Lickert scale questionsTest attitude statements
Internal
CLient
Strongly agree - Storngly dissagree
Relationship Learning
Skills
Deliver a Pecha Kucha format and see the impact on remote audiences
Know how to contribute effectively to the backchannel using microblogging
Participant
Level 3 ObjectivesCategories of stakeholders?Stakeholders
ParticipantsLevel 3 question
In our minds, Event Camp will have succeeded if you can go home andsay the following:“Yes, I have now seen how hybrid event technology, social media and new collaborative formats can transform events. Yes, we could try ________ at our event. We may not be able to do it without help – but yes we could do it. AND, we could do it 10 times better than those crazy people at Event Camp Twin Cities.”This event is going to be a lot of fun!
Level 2
Learning
Know where to keep upto date on dev't in the hybrid technology & social media
IDentify the multilingual problems of involving hybrid audiences
AttitudeUsing Lickert scale questionsTest attitude statements
Internal
CLient
Strongly agree - Storngly dissagree
Relationship Learning
Skills
Deliver a Pecha Kucha format and see the impact on remote audiences
Know how to contribute effectively to the backchannel using microblogging
To seek the support of profession-als if I can't do it myself
Organiserscheck
Minneapolis
Interactive Meetings Technology LLCSam SmithConsultancy
Speaking
AppevisionRay HansenApp devt
Basel
©TNOCRuud Janssen
Online collaboration tools for communi-ties in a collective Collective
People LearningMake a list of people you met who within the next 6 months can help you setup an online community
Consultancy
Speaking
PublishingCase study
ROI Case study
Marketing of Services provided by TNOC
Test the waters for Event Camp Europe eventIncrease Titter following
Increase reach
Kahlweiss Video ProductionsMagnus KahlweissVideo productions
Wordcamp experience
When they need help they come to the specialist consultant
What can I he;p them with
How to contact me
I now know the I know that Inxpo is the leading supplier in putting
Dallas
Press
MIC magazine
Headquarters Magazine
One+
©TNOC | BlogArticleDrive traffic
Increase Titter following
VenueCafe Tacuba BaselGet to know Cafe Tacuba as a new venue in Basel
ActionCome back to Cafe Tacuba for meals
Use Cafe Tacuba for small meetings
SpeakersPechu Kucha
Elling
Level 2 learning
There is awareness of the existence of the ROI methodology
Recognized in multiple industries
Easily learn how to do it
Read the book
Attend the certification course
Level 3Share the message of ROI + know where to learn more on ROI methodology
Ruud
Level 2
Awareness of workings of collective
Opportuntiy to use
Experience how to present hybrid & Pecha Kucha
10 changes in working in the next 10 years
Level 3Become active participants and become part of the TNOC collective
Become contributor to #eventprofs community
#Eventprofs community
Ideas
digital footprint What to measure
(Who) Would you recommend attending Event Camp to your industry colleagues in the next 2 weeks
Measuring ROI for Events & MeetingsDr Elling Hamso
Managing Partner European Event ROI institute
Jack PhilipsPresident ROi Insititute
Elevator pitch
The ROI Methodology enables meeting owners to create events that deliver maximum value and prove the value created
Event is developed on the basis on evaluation objectives
Starting with the impact the event will have on the bottom line
4 levels of meetingMeasure success from good learning, environment
Problem solvedEnables you to ensure you get the maximum value and impact and prove that you did achieve it