Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
WHAT IS EVALUATION?
Evaluation is an activity that seeks to understand and measure the extent to which an event has succeeded in achieving its purpose.
Evaluation is the process of measuring the success of an event against its objectives.
There can be two attitudes with which evaluation can be put in its proper perspective. One is a critical examination digging out what went wrong. And the other is a more constructive focus to make recommendations about how an event might be improved to achieve its aims more effectively.
EVENT EVALUATION
THE BASIC EVENT EVALUATION PROCESS
In events, the basic evaluation process involves three steps:
Establishing tangible objectives Measuring the performance before, during
and after the event Correcting deviations from plans
MEASURING PERFORMANCE
Although perfect measurement is not always practicable, the measurement of
performance against the objectives should ideally be done on a forward looking basis so that deviations may be detected in advance
of their occurrence and avoided by appropriate actions.
MEASURING PERFORMANCE
Following are the methods of Measuring Performance:
Concept Research Formative Evaluation Objective Evaluation Summative Evaluation
CONCEPT RESEARCH At the conceptualization stage, if a concept team
does not have a sound basis upon which to make a decision ‘between various options, a commissioning of audience research to help in defining the strategic approach to be adopted in the event is appropriate.
It essentially involves presenting the various options to a representative sample of the target audience in a story form and inviting their reactions.
This provides enough material for understanding the pros and cons of the various available alternatives.
The downside to this method is that it is speculative in nature since it deals with plans that nobody has as yet tried to implement. This method is called concept research.
FORMATIVE EVALUATION
Evaluation at this stage focuses on things that are actually happening.
After the conceptualization team makes an attempt to customize and implement an agreed strategy, steps can be taken to evaluate the success with which customization is proceeding.
These evaluations are aimed at shaping the form of the final event. Mock-up displays and presentations of the event are used to carry out research to check whether they are achieving the desired reactions from the audience.
FORMATIVE EVALUATION These evaluations are conducted among small
sample representative of the target audience in an open-ended and qualitative fashion since the main emphasis is on discovering how the concept might be better represented.
The outcome of these formative evaluations lead to a discussion among the team in which proposals for rectifying any weak points in the communications can be put forward.
A point, which should be safeguarded against whilst using this technique, is to interpret consumer reactions with considerable sensitivity to stimulate the creative process further and also to ensure that good ideas are not killed simply because they were not properly presented in mock-up form.
OBJECTIVE EVALUATION
This is the stage when approval from the client is sought before starting the execution related activities of an event.
The evaluation team has to provide the objective evidence that has been collected which justifies the proposed concept solutions.
The team also provides reassurance on how and why the particular event will work among its intended audience.
Since taking the client into confidence requires certain amount of objectivity and professionalism, this technique is called objective evaluation.
SUMMATIVE EVALUATION After the event has started, the evaluation team
should be concerned with measuring the impact of the event upon its audience.
Among other things, they should establish the extent to which the objectives or aims of the event have been met and whether the event can be improved in any way and if so, how This will not apply for short term events though.
A major purpose of evaluating an event after it has opened to the public is that it provides the team with the opportunity of learning from their mistakes.
The team should assimilate the information thus collected so that they can avoid making similar mistakes in the future.
CRITICAL EVALUATION POINTS
Events can be evaluated based on the critical success factors listed below;
From Event organizer’s point of view From the Client’s point of view
CRITICAL EVALUATION POINTS FROM EVENT ORGANIZER’S POINT OF VIEW There are multiple criteria for evaluating the success
of an event from the event organizer’s point of view. Ensuring perfect reach and interaction for the client
by networking on-time & at lowest cost. The client event-target audience fit should match the
clients’ brand/product/company image and personality perfectly, keeping the target audience as the focal point.
Ensuring the profitability of an event such that there is maximum profitability with minimum mark ups is another critical evaluation point.
Resources are also a major constraint for event organizers, the resource management efficiency i.e., resources committed and span of time for which it stays committed – financial, human, equipment and infrastructure should be a minimum.
CRITICAL EVALUATION POINTS FROM EVENT ORGANIZER’S POINT OF VIEW
Logistics and efficiency of event execution for ensuring smooth proceedings without unnecessary delays and damages is another critical success factor.
Creating avenues for lead generation & its proper management during the event is a critical factor.
Another important critical success factor is the degree of localization or customization accommodated in the concept to suit the demographic and other variables of various places where the event is to be carried out.
CRITICAL EVALUATION POINTS FROM CLIENTS’ POINT OF VIEW Immediate and long-term benefits that accrue from
an event are important when evaluating an event from the clients’ point of view.
A cost-benefit analysis concerning the effectiveness of reach and interaction is a must as a pre-event activity.
Post-event stock taking activity should be done to confirm whether the event has occurred as per plans. This analysis should consider the actual cost of the event that includes the non-budgeted expenditure as well as the actual benefits that accrued to the client from the event.
The accrual of benefits can be judged by measuring the tangible parts of the objectives that have been achieved.
MEASURING REACH
Reach is of two types
External event reach Actual event reach
EXTERNAL EVENT REACH Since events require massive external publicity-press,
radio, television and other media are needed to ensure that the event is noticed and the benefit of reach is provided to the client.
Measurement of external reach is possible by using the circulation figures of newspapers and promotions on television and the radio. The TRP ratings that rate the popularity of programmes on air and around which the promotion is slotted, is a very tangible though approximate method for measuring the external reach of a promotion campaign on television.
Releasing ads and promos one month in advance should be considered more as an awareness exercise for propagating the event concept, time, date and venue to the audience. The entry criteria – free, invited or ticketed show should be clearly mentioned here.
ACTUAL EVENT REACH
The measurement of the actual reach of an event is relatively simple.
The capacity of the venue is a figure that provides the upper limit for the actual reach.
Ticket sales or numbers of invitees are also direct measurement tools.
Registration of participants and requests for filling in questionnaires are also common methods of measuring the actual reach of an event.
CRITERIA FOR EVALUATION
Hard and Soft Criteria
Hard criteria - concerns the outputs, tend to be tangible and quantitative
Soft criteria - concerns the process and tend
to be intangible and qualitative.
HARD CRITERIA FOR EVALUATION OF AN EVENT
Performance specifications Resource constraints Sales data Visitor numbers Financial reports and accounts Deadlines Cost requirements Visitor flow rates Visitor profiles Specific quality standards
SOFT CRITERIA FOR EVALUATION OF AN EVENT
A positive image Total quality Staff commitment Visitor perceptions Management notes and commentary A co operative attitude Ethical conduct Social benefits Exit surveys Staff and volunteer feedback
TOOLS AND TECHNIQUES OF EVALUATION.
Observation Audio tape recording Interviews Focus groups Video tape recording Suggestion box Media monitoring – newspaper clippings etc Photography Questionnaires Personal log Staff meeting
METHODS OF EVALUATION
Surveys Staff Debriefings Financial records
SURVEYS
Surveys can be conducted before, during and after an event
This can be done through completion of feedback forms or through personal interviews
Also, a small focus group of participants can provide valuable information through group discussion
To obtain a more reliable report, the survey would need to be designed and analyzed by a market research company
FOLLOWING ARE EXAMPLES OF QUESTIONS THAT MAY BE INCLUDED IN A CUSTOMER SURVEY:
Q1) Did you enjoy the event? If no, then please state the reason.
Q2) what do you like most in the event?
Q3) what do you like least in the event?
Q4) what are the problems you faced during the event?
Q5) what could have been done to make this event better?
Q6) How do you rate the various services provided by us (please check one of the option):
Hospitality: Excellent, good, average, poor
Catering: Excellent, good, average, poor
Transportation: Excellent, good, average, poor
Management staff behaviour: Excellent, good, average, poor
Management staff services: Excellent, good, average, poor
Q7) would you like to participate in our next event?
STAFF DEBRIEFINGS
Meetings of event staff and stakeholders can generate valuable information for the evaluation report
Some of the questions addressed in this type of meeting include:
What went well, and why? What went badly, and why? How could operations be improved? Were there any significant risk factors that we did
not anticipate? Was there a pattern to any of the incidents
reported?
STAFF DEBRIEFINGS
Are there any outstanding legal issues, such as injuries or accidents?
Are there any implications for staff recruitment and training?
How would you describe the organization and management of the event – in the planning and the operational phase?
What can we learn from this event?
FINANCIAL RECORDS
Audited financial records, together with a number of planning and other documents, are an essential component of post event analysis and reporting.These include the following;
Audited financial statements Budgets Revenue, banking and account details Point of sale reconciliation Payroll records
Incident reports Minutes of meetings Contracts with other agencies and
organizations Asset register Promotional materials Event report
MONITORING AND CONTROL SYSTEMS
As discussed earlier Evaluation is the process of measuring the success of an event against its objectives
Now in case of a marathon, evaluation will involve an analysis of the questions on the registration form and feedback after the event, whereas, we need to have Control systems in place to ensure that all participants were registered
If a significant number of participants joined the Marathon without registering, this will indicate lack of control measures and will have an impact on the evaluation findings.
MONITORING AND CONTROL SYSTEMS
The challenge for an Event manager is to delegate and monitor effectively and not to micromanage.
During the peak time of an event, unexpected situations and incidents will require the Event Manager’s time, hence all routine procedures and control systems need to be in place before the event.
PREVENTIVE CONTROLS AND FEEDBACK CONTROLS
There are 2 types of controls;
Preventive Feedback
PREVENTIVE CONTROLS
A preventive control is established early in the planning process.
For E.g. Checking the quality of incoming food for a banquet, i.e. monitoring food temperatures to avoid food poisoning.
A checklist for setting up sporting equipment before an international gymnastics event, this is needed in order to ensure that setup would meet international specifications, if measurements were inaccurate, injury could be caused to an athlete or an athlete could be disqualified.
FEEDBACK CONTROLS
Feedback controls are put in place to assist with decision during an event.
OPERATIONAL MONITORING AND CONTROL
There are a number of issues in relation to operational procedures that need to be addressed before the event begins
These include;
Implementation of priority or high – risk procedures
Delegation of Decision Making Flexibility in Operational Procedures Assuring Customer Satisfaction Controlling Finances
MONITORING AND CONTROL SYSTEMS The following are a few suggestions for monitoring
and controlling event operations:
Check everything, over and over again Write down everything, including promises made by
your contractors and requests made by your client Develop checklists for everything possible Check the venue before you move in, note any
existing damage Never leave the venue until the last staff member has
finished Check the venue before leaving – some things may
have been accidentally left on (gas) or left behind (including people)
MONITORING AND CONTROL SYSTEMS
Pay attention to detail at every stage Schedule carefully, since the audience has
little patience with long – winded speeches Maintain a contingency fund for unexpected
expenses Involve the sponsor at every stage Get approvals for use of logos before printing Don’t take safety knowledge for granted Train staff to be observant
Related Documents
