Evaluation

Why to evaluate your event 

To improve event management

• Tracking and measuring progress improves any form of management – if you cannot measure it, you cannot manage it.

To build an organisational memory and enable a learning process

• The evaluation process and the data and information stemming therefrom contribute to building a “project’s/event’s memory”, which enables the transfer of knowledge from one generation of the event to another, as well as from one organiser to others.
• It can enhance cost effectiveness – avoiding the same mistakes, improving processes, building on good practices.

To get a holistic view of the events’ impact

• Event organisers and hosts frequently underestimate the true impact of their events, which often goes well beyond the direct economic impact and consist of intangible, for example, societal benefits.

To demonstrate the return on investment

• A holistic evaluation of the event provides organisers with information, data and assessment which is of great interest to the main stakeholders, and will help the organisers to promote the value of the Olympic Festival to potential future host cities/regions and sponsors/partners (good “lobbying” material).

To enhance the positive image of the Olympic Festival and link to Olympic Values - communications

• The evaluation of the Olympic Festival will yield much communications-worthy information which could and should be used in the promotion of the OF and its future editions.
• The evaluation should bring results which will underline the embedment of the Olympic Festival in the Olympic movement and the implementation of Olympic values.
• For more information, see the Sustainability chapter.

Strategy

Plan, prepare and be strategic in the evaluation of your Olympic Festival

• Define what you want to achieve (KPIs) at the beginning of the planning process and measure whether you have achieved it at the end (compare objectives with real results).
• Evaluation has to be considered an integral part of the event management cycle – include it in your planning at the very beginning.  
• Dedicate appropriate resources (financial and human) to the evaluation process. Decide which tools and methods you will use and reflect it in the budget (e.g. hiring an external evaluator).

Implement a step-by-step (gradual) approach

• The scope of the evaluation exercise should reflect the size, budget and duration of the Olympic Festival as well as the experience of its organiser.
• Prioritise. Select the most important aspects/sectors at first and gradually build on those in the future.
• However, it is strongly suggested that the following areas be included in any evaluation: economic impact, stakeholder involvement and satisfaction, visitors’ satisfaction, internal organisational performance, and, if possible, environmental impact.  

Do not forget about qualitative evaluation

• Narrative description is as important as quantitative data, especially when assessing the social impact or legacy of the Olympic Festival.
• Often, contribution to social development cannot be expressed in bare numbers.

Consider involving respected and experienced external evaluators in the process

• They will add more (external) credibility to the results, since external evaluators are considered to be impartial professionals with no vested interest in the outcome of the research.
• They have the necessary expertise you may be lacking in your organisations – e.g. the economic and sociological analysis, or the evaluation of social impact.

Consider well what you communicate, how and to whom

General tools

• Planning and implementing an (environmental) evaluation - MEERA
• A useful guide which will take you through the main steps and phases of event evaluation. Even though the main focus is on the environment, it can be applicable to the whole evaluation process.

Visitors’ survey

• It should be conducted during the event, preferably during several days, including week-days and weekends.
  

  Visitor´s survey
• Participation of a sponsor (competition for goodies from the sponsor) can substantially increase the response rate.
• The survey can be done as a combination of a short questionnaire at the venue followed by a more detailed one via email (e-mail addresses collected from the visitors – do not forget about the obligations under the EU GDPR).
• It is advisable to use electronic means such as Google Forms or Survey Monkey.                    
• Example of survey questions in the pdf file.                                                                                                                                      

Stakeholders’ interview

• It is recommended to do on-site interviews as well as post-event ones in order to discuss with stakeholders their overall impression.
  

  Interviews - questions
• Part of the interviews can be conducted via email, phone or an online questionnaire.
• The interviews can also be conducted as a debriefing with a group of stakeholders.
• Who should be interviewed? – Representatives of the host city/region, sponsors, organising committee members, volunteers, representatives of sport organisations and possibly other stakeholders.
• Example of interviews questions per group of stakeholders – see the pdf file.                                                                                      

Event observation

• Preferably conducted by somebody who is not involved in the organisation of the event. 
  

  Event observation form
• Example of the event observation form - see the pdf file.                                                                                                                   

Media monitoring

• General media and social media monitoring – For more information, see the Communications chapter.

Working with external evaluators 

• External evaluators (universities, consultancies, economists etc.) can play a very important role in shaping and managing the evaluation, in the design of instruments or data analyses, as well as in interpreting the results of the evaluation. It is therefore critical to reflect on how to best identify, choose, and work with external evaluators.
• A good overview is provided on the MEERA website.

Basic steps

• Decide which elements of your event’s evaluation you would want to entrust to external evaluators, and describe the expected responsibilities.
• Find the right evaluator for your purposes
  • Consider organisations and individuals with whom you have already worked on other projects.
  • Consult databases and specialised (public) organisations and NGOs for areas such as the environment – fore more information, see the Sustainability chapter.
  • Consider working with universities which have experience in evaluating sport-related events and projects, including from a sociological point of view (visitors’ surveys etc.).
  • It may prove valuable to involve graduate students of communications, statistics, sociology or psychology, especially with regard to the observation and visitors’ surveys.
  • Consult the following list for tips on which questions you should ask the potential candidates.

Working with the evaluator

FMB Handbook

• Tips on structuring the work relationship into several basic stages (1. Initial meeting, 2. Contract, 3. Project management, 4. Providing feedback, 5. Payment 6., Deliverables, 7. Evaluating or debriefing the evaluation project) are detailed in this handbook (pgs. 90-95).