FINAL EVALUATION REPORT May 2003 · EVALUATION OF THE SME ACCIDENT PREVENTION FUNDING SCHEME 2001-2002 2 2. The Scope The Evaluation of the Funding Scheme was launched in November

The European Agency for Safety and Health at Work

EVALUATION OF THE SME ACCIDENT PREVENTION FUNDING SCHEME 2001-2002 BIO 8958

EVALUATION OF THE SMALL AND MEDIUM-SIZED ENTERPRISES (SMEs) ACCIDENT PREVENTION

FUNDING SCHEME 2001-2002

EUROPEAN AGENCY FOR SAFETY AND HEALTH AT WORK

FINAL EVALUATION REPORT

IDOM Ingeniería y Consultoría S.A.



Foreword IDOM INGENIERÍA Y CONSULTORÍA S.A., Bilbao, Spain, carried out evaluation. The company gratefully thanks all those people who answered its questionnaires and interviews for their useful feedback and kind collaboration. The Evaluation Team Bilbao, 2003



TABLE OF CONTENTS EXECUTIVE SUMMARY _________________________________________________________ 1 0- THE STRUCTURE OF THE EVALUATION REPORT ________________________________ 15 1- OBJECTIVES AND METHODOLOGY____________________________________________ 16

1.1- OBJECTIVES OF THE EVALUATION ________________________________ 16

1.2- METHODOLOGY FOR THE EVALUATION OF THE FUNDING SCHEME 17 1.2.1- METHODOLOGY STRUCTURE ____________________________________ 18 1.2.2- EVALUATION PHASES ____________________________________________ 19 1.2.3- QUESTIONNAIRES AND SURVEY SAMPLES ________________________ 21 1.2.4- IMPLEMENTATION OF THE SURVEY ______________________________ 21 1.2.5- IMPLEMENTATION OF INTERVIEWS ______________________________ 22

2- THE SME 2001-2002 ACCIDENT PREVENTION FUNDING SCHEME: PROGRAMMING, MANAGEMENT AND MONITORING ISSUES________________________________________ 25

2.1.DESCRIPTION OF THE FUNDING SCHEME ___________________________ 25 2.1.1- AIMS AND SCOPE OF THE FUNDING SCHEME______________________ 25 2.1.2- PHASES OF THE SME ACCIDENT PREVENTION FUNDING SCHEME _ 26

2.2.PROGRAMMING OF THE FUNDING SCHEME_________________________ 27 2.2.1- CONTENTS OF THE AGENCY’S CALL FOR PROJECT PROPOSALS___ 27 2.2.2- THE CALL, APPLICATION AND TECHNICAL SPECIFICATION FORMS33 2.2.3- PROCESSING APPLICATIONS BY THE AGENCY ____________________ 35 2.2.4- SELECTION PROCEDURE AND ADVISORY BODIES _________________ 38

2.3- MANAGEMENT AND MONITORING OF THE FUNDING SCHEME ______ 43 2.3.1- AIM OF THE CHAPTER____________________________________________ 43 2.3.2- AGENCY’S PERFORMANCE WHEN HANDLING AND MONITORING THE PROJECTS_____________________________________________________________ 43 2.3.3- EVALUATION OF THE ACHIEVEMENTS ___________________________ 45

2.4- PROMOTION OF THE FUNDING SCHEME BY THE AGENCY __________ 46 2.4.1- PROMOTION OF THE AIMS AND OBJECTIVES OF THE FUNDING SCHEME AND THE CALL _______________________________________________ 46 2.4.2- INFORMATION CONCERNING CO-FUNDED PROJECTS _____________ 47 2.4.3- DISTRIBUTION OF PROJECT RESULTS ____________________________ 47

3- ANALYSIS OF THE COHERENCE OF THE SME 2001-2002 ACCIDENT PREVENTION FUNDING SCHEME ____________________________________________________________ 48

3.1. CONTEXT FOR ANALYSING COHERENCE ___________________________ 48

3.2. INTERNAL COHERENCE OF THE FUNDING SCHEME ________________ 48

3.3. COHERENCE OF THE FUNDING SCHEME REGARDING OTHER COMMUNITY POLICIES _______________________________________________ 52

4- RESULTS AND IMPACTS OF THE FUNDING SCHEME _____________________________ 25

4.1- SELECTED PROJECTS: LEVEL OF ORGANISATION AND CATEGORIES 60

4.2- THE OUTCOME OF THE PROJECTS _________________________________ 61



4.2.1- RELEVANCE OF THE FUNDING SCHEME IN RELATION TO OSH NEEDS AND PRIORITIES_______________________________________________________ 62 4.2.2- PERFORMANCE OF PROJECT HOLDERS. EFFECTIVENESS AND EFFICIENCY OF THE PROJECTS________________________________________ 64 4.2.3- OUTCOME OF THE PROJECTS IN RELATION TO THE CALL PRIORITIES ___________________________________________________________ 69

4.3 PROJECT SUSTAINABILITY _________________________________________ 74

4.4- IMPACT OF THE FUNDING SCHEME ________________________________ 75 4.4.1- IMPACT OF THE FUNDING SCHEME IN HEIGHTENING AWARENESS OF OCCUPATIONAL ACCIDENTS RISKS AT EU LEVEL___________________ 76 4.4.2-TRANSFERABILITY OF PROJECTS _________________________________ 77 4.4.3- EW 2001 AND THE TOPIC ON ACCIDENT PREVENTION _____________ 78

4.5- OVERALL ASSESSMENT ___________________________________________ 60 5- RECOMMENDATIONS FOR FUTURE SME FUNDING SCHEMES _____________________ 85

5.1- PRIORITY RECOMMENDATIONS FOR THE ENTIRE FUNDING SCHEME_______________________________________________________________________ 86

5.2- SPECIFIC RECOMMENDATIONS FOR THE FUNDING SCHEME________ 90

5.5- RECOMMENDATIONS FOR FUTURE EVALUATIONS _________________ 95 ANNEX ANNEX1: METHODOLOGY PROCESS EMPLOYED FOR THE COLLECTION AND ANALYSIS OF INFORMATION_____________________________________________________________ 97 ANNEX 2: QUESTIONNAIRE FOR PROJECT HOLDERS ______________________________ 98 ANNEX 3: QUESTIONNAIRE FOR NATIONAL FOCAL POINTS________________________ 103 ANNEX 4: DATA FROM THE SURVEY (PROJECT HOLDERS) ________________________ 106 ANNEX 5: DATA FROM THE SURVEY (NATIONAL FOCAL POINTS) ___________________ 114


EVALUATION OF THE SME ACCIDENT PREVENTION FUNDING SCHEME 2001-2002 1

Executive Summary 1. Introduction The SME Accident Prevention Funding Scheme 2001-2002 is a Grant Scheme focusing on reducing accidents in Small and Medium-sized Enterprises (SMEs) throughout the European Union. It was launched by the European Agency for Safety and Health at Work (hereafter referred to as the “Agency”) in April 2001 and has co-funded a total of 51 projects at national and transnational levels. The European Parliament allocated a 4.5 Million € budget to the Agency. The Agency is a tripartite European Union organisation uniting representatives from three key decision-making groups in each of the European Union’s Member States – namely, governments, employers and workers’ organisations. The different bodies comprising the Agency network are:

The Administrative Board: that sets the Agency’s goals and strategy, The Bureau: that oversees the Agency’s operational performance, The Agency: the Director of the Agency is the official representative of the Agency

and is responsible for the day-to-day running of the same, National Focal Points: these are the Agency’s principal safety and health

information network in each Member State. National Focal Points are nominated by each government as the Agency’s official representative in that country and are normally the competent national authority for Safety and Health at Work. The role of National Focal Points is to provide information and feedback to the Agency concerning initiatives and products and they are consulted about all information activities related to the national level.

The general objective of this appraisal is to establish whether the objectives of the SME Accident Prevention Funding Scheme have been met in terms of impact, effectiveness, efficiency and sustainability, in addition to making recommendations and assisting the Agency in the formulation of future Funding Schemes. In order to attain these objectives, the Evaluation Team has developed and implemented an evaluation method based, on one hand, on the analysis of secondary sources such as project documentation (Application Forms and Final Activity Reports) and, on the other, analysis of information obtained from a survey and interviews conducted for this purpose. This, the First Evaluation of the Funding Scheme, covers the management and accomplishment of the objectives of the Scheme, as well as the performance of the Agency and the project holders during the Scheme’s different phases, in addition to including recommendations for improving future Schemes.



2. The Scope The Evaluation of the Funding Scheme was launched in November 2002 and implemented over a period of 4 months. Evaluation concluded with this Final Evaluation Report. In accordance with the specifications contemplated in the Restricted Call for Tender (OSHA/TF/2002/T2) regarding External Evaluation of the Funding Scheme, evaluation has focused upon the following items:

1. Quality and suitability of the Call for Project Proposals. 2. Agency performance regarding dissemination of the Funding Scheme. 3. The processing of applications by the Agency in order to evaluate the extent to

which services provided by the Agency ensured quality at the outset and supported implementation.

4. The selection procedure regarding evaluation of the composition and effectiveness of the bodies evaluating the applications and the quality of selected applications.

5. Evaluation of the implementation of selected projects. 6. Results and impact of the Scheme in terms of outcome, sustainability and

transferability. 3. Methodology The methodology used for this evaluation exercise was a combination of desk- (compilation and analysis of information) and fieldwork (questionnaires and interviews). Two main sources of information were used for the evaluation:

Information provided by the Agency; namely Application Forms, Final Activity reports of all co-funded projects, the Funding Scheme reference documents (Call for Project Proposals, Application Form, Technical Specifications) and other related documents.

Information generated during the evaluation. This qualitative information was

gathered using a survey conducted among 51 project holders and 15 National Focal Points, all of whom were sent questionnaires, in addition to interviews held with project holders, National Focal Points, the Agency and other related parties.

Other documents and publications such as newsletters, reports, etc.

The information was gathered and analysed according to the following stages:

1. Information was gathered from the Agency. 2. Identification and preliminary analysis of essential aspects of the Funding Scheme. 3. Survey: Elaboration of Questionnaires for Project Holders and National Focal

Points. 4. Personal interviews.



5. Information generated during the evaluation process was analysed. 6. The Final Evaluation Report was drafted, including issues identified in the Terms of

Reference. It should be mentioned that there was a high degree of response to questionnaires (100% National Focal Points and 90% project holders) and collaboration among the parties interviewed. This fact provided the Evaluation Team with important information that further improved the results of the evaluation. 4. Organisation of the Evaluation Report The Evaluation Report has been set out in Chapters and drawn up in accordance with the following issues: Chapter 1 “Objectives of the Funding Scheme and Evaluation Methodology”. Chapter 2 “Programming, Management and Monitoring of the Funding Scheme”. Chapter 3 “Coherence of the Funding Scheme”. Chapter 4 “Results and Impact of the Funding Scheme”. Chapter 5: “Recommendations for future Funding Schemes”.

This Executive Summary has been structured in accordance with the Evaluation Report and each of its Chapters includes both the main conclusions and recommendations elaborated by the Evaluation Team. 5. Chapter 1: Objectives of the Funding Scheme and Evaluation Methodology In 2001, the European Agency for Safety and Health at Work launched the Accident Prevention Funding Scheme for Small and Medium-sized Enterprises (SMEs). The Scheme supported projects aimed at reducing work-related accident risks throughout European SMEs. The overall objective of the Funding Scheme was to “identify, communicate and support activities and projects with effective added value that could motivate, develop, support and sustain the organisation of OSH/OSH management in SMEs”. It was imperative to achieve this goal by providing effective Good Practices that would meet the relevant legislative requirements established by European Directives, national legislation and guidelines. Three project categories, focusing on priority hazards and high-risk sectors at both national and transnational levels, were financed:

♦ Training ♦ Information and Communication ♦ Provision of Good Practices

The Evaluation Process was divided into four phases:



Phase 1- Structuring. The objective of this phase was to determine the most appropriate evaluation method and gather relevant information. It was carried out from the 21st November to the 5th December 2002 and concluded with a Progress Report being submitted on the 5th December.

Phase 2- Data collection and preliminary analysis. This phase focused on data

collection related to the identification of those target groups that would respond to the questionnaires, the selection of project holders for interviews and the elaboration of questionnaires and their despatch to all project holders and National Focal Points. The timeframe for this phase was from the 5th December to the 26th December 2002. This phase concluded with the Inception Report being submitted on the 19th December.

Phase 3- Implementation of fieldwork. Interviews, reception of questionnaires

and complementary activities were carried out. This phase was implemented from the 26th December 2002 to the 30th January 2003 and concluded with the Second Progress Report being submitted on the 30th January. Personal interviews during Phase 3 were held with:

1. Thirteen project holders. 2. The Agency (Director, Task Force Campaigns and Programmes, Head of the

Working Environment Unit and Head of the Information and Communication Unit).

3. The Agency’s Administrative Board Members. 4. National Focal Points in Greece, Denmark, Germany and Belgium. 5. The European Jury. 6. Other interested parties (European Commission, European Parliament and

Social Partners at EU level).

Phase 4- In-depth evaluation and judgement. This phase was implemented from February to the end of March 2003. It focused on an in-depth analysis of all information gathered during Phases 2 and 3 and served to produce this Final Evaluation Report.

6. Chapter 2: Programming, Management and Monitoring of the Funding Scheme This chapter describes the Funding Scheme and assesses its implementation in accordance with three basic, inter-related steps: Programming of the Funding Scheme: this is the planning of the Agency’s Call for

Project Proposals and the Selection Procedure. Management and Monitoring of the Funding Scheme: the Agency’s performance while

handling and monitoring projects, in addition to evaluating the Scheme’s achievements.

Promotion of the Funding Scheme: assessment of the promotion of the Scheme at different levels.



6.1. Agency’s Call for Project Proposals The Call for Project Proposals was based on a decision of the European Parliament. It was published in the Official Journal of the European Communities on the 21st April 2001 (2001/S 78-053024). The Call for Project Proposals, Application Form, Technical Specifications and Checklist were drafted in eleven languages in order to allow project holders to submit their applications. The majority (88%) of project holders stated that these documents were drawn up in a clear and comprehensible manner and that they contained sufficient information for potential applicants. In spite of this, some room for improvement was identified. Avoiding repetitive questions would shorten the Application Form. In addition to which, relevant information concerning Budget or Administrative Requirements could be more precise. These modifications should ease the task of Project Holders when filling out their applications. Evaluation also identified the need to include reference indicators in the Application Form in order to allow projects to be monitored and evaluated. Potential applicants are required to quantify these indicators upon submitting their Application Forms. It would be advisable to provide potential applicants with an indication of how their applications are to be assessed. Introducing scores to measure the quality of the technical and financial proposal of the project holder could do this. The Agency promoted the Scheme in an efficient manner. The Call for Project Proposals, Application Form and Technical Specifications were published on the Agency’s website, together with the OJEC publication. Promotional activities were held by the Agency within its own network and presented to other interested parties. Most potential applicants knew about the Funding Scheme through the Agency’s website or network. However, in order to increase Scheme visibility, it would be advisable to reinforce promotional campaigns. The vast majority (96%) of project holders considered that the deadline for submitting Applications was appropriate. From the date of publication of the Call, national applicants had 10 weeks in which to submit their applications. Transnational applicants had 12 weeks. 6.2. The Agency’s Performance The Agency answered enquires from potential applicants from the date the Call was published till the deadline for submission of applications. Two-thirds of the project holders expressed their opinion that the Agency provided project applicants with “excellent service”. In addition to which, 41% of all project holders assessed support provided by the Agency as “excellent” and a further 41% assessed it as being “good”.



The Agency registered and filed all applications. In the year 2001, the Agency received 456 applications, most of which (90%) had been properly completed according to the specifications of the Call and received by the Agency within the specified deadline. Applications checked against these eligibility criteria were assessed in accordance with selection criteria regarding specific priorities. The final selection of co-funded projects was carried out by the Agency in October 2002. The selection procedure was impartial, transparent and fair. The Agency established a schedule and appointed groups of experts to assess eligible applications. As a result of the interviews carried out, it can be concluded that the timeframe for receiving, processing and selecting applications was extremely short. National and transnational project selection groups were made up of specialists in the field of Safety and Health at Work, in addition to a number of social partners. 6.3. Selection procedure of national and transnational projects Evaluation procedure took approximately three months and particular attention was paid regarding the compliance of applications with selection criteria. Interviews held led to the conclusion that having more time for the selection of applications would have produced a more positive impact. National Focal Points assessed national applications in co-operation with social partners and a “European Jury” comprising government representatives, social partners, a delegate from the European Commission and two independent experts evaluated transnational applications. Of 410 eligible applications, only 51 passed the selection process. This illustrates that only 12% of all applications submitted received co-funding from the Agency. In view of the limited amount of human resources allocated to the European Agency Task Force Campaigns and Programmes (responsible for management the Funding Scheme) and many National Focal Points, we suggest that more time and resources be allocated to the evaluation process. Should the Scheme continue to function on an annual basis, then it might be necessary to reduce the number of applications to be assessed. This could be achieved by introducing stricter eligibility criteria. Priority was given to the transparency and fairness of the evaluation process at both national and transnational levels. In order to attain these conditions, the Agency provided national and transnational evaluation teams with common rules and precise guidelines for assessing applications in order to avoid “judge and jury problems” and “conflicts of interest”. Posterior analysis and interviews carried out reveal that the selection procedure was impartial, transparent and just. Nevertheless, an alternative option that would avoid any problem in the application assessment procedure would be to subcontract this work to external specialists. This external evaluation would be conducted under the surveillance and monitoring of the Agency.



6.4. Selected Projects The 2001-2002 Funding Scheme co-funded 35 national and 16 transnational applications with a total budget of 4.5 Million EUR spent on an annual basis. The grant received by each project varied between €25.000 and €190.000. National projects had smaller budgets than transnational ones. This allocation enabled 51 projects to be co-funded. Nevertheless, this budget is very small in comparison with other allocations of Community funds and the impact is not, therefore, easily detected in the SME sector. Evaluation reflected the need to increase the SME Accident Prevention Funding Scheme visibility. This could be achieved as follows: By financing a lower number of larger-size projects (broad partnerships) in order to

ensure that the message can reach and be effectively used by others. By increasing project budgets and promoting the creation of integrated projects

(projects that include more than one category: Training, Information and Communication and Good Practices).

By making the Funding Scheme an "Integrated Programme". This would permit the creation of a specific Scheme for SMEs to be developed regarding issues related to accident prevention in the workplace and would prevent the Funding Scheme from being isolated to a greater, or lesser, degree from other activities occurring at Community level.

Selected Project Holders received written confirmation from the Agency. However, this letter did not include any indication or evaluation regarding those strong or weak points of their applications. Project holders interviewed expressed their interest in receiving the Agency’s feedback in this particular respect in order to permit them to improve the quality of their applications in future Calls for Project Proposals. Depending on the amount of the conceded grant, Project Holders were contractually required to present Interim and Final Activity Reports. This allowed the Agency to better monitor projects. The Survey and interviews carried out showed that project holders had no problem in understanding what the Interim and Final Reports should contain. Time was the main impediment regarding project implementation under the Funding Scheme. Approximately half of the project holders stated that the deadline for submitting the Final Activity report was unsuitable. Additionally, project holders emphasised the difficulties encountered in submitting their financial reports. Since the Agency works on an annual basis, it might be advisable to follow the following steps in order to gain time:

Provide project holders with clear information concerning the need to comply with these tight deadlines.

Ensure that all contracts are signed rapidly. Ensure intermediate monitoring of all projects, via telephone or Interim Reports.

This assistance from the Agency could help project holders to resolve their problems whilst the project was still in progress.

Once the project has been completed, allow for one or two additional weeks for the submission of financial reports.



The three project categories presented were appropriate for contributing to a reduction in accidents in SMEs. Likewise, with regard to co-funded projects, it was observed that these categories corresponded more closely to the needs of intermediate bodies (trade unions, associations, business organisations, etc.) than to those of SMEs, who might prefer direct funding for “fixed assets” and safety and health plans in the workplace. 6.5. Promotional Activities of the Funding Scheme Agency Funding Scheme promotional activities were carried out at the following levels:

Dissemination of the objectives of both the Funding Scheme and the Call for Project Proposals. A public launching of the Scheme accompanied the publication of the Call in the OJEC, the Agency’s website dedicated a specific section to the Funding Scheme, the Agency network participated in dissemination activities, etc. At this stage, the promotional campaign of the Scheme was sufficient, though it would be advisable to extend publicity campaigns in collaboration with National Focal Points in order to increase diffusion of the Call.

Dissemination of selected projects: the Agency prepared a report containing a brief summary of selected projects and also published this information on its website. Those project holders consulted considered this to be satisfactory.

Dissemination of results of co-funded projects. The 2001-2002 Funding Scheme is still at a promotional stage. At the time this Evaluation Report is still being drawn up, a document containing the results of the Scheme is being confected by the Agency. Though it is not possible for us to give an assessment of promotional activities at this stage, it is important to underline the need to make greater efforts to disseminate projects that have received funding, including their results. In this sense, the Evaluation Team recommends an initial, “kick-off” meeting being set up before launching the Scheme in order to make project holders familiar with the results of previous Schemes.

7. Chapter 3: Coherence of the Funding Scheme Funding Scheme coherence was analysed at two levels: Internal coherence: evaluating the extent to which Funding Scheme activities

contributed to achieving the objectives pursued by the Agency. Coherence with other Community policies: evaluating the degree of complementarity

between activities contemplated by the Funding Scheme regarding other Community Policies that might also include activities related to Safety and Health in the Workplace.

The results of the evaluation permit us to draw the conclusion that the Funding Scheme’s degree of coherence with Agency objectives is high. The results of co-funded activities contributed to meet the Agency’s objectives to a great extent. Regarding its coherence with other Community Policies and Community Institutions, research and analysis carried out conclude that the SME Funding Scheme has its own, peculiar characteristics and objectives that are clearly different to other instruments



developed by Community Institutions. Accordingly, it should be pointed out that SME Funding Scheme does not overlap other Community programmes and activities, but rather complements them. Likewise, it might be advisable to ensure co-ordination between various community institutions concerning matters dealing with Health and Safety. This would avoid overlapping of similar activities. 8. Chapter 4: Results and Impact of the Funding Scheme The qualitative information gathered from the Survey and interviews was taken into account in order to draw up this Chapter which analyses the outcome of the co-funded projects, their sustainability, the impact of the Scheme and the overall evaluation. 8.1. Outcome of the Projects The 2001-2002 Funding Scheme produced an impact at both national and EU levels. The results of the projects led to an increase in the prevention of accident hazards and fostered the development and identification of Good Practices. Those project holders consulted appraised the Funding Scheme most favourably. National Focal Points consulted indicated that the Scheme contributed very positively to:

The development and identification of Good Practices The development of risk evaluation and prevention practices The development of sustainable and European added value in Occupational Safety

and Health Activities in different countries. Projects were implemented according to the agreements signed to that purpose, with no significant deviations from the agreed budget. Most project holders (62%) reportedly accomplished their project objectives. Performance level achieved by project holders was very good and, when interviewed, project holders declared their satisfaction in having taken part in the Funding Scheme. As those indicators identified by the Funding Scheme’s Evaluation Team illustrate, co-funded projects proved extremely effective. The 2001-2002 Funding Scheme produced more than 250 publications, almost 500 Information and Communication Activities and more than 500,000 SMEs who benefited indirectly from the Funding Scheme. These indicators were quantified with the information obtained from questionnaires, Application Forms and Final Activity Reports from the 51 co-funded projects. Project holders were highly motivated and showed interest in completing their projects, something that was a key issue for the success of this first Funding Scheme. Co-funded projects also showed a high level of efficiency. Established objectives were achieved with no significant deviations from the budget originally set for the projects. The projects presented the following positive results:



An effective partnership. The average number of partners per project was three (five for transnational projects and two for national projects). Project holders appraised the partnership work carried out as being “very good”. Project holders particularly stated that transnational partnerships gave them the opportunity to learn about occupational safety and health situations in other countries and sectors.

An effective participation of the social partners and workforce. Project holders stated that the social partners and workforce provided constant support throughout the project implementation process. The workforce was the clear recipient of training activities, seminars and information. Project holders consulted expressed their satisfaction regarding both the participation of the workforce and their social partners in the projects.

Projects targeted SMEs and micro-businesses. This was especially clear in transnational projects, 88% of which concentrated on SMEs, as compared to 71% in the case of national projects.

Projects targeted priority and high-risk sectors (construction, metalworking, fisheries, etc.).

The lack of time available to implement projects reduced the effectiveness of projects. This aspect had a more damaging effect on smaller organisations that had little experience in implementing projects and programmes of this nature. 8.2. Project Sustainability Project sustainability was one of the selection process criteria to which projects had to comply. Accordingly, all projects developed under the Funding Scheme were sustainable. However, an independent assessment of project applications and Final Activity Reports showed that some project holders misunderstood the terms sustainability and dissemination, thinking that projects were sustainable when they were properly disseminated. A project is sustainable when its results last in the middle- or long-term. The definition of sustainability should be clarified in the Call for Project Proposals. It would be then advisable to insist in the Call that project sustainability should focus on: Sustainability of the results of projects, once funding has ceased. Project results that do not harm the environment.

Sustainability was a key criterion for evaluation and a precise, clear definition of the same in the future would avoid confusion among project holders and guarantee potential applicants making an effort to submit sustainable projects. In view of the brief period of time that has elapsed since the projects were completed, it is difficult for us to assess the impact of the Funding Scheme. In spite of this, we believe that the Scheme had a favourable impact on heightening the awareness of occupational accident risks at national and EU levels. As a result, most project holders (56%) pointed out the excellent contribution of the Scheme concerning SME needs regarding occupational safety and health. In the opinion of project holders, the Scheme allowed for work to start in a field where there was an important need for “prevention of accidents in



SMEs”. Direct participation of SMEs and micro-businesses also gave an insight into the true situation small companies faced. The Funding Scheme heightened the awareness of accident prevention, fostered training and produced information material allowing project holders to continue working on accident prevention in the future. 8.3. Transferability of project results The transferability of project results was a key issue concerning guaranteeing the success and impact of the Funding Scheme. Transferability of project outcomes was a selection criterion applied in the selection process, a condition that automatically lead to the conclusion that all results from co-funded projects were transferable. Main project results such as publications, CD-ROMs, websites and training materials were, in general, easy to transfer at low cost. All project holders stated that their project results were easily transferable to other SMEs. The evaluation of Project Applications and Final Activity Reports confirmed this fact. Consulted National Focal Points were of the same opinion inasmuch as the dissemination of project results provided excellent opportunities for exchanging experiences and practical solutions. National Focal Points insisted on the need to disseminate all projects at a national level, not only those corresponding to their own country, but also those of other countries as well. In this respect, the Evaluation Process includes a number of recommendations to ensure transferability: An initial “kick-off” or “start-up” meeting with selected project holders. This would allow

project holders to exchange ideas and experiences and could well be the first step towards encouraging and reinforcing the transfer of results.

The Agency will edit a publication containing all project results for dissemination purposes.

Inviting project holders to participate in promotional events organised by the Agency. Dissemination of documents in several languages.

8.4. Impact of the Funding Scheme Due to the fact that it heightened awareness of Occupational Safety and Health at EU level, the impact of the Funding Scheme was considered satisfactory. The SME Scheme was successful in improving working conditions and preventing accidents in the workplace, especially regarding those projects oriented towards Information and Communication Activities. Of all activities included in the Funding Scheme, those activities in the Information and Communication Project category reached the largest target public. The Agency intended to spend its budget equally between national and transnational projects. Most of the co-funded projects, however, were national (69%). Transnational



projects required larger budgets in order to carry out their activities and more time for co-ordination and dissemination purposes. The majority of co-funded projects fell within the Information and Communication category. The target public of the projects in this category is the largest of the three. Most Information and Communication activities concentrated upon improving access to occupational safety and health information and encouraging workforce participation in Occupational Safety and Health Practices. The survey indicated that 80% and 78% of project holders respectively were of this opinion. Analysis of the evaluation results and assessment of Applications and Final Activity Reports allows us draw the conclusion that this category was the most efficient project category, especially concerning transnational projects. The Provision of Good Practices was the second largest category. An analysis of the qualitative data, Project Applications and Final Activity Reports illustrated that project holders had difficulties in understanding the meaning of Good Practices. It is therefore recommended to include a more precise definition of this project category in the Call for Project Proposals. Training was the project category that received the least funding. The target public of this category - mainly workforce - was also quite restricted. As a result of national regulations, language differences and other factors, these projects had limited impact in terms of dissemination and transference of final results at EU level. The strategy and objectives of the Funding Scheme did indeed contribute to a reduction in the risk of accidents and the objectives it pursued coincided with the needs of project holders to a great degree. Hence, the Scheme proved most useful in preventing accidents in the workplace.



9. Chapter 5: Recommendations for Future Funding Schemes

Priority Recommendations 1. To restructure the SME Accident Prevention Funding Scheme into a Pluri-annual

Programme in order to avoid time constraints. 2. To heighten the visibility of the SME Accident Prevention Funding Scheme in relation

to other Community Programmes. 3. To establish a co-ordination mechanism to assure the complementarity between

different Community institutions in occupational safety and health matters, thereby avoiding overlaps and activities with ones having similar contents.

4. To assure transparency and impartiality in the evaluation process. Subcontracting the evaluation of applications to external specialists will prevent problems arising in the evaluation procedure.

5. Should the Funding Scheme continue to operate on an annual basis, it would be advisable to slim down the entire procedure. The Application Form should be simplified as much as possible. This would reduce the work involved for potential project applicants as well as for those taking part in the task of project evaluation.

6. The Funding Scheme should facilitate the participation of Eastern European countries, conceding greater preference to those projects including partners from Eastern countries. This will contribute towards reducing the differences in Occupational Safety and Health concerns between EU and Eastern countries.

7. To dedicate more staff to management the SME Accident Prevention Funding Scheme. The Campaigns and Programmes Task Force managed to put this Scheme into operation with a team made up, basically, of only four people. The team worked in a high workload environment under a great deal of pressure in order to comply with the Evaluation Scheme’s objectives and tight deadlines. In view of the Scheme’s timetable, allocating additional staff to handle the Funding Scheme is recommended.

8. Documents disseminated across Europe should be published in the highest number of languages possible. In order to do this, the possibility of collaborating with National Focal Points in the translation of these documents should be considered.

9. Barely six months have elapsed since completion of the Funding Scheme projects, something that has made assessing the impact of these projects a difficult task. Simple monitoring of projects from the 2001-2002 Funding Scheme is recommended.



Specific Recommendations

1. The need for more time to complete the projects. Project holders had difficulties

implementing their projects within the deadlines, which reduced project effectiveness. 2. To stress the sustainability of the projects, thereby avoiding confusion between

sustainability and dissemination of project results. 3. To improve the transferability of the results of projects. A start-up, initial meeting with

project holders might well be a first step towards reinforcing the transfer of results, together with the need to edit a publication contemplating the results of the 2001-2002 Funding Scheme in the highest number of languages possible.

4. To encourage the participation of SMEs, associations and consortiums in order to increase the number of SME project holders.

5. The Agency should organise an initial, “kick-off” meeting with project holders in order to clarify evaluation scheme procedures and to allow project holders to establish contact with both the Agency and other project holders.

6. The need to support potential applicants in creating partnerships. The Agency should create and disseminate a database containing details of those applicants who submitted applications to the Funding Scheme.

7. To improve the definition of each project category. The Funding Scheme co-funded three different types of activities: Training, Information and Communication Activities and the Provision of Good Practices. These categories caused a certain degree of confusion among project holders, some of whom did not really know in which category they should participate.

8. Much stricter eligibility criteria should be applied in order to ensure the technical and administrative quality of the projects submitted. This would stop the Agency’s receiving applications having little likelihood of being accepted.

9. To improve the clarity of the Call for Project Proposals and the Application Form. To include reference indicators in the Application Form in order to allow for projects to be monitored and evaluated.

10. To extend the Call for Project Proposals publicity campaigns in order to improve the diffusion of the message.

11. To establish a minimum number of promotion activities by all National Focal Points in order to counter-balance the dissemination and publicity of the Call for Project Proposals.

12. To use the Agency’s website as a means for channelling all the information produced as a result of the Funding Scheme. The SME Accident Prevention Funding Scheme should have a heightened, more visible presence on the Agency's website.



0- The Structure of the Evaluation report The Evaluation Report contains an Executive Summary and five Chapters. Chapter 1 introduces the objectives of the evaluation, evaluation questions, methodology used and the different evaluation phases. In Chapter 2, the report proceeds with a description and detailed analysis of programming, management, monitoring and promotion issues related to the Funding Scheme. Chapter 3 examines the coherence of the Funding Scheme. This analysis is carried out within the framework of internal coherence with other community policies. Since the Funding Scheme was launched by the European Agency on Safety and Health at Work, this analysis should consider the context of other related policies and programmes of the European Commission. Chapter 4 deals with that information collected either through surveys or by means of personal interviews with beneficiaries of the Funding Scheme, the Agency, the National Focal Points and other parties involved. Likewise, an independent assessment was made of the project documents provided by the Agency (Application Forms and Final Activity Reports) together with other relevant documents. After obtaining a clear idea of the Funding Scheme and the context in which it is operating, the next step of the evaluation process focused on the analysis of results from the survey and interviews held with parties involved in the Scheme. The Evaluation Report includes a set of recommendations to improve the Funding Scheme in the future.



1- Objectives and Methodology 1.1- OBJECTIVES OF THE EVALUATION The purpose of this Evaluation Report is to analyse the SME Accident Prevention Funding Scheme 2001-20021, the Scheme set up to tackle work-related accidents in Small and Medium-sized Enterprises (SMEs) throughout the European Union. The European Agency of Safety and Health at Work2 launched the Funding Scheme in April 2001. The Agency is a tripartite European Union organisation bringing together representatives from three key decision-making groups – namely, governments, employers and worker’s organisations - in each of the European Union’s Member States. The different bodies making up the Agency network are: The Administrative Board: establishes the Agency's goals and strategy. The Bureau: oversees the Agency's operational performance. The Agency: The Director of the Agency is the official representative of the Agency

and is responsible for day-to-day Agency operations. The National Focal Points: these are the Agency’s principal Safety and Health

Information networks in each Member State. Each government designates National Focal Points as the Agency's official representative in that country and these are normally considered the competent national authority for Safety and Health at Work. The role of the Focal Points is to provide information and feedback to Agency initiatives and products and they are consulted on all information activities related to national levels.

This report is the First Evaluation of the Funding Scheme. To a great extent, the results stemming from this evaluation will provide useful information for improving future Funding Schemes. The specifications contemplated in the Restricted Call for Tender (OSHA/TF/2002/T2) stipulate that external evaluation should be made of the Funding Scheme. This evaluation has focused upon the following items: Agency’s Call for Project Proposals

− Formulation of the Call for Project Proposals, Application Form and Technical Specifications.

− Promotion of the Funding Scheme by the Agency. − Deadline for submitting applications (from the publication of the Call to the closing

date for the submission of applications) Processing applications by the Agency

− Quality of the service rendered by the Agency to potential applicants. − Suitability of the timetable for receiving, processing and selecting applications.

Selection procedure (national and transnational projects) and advisory bodies − Was the selection procedure reasonable, transparent and fair?

1 Hereinafter called “the Funding Scheme” 2 Hereinafter called “the Agency”



− Suitability of timetable for project selection. − Suitability of national and international groups of experts. Involvement of social

partners. − Quality of the information provided by the Agency for project selection. − Quality of selected projects in relation to the objectives of the Funding Scheme.

Implementation of selected projects − Performance of project holders. − Agency performance when accompanying monitoring projects. − Clarity of the Interim and Final Activity Reports. − Quality of the information provided by the Agency concerning project selection.

Results and Impact − Outcome of projects. − Sustainability of projects. − Impact of projects. − Transferability of project results. − Overall Assessment of different project categories

1.2- METHODOLOGY FOR THE EVALUATION OF THE FUNDING SCHEME Before describing the methodology used in the Evaluation, a few comments must be made concerning the difference between programme evaluation and monitoring. These activities are inter-related, but their objectives are quite different. Monitoring refers to regular analysis of the progress being made with the programme,

using a follow-up of all programme-related activities. By and large, monitoring implies gathering a great deal of information and adopting clear criteria in the selection and organisation of this relevant information. Information produced by the monitoring process should be used as input in the evaluation process.

Evaluation assesses the implementation of a programme, not only in terms of its

impact, but also from the viewpoint of those planning, management and monitoring procedures adopted. It should be clearly stated that, since monitoring does not, on occasion, produce information in an easily accessible way, evaluators are mistakenly asked to supply this. However, the role of the evaluator is neither to produce the information supplied or produced by the monitoring process mechanism, nor to organise this in a proper manner when it is found scattered over numerous files and documents.

As the Commission itself states,3 “the expectations placed on evaluation should not exceed what it can really contribute. Evaluation should not be given roles it is not designed to play, such as auditing and monitoring”.

3 MEANS Collection, Volume 1. Evaluating socio-economic programmes. Evaluation design and management. European Commission.



1.2.1- Methodology Structure From a methodology viewpoint, evaluation of the Funding Scheme was based on gathering and analysing two different sources of information: Information provided by the Agency in relation to the 2001-2002 Funding Scheme:

− Application Forms and Final Activity Reports of the 51 co-funded projects − The SME 2001 Call for Project Proposals − The SME 2001 Application Form − Technical Specifications and Checklists − The SME 2001 Decision and Notification of the SME 2001 Decision to National

Focal Points − Financial Planning for the Selected SME 2001 Projects − Documents (Matrix Document for the Bureau, Feb-March 2001, SME 2001 Key

Issues, Common Guidelines for National Focal Points) − Notes (to the Bureau, Feb-March 2001, to National Focal Points and Board

Members to inform of the SME 2001 Call, Overall Note for the Assessment Meeting, 17th –18th October 2001)

− Minutes (SME/EW Group Meeting 29th – 30th March 2001, SME European Jury Meeting 13th – 14th September 2001 and Jury Recommendations, Assessment Meeting, 17th – 18th October 2001).

Information generated during the evaluation The evaluation has made use of qualitative information collected using surveys (questionnaires sent to project holders and National Focal Points), complementary activities (telephone calls) and interviews (personal and telephone interviews with project holders, interested parties, National Focal Points and the Agency).

The following groups were targeted: Beneficiaries of the 2001 Funding Scheme (mainly project holders) Agency staff Agency Administrative Board Members European Jury and SME/EW Group National Focal Points Other Interested parties: European Commission, European Parliament, European

Trade Union Confederation (ETUC/TUTB), Union of Industrial and Employers Confederations of Europe (UNICE), Federation des Entrepreneurs Generaux de la Construction, ECOSOC (Economic and Social Committee of the European Union)



The methodology used for collecting and analysing the information was based on deskwork (compilation and analysis of information) and fieldwork. These processes are set out below (see chart in Annex 1): Gathering of information from the Agency Identification and preliminary analysis of essential aspects of the Funding Scheme Elaboration of Questionnaires for Project Holders and National Focal Points: Survey Personal interviews (Agency Members, Project Holders, National Focal Points,

Interested Parties) Summary of Information generated during the evaluation process

1.2.2- Evaluation phases Funding Scheme evaluation was structured as per the 4 phases described in the following table:


EVALUATION OF THE SME ACCIDENT PREVENTION FUNDING SCHEME 2001-2002

20

Table1: Evaluation Phases (from the 21st November 2002 to the 31st March 2003) Main Objectives Timescale Meetings Reports Phase 1: Structuring

Structure of general evaluation method

Collection of information provided by Agency

21st November – 5th December 2 weeks

Kick-off meeting 21st November

First Progress Report: 5th December This Progress Report covered progress made from the kick-off meeting. The objective of the report was to present, on one hand, the planning for implementing the foreseen evaluation tasks (work plan, time schedule) and, on the other, the methodology used in carrying out the evaluation.

Phase 2: Data Collection and Preliminary Analysis

Questionnaire target groups Preliminary selection of project

holders and other interested party interviews

Templates for questionnaires (in English, French, Spanish, Italian and Portuguese)

Despatch of questionnaires to 51 project holders and 15 National Focal Points (20th December)

5th – 26th December 3 weeks

Progress meeting 11th December

Inception Report: 19th December The Inception Report was based on the First Progress Report. The objective of the Report was to present the conclusions of the Planning Phase. The Report included an agreed work-plan and evaluation framework (evaluation focus, methodology used, target groups for interviews and indicators). Templates for questionnaires were included in the Annex.

Phase 3: Fieldwork

Reception of questionnaires Reinforcement activities Interviews

26th December – 30th January 5 weeks

Progress meeting 31st January

Second Progress Report: 30th January This report described the results and activities carried out during Phase 3: the activities implemented regarding the questionnaires and interviews. Checklists for interviews were included in the Annex.

Phase 4: In-Depth Evaluation and Assessment

Final reception of questionnaires End of Interviews Draft of the Final Evaluation Report Final Evaluation Report

30th January –31st March 8 weeks

Progress meeting: 21st February Meeting on Draft Evaluation Report: 20th March

Findings: 19th February Main results obtained from the interpretation of the survey process were presented on this report. Third Progress report: 28th February The Third Progress Report described the results and activities carried out during Phase 4 (from the end of Phase 3 to the end of February). Draft Evaluation Report: 13th March Final Evaluation Report. 31st March



1.2.3- Questionnaires and survey sample Two different questionnaires were designed: one for project holders and another for National Focal Points, both of which are enclosed in Annex 2 (Questionnaire for Project Holders) and Annex 3 (Questionnaire for National Focal Points). Collaboration with the Agency prior to the survey was essential, not only in order to confirm and approve the questionnaires, but also to draw up an official letter informing project holders and National Focal Points of the aim and scope of the evaluation. On the 20th December 2002, English-language questionnaires were despatched by E-mail to all project holders (51 questionnaires) and EU National Focal Points (15 questionnaires). The deadline for receiving the completed questionnaires was the 20th January 2003. The E-mail contained the following points:

1- The letter from the European Agency for Safety and Health at Work signed by the Director of the Agency, Hans-Host Konkolewsky.

2- The letter from IDOM presenting the evaluation of the SME Accident Prevention Funding Scheme 2001-2002.

3- The Questionnaire.

1.2.4- Implementation of the survey In order to facilitate response, questionnaires were produced in six languages (English, French, German, Italian, Portuguese and Spanish) and despatched individually according to the country of the project holder or National Focal Point. Back-up activities commenced on the 20th January:

1. On the 21st January, the reception date was extended by one more week: i.e. till the 27th January 2003. This postponement was notified by E-mail.

2. From the 28th – 30th January, all project holders who had not responded to the questionnaires were contacted by telephone.

3. On the 4th February, the final submission deadline was postponed till the 7th of February. This postponement was notified by E-mail.

The results obtained in terms of a response to the questionnaires were very satisfactory for the following reasons:

The very high response rate (90% project holders, 100% National Focal Points) Most of the questionnaires were fully completed.

This provided the Evaluation Team with important information that, in turn, improved the results of the evaluation. The results obtained in the survey process are reflected in the following table:



Table 2: Survey process Number of

questionnaires received from

Project Holders

% of total Project Holders

Number of questionnaires received from

National Focal Points

% of total National

Focal Points

Deadline 20th January

21

41.1 6 40

Deadline 27th January

31 60.7 10 66.6

Deadline 7th February

44 86.2 14 93.3

Questionnaires Received

46 90.1 15 100

Questionnaires still pending reception

5 9.8 0 0

TOTAL Questionnaires despatched

51 100 15 100

1.2.5- Implementation of Interviews As specified in the Terms of Reference, a minimum of 10 project holders had to be selected for personal interviews. The criteria used for this selection was: At least one Project from each category (Information and Communication, Provision of

Good Practices, Training) Balance between national and transnational projects Number of projects presented per country Geographical distribution Partnership Proximity to the Evaluation Team’s offices Logistics: Proximity to an European capital or international airport

With regard to transnational projects, the following specific criteria were used in addition to the general criteria, The European dimension of the project The EU partnership

Overall, 13 project holders were consulted by means of personal interviews (three more than initially envisaged) with Agency personnel, National Focal Points and other interested parties at EU level. A detailed list of the interviews carried out is provided in the following table:



Table 3: Interviews National Project Project Holder Project Reference/

Location Type of Project

CONFEBASK - Confederación Vasca Empresarial- 4766 - Spain (Bilbao) Provision of Good Practices TREVISO TECNOLOGIA 4744 - Italy (Treviso) Provision of Good Practices ACS - Association of Charity Shops 4975 - UK (London) Provision of Good Practices TECHNIKI EKPEDEFTIKI 4702 - Greece (N.

Ionia) Information/Communication

OPPBTP – Organisme Professionnel de Prévention du Bâtiment et des Travaux Publics

4799 - France (Paris) Training

ISTAS- Instituto Sindical de Trabajo, Ambiente y Salud 4822 - Spain (Madrid) Information/Communication ISME- Irish Small & Medium Enterprises Association Ltd 4962 - Ireland (Dublin) Training Transnational Projects ETUC-European Trade Union Confederation 5335 - Belgium

(Brussels) Information/Communication

Union de l´Artisanat et des Petites et Moyennes Entreprises 5457 - Belgium (Brussels)

Provision of Good Practices

Chambre de Commerce Grand-Duché de Luxembourg 5557 - Luxembourg Provision of Good Practices Federación Estatal de Transporte, Comunicación y Mar, UGT

5681 - Spain (Madrid) Information/Communication

Preventie en Interim 5478 - Belgium (Brussels)

Training

Berufsgenossenschaft Nahrungsmittel und Gaststätten 5717 - Germany (Mannheim)

Information/Communication

The European Agency for Safety and Health at Work Hans-Horst Konkolewsky, Agency Director Françoise Murillo, Head of the Task Force Campaigns and Programmes Pascale Turlotte, Financial Assistant, Task Force Marta de Pardo, Secretary of the Task Force Christa Sedlatschek, Head of Working Environment Unit Andrew J. A. Smith, Head of Information and Communication Unit Agency Administrative Board Members Bertil Remaeus, Chairman Luis F. Do Nascimento Lopes, Vice- Chairman Celia Alexopoulou, European Commission Representative Christa Schweng, Employer’s Representative SME European Jury Bertil Remaeus, Government Representative Claudio Stanzani, Worker’s Representative Christa Schweng, Employer’s Representative Celia Alexopoulou, European Commission Representative



National Focal Points Reinhard Gerber (Germany) Willy Imbrechts (Belgium) Ms. Loft (Denmark) Elizabeth Galanopoulou (Greece) Other Interested Parties European Commission Angel M. Fuente Martín, DG, Employment and Social Affairs Celia Alexopoulou, DG, Employment and Social Affairs European Parliament Stephen Hughes European Trade Union Confederation (ETUC/TUTB) Janine Delahaut Marc Sapir Federation des Entrepreneurs Generaux de la Construction Andre Pelegrin Union of Industrial and Employers Confederations of Europe UNICE Christa Schweng ECOSOC (Economic and Social Committee of the European Union) Alan Hick



2- The SME Accident Prevention Funding Scheme 2001-2002: Programming, Management and Monitoring Issues The aim of this Chapter is to analyse the programming, management and monitoring stages of the 2001-2002 Funding Scheme, in addition to providing an analysis of the goals of the Scheme and the tools used to achieve such goals. 2.1. DESCRIPTION OF THE FUNDING SCHEME

2.1.1- Aims and Scope of the Funding Scheme In 2001, the Agency launched a Funding Scheme to support projects aimed at reducing the risk of work-related accidents in European SMEs. The objective was to bring this reduction about by providing Effective Good Practices drawn up in accordance with the relevant legislative requirements established by European Directives, national legislation and guidelines. Three different categories of national and transnational projects focusing on priority hazards and high-risk sectors were financed:

1. Training 2. Information and Communication 3. Provision of Effective Good Practices

As stated in the Call for Project Proposals, the overall objective of the Scheme was to “identify, communicate and support activities and projects with effective added value that could motivate, develop, support and sustain effective organisation of OSH/OSH management in SMEs”. Table 4: Objectives of the 2001- 2002 Funding Scheme Objectives of the 2001- 2002 Funding Scheme To heighten awareness of accident risks and of the heavy load borne by workers and their families, as well

as the considerable economic consequences for SMEs. To promote the development and identification of effective good practice examples reducing accident risks

in SMEs and facilitating their dissemination across Europe. To promote the development of risk evaluation and prevention practice as embodied in Safety and Health

Directives and the provision of early intervention regarding diagnosis and treatment and to motivate decision-makers to take action and develop preventive measures.

To encourage the development of sustainable and European added value Occupational Safety and Health (OSH) Activities involving employers, workers and their representatives, or partnership programmes, particularly those developed by intermediaries working directly with SMEs.

To contribute to reducing the number and gravity of work-related accidents in SMEs. To promote OSH at work as part of the business thinking and organisational development and

demonstrate to SMEs that “Good Safety and Health is Good Business”. To address the diversity of SMEs in Europe and respond to specific needs.

The budget for the SME Scheme (5 Million EUR) was allocated to the Agency by decision of the European Parliament. This budget was drawn up on an annual basis.



2.1.2- Phases of the SME Accident Prevention Funding Scheme The implementation of the Scheme involved a number of different, but inter-connected phases or steps. This process was structured into three basic sections. The Programming of the Funding Scheme: including the planning of the Agency’s Call

for Project Proposals (Call, Application Form, Technical Specifications, Deadline for the Submission of Applications) and the Selection Procedure.

Management and Monitoring of the Funding Scheme: including monitorisation of co-funded projects and evaluation of the Funding Scheme’s achievements.

Promotion of the Funding Scheme. This stage is horizontal and was carried out during the entire process.

The aspects assessed during each of these stages are highlighted in the following table: Table 5: Programming, Management, Monitoring and Promotion of the 2001-2002 Funding Scheme 1-Programming the Funding Scheme Contents of the Agency’s Call for Project Proposals - Overall strategy and objectives - Types of activities to be co-funded - Eligibility of applicants - Eligibility criteria - Selection criteria and priorities - Budget and financial conditions Evaluation of the Call for Project Proposals, Application Form and Technical Specifications - Clarity of the Call, Application Form and Technical Specifications - Deadline for submitting applications - Means by which applicants learnt about the publication of the Call Processing of applications by the Agency - Quality of the service provided to potential applicants - Suitability of the timetable for receiving, processing and selecting applications for both applicants and the

Agency Selection procedure and advisory bodies Selection procedure for funding projects

- Publication of the Call for Project Proposals in the Official Journal of the European Community (OJEC)

- Pre-selection of applications received - Assessment of projects for funding - Publication of a list of selected projects

Evaluation of the Selection procedure - Was the selection procedure impartial, transparent and fair? - Was the timetable for assessment appropriate? - Was the composition of the national and international groups of experts suitable? - Did social partners get significantly involved? - Did co-operation with the groups of experts receive sufficient assistance from the Agency?

2-Management and Monitoring of the Funding Scheme Monitoring of the implementation of co-funded projects and financial audits

- Agency’s performance when supervising and monitoring the projects - Clarity of the Interim and Final Activity Reports

Evaluation of the achievements of the Funding Scheme - Quality of awarded projects regarding the objectives of the Funding Scheme

3- Promotion of the Funding Scheme (horizontal)



2.2. PROGRAMMING OF THE FUNDING SCHEME

2.2.1- Contents of the Agency’s Call for Project Proposals The aim of this section is to assess the planning of all those preparatory activities carried out by the Agency up to the drafting and publication of the Call. 2.2.1.1- Overall strategy and objectives Project holders surveyed and interviewed during the evaluation process emphasised the need to begin work on accident prevention and the Funding Scheme represented a good opportunity for doing exactly this. The Scheme stimulated an increase in accident prevention awareness, fostered training and produced material that would permit project holders to continue working in the future. Project holders evaluated the strategy and objectives of the Funding Scheme as being positive. Hence, the overall strategy and objectives of the Funding Scheme contributed towards reducing the risk of accidents. Most of the project holders considered that the Funding Scheme was very useful in preventing accidents in the workplace and that the objectives it pursued coincided with their needs to a great extent. Among those objectives of the Scheme most valued by project holders, the following are especially relevant: promoting occupational safety and health (OSH) at work as a part of business thinking, developing and identifying Good Practices and heightening awareness concerning accident risks. The National Focal Points interviewed also considered that the objectives of the Scheme coincided with the needs of SMEs, as well as national priorities regarding OSH. 2.2.1.2- Types of activities to be co-funded Project Category The Funding Scheme co-funded three different types of activities: Training, Information and Communication and the Provision of Good Practices. Training related to the prevention of work-related accidents in SMEs

The Call specified the conditions that training-focused projects had to comply with. Training activities at all levels were included, in addition to all training methods (including vocational training, training of trainers) targeted towards SMEs. The Call specified that training was to be focused on owners and managers of SMEs, the workforce and safety representatives working in SMEs and be aimed at enabling them to effectively manage and face OSH challenges. Preference was given to accident prevention training programmes rather than single training courses.

Information and Communication on accident prevention targeted at SMEs In line with the Call, this category covered activities intended to: − Improve access to OSH information about Good Practices. − Encourage the participation of workers and /or their representatives in OSH. − Broadcast the OSH message to SMEs.



Provision of Effective Good Practices reducing accident hazards in SMEs The Call pointed out that projects centred on Good Practices had to: − be related to innovative Good Practices and provide added value to the prevention

of work accidents at national and European levels. − be applicable to SMEs, thus demonstrating a real (i.e. not theoretical) identifiable

intervention regarding risk prevention in the workplace. − be focused on priority, high or new risks.

In accordance with the survey, it might be concluded that these three categories were sufficient for contributing to accident reduction in SMEs. Analysis of Application Forms and Final Activity Reports reflected a certain degree of confusion regarding the category in which project holders had participated. This was particularly the case for projects oriented towards the Provision of Good Practices. Consequently, this has hampered the work of the Evaluation Team when drawing up and quantifying the final indicators for the Scheme. Therefore, it is recommended to define each category clearly and, more specifically, especially set out and clarify what is meant by the term “Good Practices”. Regarding projects co-funded by the Agency, it has been observed that these categories correspond more to the needs of intermediate bodies (trade unions, associations, business organisations, etc.) than those of SMEs. Project holders indicated the difficulty of an SME to act as the leader of an Information and Communication or Good Practices Project due to the dynamics of the work and the size and resources of SMEs or micro-businesses. Exclusion Criteria The Call for Project Proposals specified that research projects, conferences, capital purchases as well as already-completed activities were not eligible. On the whole, these exclusion criteria proved satisfactory. Particularly regarding research projects, the Evaluation Team identified specific Community financing for SMEs (within the Framework Programme of the Directorate General for Research of the European Commission). The Framework Programme allows for the direct participation of SMEs in research projects that might deal with activities in the field of accident prevention (see Chapter 3, Coherence with Community Policies). Regarding conferences, the Evaluation Team identified a certain degree of confusion as a result of the independent assessment of projects, especially at transnational level. Project holders named several events as “Seminars” since conferences were excluded. For this reason, we recommend that a more precise definition of the concept (seminars, workshops, conferences) be included in the Call for Project Proposals. In addition to which, a conference is a good tool for dissemination purposes. If the objective of the Funding Scheme is to disseminate results, excluding a Conference might possibly be considered as being a contradiction.



National/ Transnational Projects Project applicants were able to submit projects at two levels (national and transnational). A project was considered to be national when "applications were submitted by one or several partners belonging to one single Member State". A project was transnational "if organised by at least two co-organisers from a minimum of two different Member States ". The classification of national or transnational projects was satisfactory and these categories were clearly defined in the Call for Project Proposals. Nevertheless, the need for a larger budget and duration for transnational projects was pointed out by the project holders consulted. Co-ordination and dissemination of a transnational project requires a great deal of effort in terms of budgeting and technical resources. Project holders emphasised the difficulty of finding partners in order to prepare projects at transnational level. Therefore, it might be advisable to draw up a list of potential partners, including applicants who had submitted applications in previous Schemes who could be listed as being potential partners for future Schemes. The list would be made available to any potential applicant that might require this information. Duration of the projects The projects that received funding had a maximum duration of one year due to the fact that the Agency's budget was structured on an annual basis. Most project holders highlighted the lack of time available in order to carry at this type of project. This time limit reduced project effectiveness. The Call for Project Proposals did not indicate the maximum duration of projects, nor start date. Due to the strict time limits imposed by the Funding Scheme, it would be advisable to specify this information in the Call, since it would help improve its being understood. 2.2.1.3- Eligibility of applicants As highlighted in the Call for Project Proposals “Any natural person or association of natural persons, any corporate body (institution, association, enterprise…) legally constituted and registered under the legislation of one of the Member States of the European Union may submit proposals”. The results of the assessment showed that this classification was satisfactory since it covered a wide range of potential applicants. The projects presented were SME-related, although most applicants were organisations representing either employer’s or employees’ interests. Project holders and National Focal Points interviewed manifested the difficulty for SMEs to lead a project in the Funding Scheme for several reasons:



Lack of resources Enterprises that had never participated in such initiatives confirmed encountering

difficulties in understanding the “jargon” used in the Call, Application Form and Technical Specifications.

Lack of capacity of enterprises to conduct large-scale promotion of the project. The Funding Scheme should be aimed at SMEs and micro-businesses. Given the fact that a difficulty for an SME to conduct this type of project was identified, encouraging the participation of SMEs by setting up associations and consortiums is recommended. 2.2.1.4- Eligibility criteria Eligibility criteria refer to those conditions that applications must comply with in order to be eligible for receiving funding. These criteria involve an initial selection of projects and only those strictly complying with these criteria proceed to the next selection stage. The Call for Project Proposals made it clear that only those projects that were properly completed according to the objectives and specifications of the Call, in full respect of the technical conditions and submitted (received) to the Agency within the deadline stipulated were eligible. These general criteria proved sufficient. However it might be advisable to define stricter eligibility criteria in order to avoid applications having little likelihood of being accepted being received. At the same time, more restrictive selection criteria would free the Agency, and those participating in the project selection process, from the administrative load involved in assessing the projects. Some of the aspects that might be considered as more restrictive eligibility criteria are described below:

Projects should involve a significant number of partners. Applicants should be asked to demonstrate that the project does not have any

other Community funding for the activities envisaged. Applicants should propose a realistic and balanced financing plan.

Regarding the submission of applications, the introduction of a new system for the submission of applications is recommended in order to guarantee the project holders participation under equal terms. This system would consider those applications bearing a postmark within the official deadline, date and time, established in the Terms of Reference as being valid. Nevertheless, the Evaluation Team is aware that this aspect has been attended to in further Calls for Project Proposals. 2.2.1.5- Selection criteria and priorities Selection criteria permitted measurement and assessment of the quality and relevance of those applications presented, as compared with the objectives of the Funding Scheme. Once the Agency had checked the applications against eligibility criteria, applications were assessed using a number of selection criteria and with regards to certain priorities. The



criteria to assess applications were both obligatory and additional, in addition to which, a further set of priorities was also taken into consideration: Table 6: Selection Criteria

Obligatory criteria Additional criteria Priorities To facilitate the provision of

Effective Good Practice examples aimed at reducing work accidents in SMEs and complying with relevant OSH requirements.

To illustrate added value and the transferability of the project’s outcome.

To be sustainable- i.e. to demonstrate the potential lasting effect.

To show feasibility, coherence and soundness of management and ability to make the outcome of the project known and available to others.

To support existing structures of SMEs/OSH.

To develop or initiate networking.

To be risk or accident sector oriented.

To contribute to risk assessment and successful management in preventing accidents.

To demonstrate effective consultation and involvement of social partners or the effective participation of the workforce.

Partnership projects (involving several organisations).

Projects involving social partners and tripartite projects (dialogue/co-operation of public authorities with social partners).

Projects with special focus on micro-businesses and small enterprises (up to 50 employees).

Action-oriented projects rather than projects oriented towards production of material, projects proposing structural schemes or models allowing spin-off and further developments.

Projects that can be generalised, having a high potential for heightening motivation/awareness and a potentially lasting effect (e.g. the ability to make the results known and available to others).

Priority risk activities, high-risk sectors and new risk-oriented projects.

Of the 410 eligible applications, only 51 passed the selection process. These results show that only 12% of the total number of applications submitted received co-funding from Agency. In view of the limited amount of human resources allocated to the European Agency Task Force Campaigns and Programmes (responsible for managing the Funding Scheme) and most National Focal Points (responsible for assessing projects at national levels), it is recommended to introduce stricter eligibility criteria for the initial project selection process. Selection criteria had also to be clarified. In principle, the difference between a selection criterion and the priorities for selecting projects does not seem to be clear. Moreover, the additional selection criteria repeat many of the priorities indicated (the need to be risk oriented, networking, partnership, etc). Considering some of the priorities (partnership, projects involving social partners, projects with special focus on SME or micro business, projects focused on priority risks, high-risk sectors and new risk-oriented projects) as eligibility criteria is recommended. This would allow for an increase in effectiveness of the Funding Scheme assessment process. Among these criteria, the Evaluation Team observed that no additional value was attached to the technical capacity or expertise of potential applicants. The experience of project holders should be a selection criterion itself for applications.



We would also recommend including a section in the Call for Project Proposals providing potential applicants with information on the project selection process at both national and transnational levels. Finally, we would include an indication of how applications were scored. This would provide potential applicants with information concerning how their applications are assessed. A proposal on how projects of this type could be assessed is set out below: Technical and financial capacity of the Project Holder: 30% Technical proposal: 45% Financial proposal: 30%

2.2.1.6- Budget and financial conditions The total budget for the Funding Scheme was 5 Million EUR, providing co-funding for a total of 51 projects, of which 35 were national and 16 transnational. Grants received by each project varied between €25,000 and €190,000. National projects received a maximum co-funding of 60% as compared to transnational projects, which received a maximum co-funding of 80%. Payments were made in two or three instalments, depending on the amount allocated: Two instalments for smaller grants (less than or equal to €100.000): advance payment

of 80% within 60 days following the signature of the grant agreement and a final payment after the approval of the Final Report and financial statement.

Three instalments for grants over €100,000: 1) an advance payment of 50% within 60 days following the signature of the grant agreement, 2) a second payment after approval of an Interim Report (30%) and, 3) final payment of the remaining balance after the approval of the Final Report and financial statement.

In general, project objectives were achievable within the available budget, although project holders pointed out that a larger budget would have allowed them to attain more effective results in terms of the dissemination of project results. The need to provide more resources for activities at a transnational level was reiterated by project holders.

We recommend financing fewer projects and providing each of them with larger budgets, especially those projects of a transnational nature. This would increase the visibility of the Funding Scheme.



2.2.2- The Call for Project Proposals, Application Form and Technical Specifications The aim of this section is to assess the Call, Application Form and Technical Specifications from the point-of-view of clarity and their usefulness to potential applicants. Deadlines for submitting applications and promoting the Funding Scheme are also evaluated. 2.2.2.1- The Call, Application Form and Technical Specifications The Call was the document informing project holders of how and when applications had to be submitted. The Call consisted of 13 paragraphs4, which have been assessed in the previous paragraph 2.2.1: “Contents of the Call”. The Application Form was the document in which project holders set out their project proposals comprised five Parts5: The Agency provided potential applicants with the Technical Specifications and Checklist, which provided potential applicants with information on how to fill out the Application Form together with providing them with other relevant information. The Call, Application Form and Technical Specifications were formulated in a clear and understandable manner, providing sufficient information to potential applicants. Project holders surveyed declared that these documents were produced in a much clearer way than in other Community Programmes, such as the European Social Fund (ESF). These results are shown in the following chart:

4 1- Background, 2- Context and Objectives, 3- Budget, 4- Types of eligible project activities, 5- Eligibility of applicants, 6- Level of organisation of projects, 7- Duration of projects, 8- Eligibility criteria, 9-Selection criteria, 10- Financial conditions, 11- Acknowledge and publicity, 12- Submission of applications, 13- Selection and approval. 5 PART I “Brief Presentation of the Project –Summary Sheet“, PART II “Administrative details related to the Applicant”, PART III “Estimated Budget”, PART IV “Detailed description of the project activities”, PART V “Administrative requirements”.



Chart 1: Project Holder Assessment on Call, Application Form and Technical Specifications

As these figures show, the Call was clear for the great majority of project applicants. Attention should be drawn, however, to our recommendation to bring forward the publication of the Call. With regard to the Application Form, the Evaluation Team has identified a number of aspects that could be improved upon in future Funding Schemes: The need to facilitate an understanding of the Application Form by avoiding repetitive

questions. An example of this is that section 11 of PART I, “A Brief Presentation of the Project – Summary Sheet“, requires specification of the main objectives, target groups and expected outputs/products. This is required again in section 15. Considering that PART I contains a brief presentation of the project, section 15 could be done away with.

With regard to PART III – “BUDGET”, a more-detailed Budget Sheet is required. The administrative requirements of PART V are difficult to comply with. In view of the

deadlines for the submission of projects in the case of projects having a large number of partners and transnational projects, many project holders found this section difficult to complete. A great deal of time is required to get each and every one of the participants to sign the same form. We recommend the incorporation of a Letter of Intent that could be attached to the Application Form. Each participant would then sign this Letter of Intent separately.

Reference indicators should be included in order to allow for project monitoring and evaluation.

Project holders, National Focal Points and interested parties pointed out that the language used was too formal and difficult for SMEs to understand, especially for those SMEs that have never taken part in this kind of programmes.

93.5

6.5

84.8

15.2

84.8

15.2

0102030405060708090

100

Call Application Form TechnicalSpecifications

% Clear % Unclear



The effort made by the Agency to deliver these documents in 11 languages was highly valued. Deadline In general terms, the deadline for submitting the Application Forms was sufficient. This was the opinion of the majority (96%) of project holders. The remaining project holders admitted that they did not have enough time to complete the Application Forms due to the fact that they received this information late or because they had problems with administrative requirements, especially at transnational level. How applicants learned about the Publication of the Call The evaluation results showed that most Project Holders learned about the Funding Scheme through the Agency’s website. Approximately one third of the project holders consulted learnt about the Scheme through Internet (including the Agency's website). A considerable number of other project holders (27%) discovered the Scheme’s existence thanks to the National Focal Point Network. 15% of project holders were informed directly by the Agency and only 4% found out about the Scheme through the Official Journal of the European Communities (OJEC). The remaining project holders gleaned information about the Scheme through other sources such as European Information Centres, trade magazines, etc.

2.2.3- Processing applications by the Agency The objective of this section is to assess the service provided by the Agency in response to the doubts expressed by potential applicants and the suitability of the timetable for receiving, processing and selecting applications. 2.2.3.1- Quality of service provided to potential applicants The service provided by the Agency to potential applicants was very good. Most of the project holders were satisfied with the Agency service, as shown in the following chart.



Chart 2: Project Holders’ Assessment of the Quality of Service Provided by the Agency

Project holders emphasised the excellent service provided by the Agency in response to their questions, its providing them with additional information and supporting and monitoring their projects (see Chart 3). Project holders contacted the Agency by phone or E-mail. In these instances, the Agency replied within 24 hours, thereby demonstrating the efficiency of those services provided by the Agency.

58.6

46.2

34.5

46.2

0

7.7 6.90 0 0

0

10

20

30

40

50

60

5-Excellent 4-Good 3-Medium 2-Bad 1-Very Bad

% National Project Holders % Transnational Project Holders



Chart 3: Project Holders’ Assessment of Agency Performance (in %)

The service provided by the Agency was assessed positively. Project holders considered the Agency as a reference-point for issues related to national and transnational projects. However, this has created a work overload in the Agency. 2.2.3.2- Suitability of the timetable for receiving, processing and selecting applications for both applicants and the Agency The deadline set for receiving, processing and selecting applications was very tight. This deadline was adhered to by the Agency. Those steps followed in the selection of national and transnational projects are detailed in the following table:

6131

. 77.

30 0

41 4117

.90 0

28.3 30

15 13. 3

13.3

36.4

45.5

12.1

06.

1

0

10

20

30

40

50

60

70

Answering questions Giving additionalinformation

Supporting Projects Monitoring Projects




Table 7: Timetable for Receiving, Processing and Selecting Applications On the 17th July, the Agency despatched paper copies of the 282 national projects received within the

deadline (29th June) by private courier service. A database was despatched by E-mail to the respective National Focal Points on the 19th July. The deadline for notifying the results of the national evaluation was the 15th September 2001. These results were despatched by the National Focal Points in a database and by E-mail.

At transnational level, 129 transnational projects received within the deadline (16th July) were eligible. On the 30th of August, the Agency despatched these projects to the European Jury in CD-Rom format. Later, on the 7th September, the Agency sent 30 project applications resulting from the previous evaluation of the Agency to the European Jury. At the meeting of the European Jury on 13th and 14th September, 15 transnational projects were proposed for co-funding.

Finally, a meeting was held with the SME/EW group in Bilbao to discuss national and transnational projects on the 17th and 18th October.

On the 24th October, the Agency adopted the final decision to co-fund 35 national and 16 transnational projects. A reserve list was also set up which included 3 national projects.

The Task Force Campaigns and Programmes operated in a high workload environment with tight deadlines. In view of this timetable and the human resources required for the Task Force to implement these activities within the aforementioned deadlines, we recommend that additional resources be available during this process.

2.2.4- Selection procedure and advisory bodies 2.2.4.1- Procedure for selecting projects for funding Publication of the Call for Project Proposals in the Official Journal of the European Community (OJEC) The publication procedure was designed in accordance with the following priorities: - The need to be as simple as possible. - The need to comply with the rules contemplated in the Vademecum for Grants by the

European Commission for Community Funding. In accordance with the Commission’s policy of transparency, the Call was published in the OJEC on the 21st April 2001 (2001/S 78-053024). There was only one Call for Project Proposals and potential applicants had two different deadlines for submitting their applications: national project proposals had a deadline of the 29th June 2001 and transnational project proposals on the 16th July 2001. As a result, national applicants had 11 weeks to submit their proposals and transnational applicants had 13. Pre-selection of applications All applications received within these deadlines were registered in an Access database by the Agency. The Agency checked that the applications submitted met the formal requirements specified in the Call - i.e., that they complied with the established eligibility criteria. Following the Publication of the Call, the Agency received 456 applications for financial support, of which 410 were eligible.



Assessment of projects for funding The Agency first checked that the Application Forms complied with the established eligibility criteria before proceeding with the evaluation procedure. The evaluation of national differed from that of transnational applications. National applications were forwarded to the Agency’s National Focal Points for assessment, whereas a “European Jury” evaluated transnational applications. Once the evaluation of the applications had been completed, the Agency compiled a “short list” which was then forwarded to a Joint Selection Committee (consisting of National Focal Point representatives and the European Jury). This Committee produced the final recommendation for project co-financing, which the Agency followed in its final decision. The project selection process is summarised in Table 8: Table 8: Summary of Project Selection Process Agency: Registers and checks that applications are eligible. Pre-selection: European Jury (transnational projects) and National Focal Points in a co-decision with the

social partners (national projects). Assessment of projects: European Jury and SME/EW Group. Agency takes the final decision

This process of internal technical evaluation took between 12 to 14 weeks. Evaluation paid particular attention to the selection criteria and priorities for project selection that had been indicated in the previous stage. Publication of a list of selected projects: contractual, administrative and financial related tasks The Agency notified applicants in writing as to the results of the assessment. The Agency was also responsible for carrying out all administrative, financial and contractual tasks with project holders. After project selection had been completed, contracts were drawn up with project holders. Generally speaking, this occurred during the months of November and December. In spite of the fact that the selected project holders were able to begin the project automatically (without signing the contracts), the majority did not do so. This led to a reduction in the time necessary to carry out the projects, as is described below:



2.2.4.2- Evaluation of the selection procedure Selection procedure The selection procedure was impartial, transparent and fair. Priority was given to the transparency and fairness of the evaluation process at both national and transnational levels. In order to do so, the Agency provided national and transnational evaluation teams with common rules and precise guidelines for assessing applications and avoiding “judge and jury problems” and “conflicts of interest” that might have resulted from the fact that evaluators could have been directly, or indirectly, related to the applications being assessed. In this respect, evaluators having any connection whatsoever with a project proposal were not allowed to take part in the evaluation of that particular application. The Agency informed project holders in writing of the status of their applications. Likewise, project holders stressed the need to include the results of their application assessment in that letter. Unsuccessful project holders also received a letter from the Agency. This letter did not include an evaluation of the strong and weak points of these projects. Project holders have requested that an assessment of their application be included in this letter so that they can improve the quality of their applications in future Schemes. Timetable for Assessment In general terms, there was not enough time to assess applications properly. This is also applicable to the type of project assessed in each case: Concerning transnational projects, most members stressed the lack of time available

for assessing projects. The CD-Rom containing the transnational projects arrived some ten days before the meeting during which final evaluations were made. Jury members emphasised the need for more time in which to assess projects. The European Jury received a CD-Rom with 129 transnational eligible projects on the 30th August. Later, on the 7th September, the Agency despatched 30 project applications resulting from their previous evaluation. They had approximately 10 days in which to assess transnational applications.

Regarding national projects, it should be pointed out that the response depended greatly on the number of projects that had to be evaluated at national level. Those countries assessing fewer projects indicated that they had sufficient time, but on the other hand, those countries receiving a large number of candidatures6 had a great deal of trouble in assessing them. Likewise, the very structure of the National Focal Points also influenced the lack of time. Other National Focal Points stated that the deadlines were tight to assess projects in co-decision with social partners, particularly due to the fact that evaluation had to be carried out over a holiday period.

6 Major part of the applications fell to the Spanish Focal Point (>100 applications).



Composition of national and international groups of experts At transnational level, the European Jury was made up of six members: government representatives, workers’ representatives, employers’ representatives, European Commission representatives and two experts, one of whom was the Chairman of the European Jury. Moreover the European Agency acted as the Secretariat of the Jury, including the Director of the Agency and the Task Force Campaigns and Programmes. The composition of the national and international expert groups was considered to be satisfactory. The European Jury and National Focal Points evaluated the proposals with the help of the tripartite network and Safety and Health at Work experts. The Agency acted as Technical Secretariat for the evaluation of transnational projects, providing specific support to National Focal Points, which has proven very useful when carrying out evaluations at national levels. Assistance and quality of the information provided by the Agency for project selection Information and assistance provided by the Agency to the “European Jury” From the interviews held, it can be deduced that all information provided by the Agency was both clear and useful. The preliminary selection of projects carried out by the Agency was considered to be satisfactory and was, generally speaking, respected and maintained by the "European Jury". The collaboration and availability of the Agency, as well as its preliminary work, were considered to be excellent. Information provided by and assistance from the Agency at national level According to the model agreed upon by the Agency Bureau, national applications had to be evaluated by National Focal Points in a co-decision with social partners. National Focal Points were therefore responsible for selecting national applications. The Agency provided a series of indications to be followed when evaluating projects. Though the selection criteria and priorities for choosing projects were clear, in the opinion of those consulted, these were very broad. The opinion of the National Focal Points regarding the service provided by the Agency is good. Likewise, they confirm that they were properly informed about the processing of applications. National applications were provided on time by the Agency (see Chart 5).



Chart 4: National Focal Points Assessment of the service provided by the Agency (in %)

The support received from the Agency was considered to be very good. The only aspect they made reference to was the lack of time, especially where certain National Focal Points, particularly the Spanish National Focal Point, were concerned. In this instance, the Agency provided the Spanish National Focal Point with special support7. Information provided by and assistance from the Agency to the groups of experts From the interviews held, it can be deduced that all the information provided by the Agency was clear. The high degree of co-operation and availability of the Agency was highly appreciated. The only aspect that might be highlighted in this respect was the lack of time.

7 The Agency provided additional support to Spanish assessors by providing a written assessment of strong and weak points of each application.

26.7

40

33.3

0 0

20

60

20

0 0

14.3

50

35.7

0 00

10

20

30

40

50

60

Organising the SelectionProcess

Processing Applications Organising the SelectionCommittee Meeting




2.3- MANAGEMENT AND MONITORING OF THE FUNDING SCHEME

2.3.1- Monitoring and implementation of co-funded projects and financial audits The aim of this Chapter is to assess the management and monitoring of the Funding Scheme by the Agency. The monitoring system, based on the Interim and Final Activity Reports, permitted the Agency to determine the level of progress and results of the projects implemented within the Funding Scheme. It should be indicated that Interim Reports only affected those projects with grants in excess of €100,000. Interim Reports, once approved by the Agency, permitted second payments to be made. Of all the co-funded projects, only 13 transnational projects received a grant in excess of €100,000. This implies that 75% of the projects did not have to present an Interim Report. Lack of monitoring (using Interim Reports) was identified in respect of all national projects. The Final Activity Report had to be completed by all project holders. This report consisted of four sections8 providing the most relevant information about the project, in addition to being a preliminary assessment of the same. Agency approval of this Report marked the end of the project and led, therefore, to final payment being made. 2.3.1.1- Agency’s performance when accompanying and monitoring the projects The opinion of project holders with regard to the work done by the Agency to monitor their projects was good. This is illustrated in Chart 4. Those project holders whose appraisal was more negative pointed out the need for a greater degree of project supervision and for the Agency providing greater feedback on Final Activity Reports.

8 Part I: General data, Part II: Detailed Activity Report (evaluation of project results), Part III: Brief presentation for the Agency’s website (short description of the project and main outputs), Part IV: Additional Comments, Part V: Signature.



Chart 5: Agency’s Performance when Accompanying and Monitoring Projects (in %)

As a fundamental part of the monitoring process, the Agency has to ensure that the results are disseminated in a satisfactory manner, once projects have been completed. 2.3.1.2- Assessment of the Interim and Final Activity Reports In general, the required contents in the Interim and Final Activity Reports enabled projects to be conducted in a satisfactory manner. One aspect that might be recommendable would be to add a duly quantified progress indicator sample in relation to project results to these Reports. Project Holders had no problem understanding the information requested for Interim and Final Reports. Project holders responded to these Reports as a result of their own monitoring system. Half the project holders stated that the deadline for submitting the Interim and Final Reports was sufficient, whereas 48% stated the opposite. Analysis of the information contained in the Final Activity Reports, questionnaires and interviews highlights a very high percentage of project holders who consider that they had encountered difficulties in delivering the Final Activity Report before the deadline. The Evaluation Team considers that it is very difficult to produce a Final Activity Report before the end of the period of time allocated to a project. It would be preferable for this to be presented one or two months later, once activities included in the project have been completed. Some project holders stated there was a delay in signing contracts and the fact that the delivery period for Final Activity Reports was not extended. The time elapsed between the

28.3

36.4

30

45.5

1512.1 13.3

0

13.3

6.1

0

5

10

15

20

25

30

35

40

45

50

5- Excellent 4- Good 3- Medium 2- Bad 1-Very Bad

Supporting Projects Monitoring Projects



signature of the contract and the delivery of the final report caused a reduction in the time available to carry out their projects. Project holders have also highlighted the difficulties they encountered in delivering their Financial Reports. After completing a project, more time is required in order to deliver the Financial Report. Most of the projects finished in the month of September and project holders had to submit their Final Activity and Financial Reports during that same month. Before beginning to implement a project, project holders should be informed directly regarding the need to comply with these deadlines. Likewise, an intermediate monitoring of all projects (via Interim Reports or direct telephone calls) would allow the Agency to advise those project holders experiencing problems while their projects were still in progress. This would also help to resolve problems in good time. Finally, we would recommend reiterating the importance of complying with these deadlines in the financial framework within which the Agency operates in the Call and the Technical Specifications documents.

2.3.2- Evaluation of the achievements of the Funding Scheme 2.3.2.1- Quality of co-funded projects with regard to the objectives of the Funding Scheme An analysis of the evaluation results at national level allows us to conclude, in general terms, that the quality of the projects with regard to the objectives of the Funding Scheme was good. A geographic balance was maintained in co-funded projects. On one hand, this proves a satisfactory measure since it prevents all projects being funded in the same country. On the other, however, it does not give priority to the quality of the projects funded. The Funding Scheme should guarantee the quality of co-funded projects. We would therefore recommend supporting the project applicant in his, or her preparation of the Application Form. Providing potential applicants with examples of good practices from already-completed projects and disseminating the results of projects implemented throughout the European Union could be two ways of guaranteeing the quality of project applications. Dissemination is a fundamental tool for allowing potential applicants to obtain a clear idea of what the Funding Scheme involves.



2.4- PROMOTION OF THE FUNDING SCHEME BY THE AGENCY Promotion of the Funding Scheme is a horizontal activity. As of the Publication of the Call, promotional activities should continue throughout the lifetime of the Scheme. Effective communication of the Funding Scheme is essential for the success of the SME Funding Scheme. The dissemination of the Funding Scheme has been carried out at three levels: 1. Promotion of the aims and the objectives of the Funding Scheme and Call 2. Information concerning the funded projects 3. Dissemination of the results of co-funded projects

2.4.1- Promotion of the aims and the objectives of the Funding Scheme and Call The Publication of the Call for Tender in the OJEC was accompanied by a public launching of the initiative, focusing on a targeted media campaign and direct mail shots to key intermediary groups. This information was available in all 11 Community languages. The Agency confected a website to disseminate information about the Funding Scheme to possible users. The Agency also developed a specific section dedicated to the SME Accident Prevention Funding Scheme on its own website. This section permitted the Agency to disseminate details of the Call, Application Form and guidelines. The web page has been the main mechanism through which project holders learnt about the Funding Scheme. Promotion of the Funding Scheme over the website should be improved, thus providing the SME Accident Prevention Funding Scheme with a more visible presence. We would also recommend including information about co-funded projects by adding a list of partners who have already participated in the Scheme. This would permit potential applicants to search for and contact partners. In addition to these mechanisms, the Agency used other systems to reinforce the dissemination of the Funding Scheme, such as Press Releases, letters to “European Business Networks”, newsletters, etc. The National Focal Points and the Agency network partners were invited to disseminate the Funding Scheme at national levels and, in the same way, the Agency participated in a large number of events and workshops. At national level, the National Focal Points played fundamental roles in order to ensure dissemination of the Funding Scheme. National Focal Points collaborated with the Agency in terms of promoting the Funding Scheme. Promotion of the Scheme varied from one National Focal Point to another and this had a direct effect on the number of applications submitted in each country. To avoid this, we would recommend establishing a minimum number of promotional activities for all National Focal Points. This is an aspect that must be improved since evaluation data shows that many project holders are unaware of its existence.



2.4.2- Information on co-funded projects Once projects had been selected, information about them was published on the website and in printed form, the main objective of which was to provide brief information about the selected projects.

2.4.3- Dissemination on the project results The Agency has made great efforts to disseminate the Funding Scheme and all parties consulted evaluated this positively. However, it is important to insist on the need to disseminate already-completed projects and to do so in several languages in order to ensure that these are disseminated as widely as possible. On the other hand, it might be necessary for the Agency to continue participating in events during which certain project holders might be invited to speak about the results of their experiences. Finally, it is important to highlight that the 2001-2002 Funding Scheme is still in a promotional stage. Project results are still being published at the very time this evaluation is still taking place, which is why it is so difficult to give a more detailed assessment regarding this matter.



3- Analysis of the Coherence of the SME 2001-2002 Accident Prevention Funding Scheme 3.1. CONTEXT FOR ANALYSING COHERENCE The analysis of the coherence of the SME 2001-2002 Accident Prevention Funding Scheme has been carried out from two different perspectives: Internal coherence: analysing to what extent activities of the Funding Scheme

contributed to achieving those objectives pursued by the Agency, and, The coherence of the Funding Scheme with other community policies: i.e. by

assessing the degree of complementarity between the activities envisaged in the Funding Scheme with other Community policies that also include activities related to Safety and Health in the Workplace.

Coherence is an instrument used to ensure that issues have been diagnosed correctly, that strategy and objectives proposed are relevant and that expected impacts are realistic, particularly whether there is incoherence/coherence between the SME Accident Prevention Funding Scheme and the Policy of the Agency, in addition to Community Policies and other similar initiatives currently under development. 3.2. INTERNAL COHERENCE OF THE FUNDING SCHEME The analysis of internal coherence aims to assess the degree to which the activities contemplated by the Funding Scheme contributed towards fulfilling the aims pursued by the Agency, in addition to the relationship between the anticipated results and these afore-mentioned activities. Article 118 of the European Union Treaty establishes that the European Commission shall promote close collaboration between Member States in social matters, particularly in order to provide protection against accidents, professional diseases and health in the workplace. In addition to which, Article 235 establishes that “If action by the Community should prove necessary to attain, in the course of the operation of the common market, that one of the objectives of the Community and this Treaty has not provided the necessary powers, the Council shall, acting unanimously on a proposal from the Commission, and after consulting the European Parliament, take the appropriate measures”. In accordance with the powers granted to the Council in Article 235 of the Treaty, the latter adopted Regulation (EC) 2062/18th July 1994, whereby the European Agency for Safety and Health in the Workplace was created. Article 2 of Regulation (EC) 2062/94, lays down that the aim of the Agency is to "provide Community bodies, the Member States and those involved in the field with the technical, scientific and economic information of use, in the field of safety and health at work.”



In accordance with this objective, the Agency will adopt annual Work Programmes after consulting the Commission and the Consultative Committee for Safety, Health and the Protection of Health in the Workplace. These annual Work Programmes shall be included within the Corporate Plans, which will be 4-year rolling programmes covering four-year periods. The analysis of internal coherence uses the Corporate Plan for 2001-2004 - Work Programme for 2001, since this was the Plan in force during the implementation of the SME 2001-2002 Funding Scheme. In order to carry out this coherence analysis, the methodology proposed by the European Commission for the assessment of socio-economic programmes was used. From this theoretical perspective, an analysis of internal coherence permits us to understand the logic of the work of the Funding Scheme under evaluation. This analysis should also enable a further analysis of how well the SME Funding Scheme adjusts to the aims of the Agency.



Table 9: Analysis of the Internal Coherence of the SME Funding Scheme General Framework

General Objective Specific Objectives Results Actions Undertaken

Corporate Plan for 2001-2004 - Work Programme for 2001

To provide Community bodies, Member States and the media with all technical, scientific and economic information that may be useful in the field of Safety and Health in the Workplace.

1. To create the principal source of safety and health information in Europe and the most comprehensive and user-friendly resource on Internet.

2. To provide active support for the formulation and implementation of safety and health policies and the organisations involved in these processes.

3. To promote the identification and sharing of information on Good Practices solutions at workplace level.

4. To promote Member State co-operation regarding the gathering of information and research, thus making the best possible use of resources.

1. Optimising Agency and network resources and co-operation

2. Extending network partnerships 3. Improving the quality of its

information services, according to the needs of the user.

4. Providing OSH information on the Net

5. Improving Web facilities 6. Developing communication

services 7. Developing a European

monitoring system for structural OSH information.

8. Collecting strategic information on specific topics.

Those included in the Work Programme for 2001

Aim of the evaluation General Aim Specific Objective Results Actions Undertaken SME Accident Prevention Funding Scheme 2001-2002

To provide active support for the formulation and implementation of safety and health policies and the organisations involved in these processes.

To collect strategic information on specific topics: Accident prevention

1. Heightened awareness of accident risks.

2. Promotion of the development and identification of Effective Good Practices.

3. Contribution to reducing the number and gravity of work-related accidents in SMEs

4. Promotion of occupational Safety and Health at Work as part of business thinking.

5. Promotion of innovative practices to prevent accidents in SMEs.

1. Training relating to the prevention of work-related accidents in SMEs

2. Information and notifications on accident prevention aimed at SMEs.

3. Provision of Effective Good Practices that reduce accident risks in SMEs.



General Aim Specific Objectives

Actions Collecting strategic

information on specific topics Accident

prevention

Heightened awareness of accident risks

To promote the development and

identification of Effective Good Practices

To contribute to reducing the number and gravity of work-related accidents in

SMEs

To promote occupational safety

and health at work as part of business

thinking

To promote innovative practices to prevent accidents in SMEs

Training related to the prevention of work-related accidents in SMEs

Information and notifications on accident prevention aimed at SMEs

Provision of Effective Good Practices that reduce accident risks in SMEs

Red indicates the link between actions and specific objectives



The internal coherence of the Funding Scheme has been analysed both vertically and horizontally. Vertically, by analysing several different objective levels. Horizontally, by assessing objectives, results and actions at different levels.

An appraisal of the above table shows that the Scheme has a high degree of internal coherence with regard to Agency objectives. The results of the activities co-funded by the SME 2001-2002 Funding Scheme contributed directly to achieving the Agency's objectives. Likewise, there was a very high degree of interrelation between the objectives of the SME Funding Scheme and the structure of the Agency's objectives. This conclusion is reinforced by the following comments: Firstly, the principal aim of the Corporate Plan for 2001-2004 is to meet the objectives

for which the Agency was created according to Regulation (EC) 2062/94. This objective has been implemented through different lines of activity, including the compilation and analysis of strategic information on safety and health in the workplace.

Secondly, the Scheme intends to facilitate the development and dissemination of Effective Good Practices that comply with Community and national legislation in force, creating information that will be useful in the field of OSH.

Lastly, the actions implemented by the Scheme intend to achieve a number of specific objectives (creating enhanced or heightened awareness of accident risks, promoting occupational safety and health at work as part of business thinking, etc.), which have a direct effect on the possibility of achieving the general objectives of the Agency.

3.3. COHERENCE OF THE FUNDING SCHEME REGARDING OTHER COMMUNITY POLICIES The analysis of the coherence of the Funding Scheme with other Community policies had two basic aims: To assess the degree of complementarily between the activities included within the

Funding Scheme and other actions relating to safety and health in the workplace in other Community policies.

To assess to what extent the activities co-financed by the Funding Scheme were in harmony with the remainder of Community policies.

European Community institutions and bodies carry out a wide range of activities in the field of Safety and Health in the Workplace. Besides the Agency, other Directorate Generals (DGs) and institutions take part in similar programs within the field of Safety and Health in the Workplace. This analysis aims to identify the activities performed by other Community bodies and institutions in the area of Safety and Health in the Workplace and analyse the extent to which these activities complement or overlap those included in the SME Funding Scheme.



COMMUNITY POLICIES LINKED TO THE FIELD OF SAFETY AND HEALTH IN THE WORKPLACE

Community Policy Improvement in living and working conditions

Health

Employment and Social Issues

Education and Culture

Business

Regional Development

Research

Public Contracts Others (transport, the environment, civil protection, agriculture and fisheries)

Herein follows an analysis of each of the Directorates General or other bodies that carry out activities in the field of Safety and Health in the Workplace: The European Foundation for Improvement of Living and Working Conditions,

based in Dublin, is a European Agency whose aim is to “contribute to the planning and establishment of better living and working conditions through actions designed to increase and disseminate knowledge likely to assist this development”.

In the field of Safety and Health in the Workplace, the Foundation carries out a series of activities relating to the issue of working conditions, the aim of which is to obtain information in order to assist in the formulation of Community Policy in this particular area. In this way, among the many other measures it carries out, the Foundation conducts surveys into working conditions in different sectors, updates indicators on the quality work and employment, runs an observatory on working conditions, etc. The activity developed by the Foundation represents a major source of information for the Agency and other Community bodies and institutions. In this sense, its activity complements the work of the Agency extremely well, providing valuable information for the planning and development of the Agency's own activities.

The Health and Consumer Protection Directorate General is responsible, among

other activities, for carrying out Programmes of Community Action in the Public Health field. The aim of these programmes is, in general terms, to protect and improve human health through a number of activities designed to improve the level of information and knowledge about these issues.



In order to develop these programmes in an efficient manner, these should be carried out together with other Community strategies and actions, including those relating to Safety and Health in the Workplace. Community policy in matters of Safety and Health in the Workplace is of special interest for public health in order to achieve a high level of health protection. This involves a preventive approach oriented towards offering protection against hazards in the workplace, accidents at work and professional diseases. In this way, the approach of the Directorate General for Health in issues of Safety and Health in the Workplace coincides with the viewpoint of the Agency and contributes to providing a more comprehensive and broader view of the policy of Safety and Health in the Workplace.

The Employment and Social Affairs Directorate General of the European

Commission is responsible for developing and monitoring the European Employment Strategy (EES), which promotes co-operation between Member States in the field of Employment by supporting and complementing their activities and ensuring that employment is taken into consideration in all Community policies. Likewise, it is responsible for the efficient application of labour legislation (Safety and Health in the Workplace). The EES is a co-ordination system whereby a common "cast" or “template pattern” for all Member States is established within which the latter must develop their activities. The EES also includes the aspect of Safety and Health in the Workplace, giving this issue importance in promoting job quality and making full use of the production potential of the European Union.

One of the basic aims of the EES is to promote the capacity of both workers and companies in order to adapt to changes in their situations. One of the guidelines of the EES is precisely to modernise the organisation of work. In accordance with this guideline, Member States will try to guarantee a more efficient application in the workplace of existing legislation on Health and Safety. To do this, they will strengthen the control on the application of the aforementioned legislation, provide guidance for companies (especially SMEs) to comply with existing legislation, improve training in the field of safety and health in the workplace and provide measures to reduce accidents at work and professional diseases in traditionally high-risk sectors. In this respect, the new Communication from the Commission entitled, "The Future of the European Employment Strategy (EES): a Strategy for Full Employment and Better Jobs for All"9 also pays special attention to the issue of Safety and Health in the Workplace. Two of the overall objectives set out in this document, to be pursued by the EES, are quality and productivity at work, including improvements in Safety and Health in the Workplace. In this sense, it should be remembered that the application of the EES is governed at a national level (or regional, depending upon the Member State) through National Action Plans and the EU is empowered only to perform tasks of co-ordination, monitoring and evaluation. Thus, the measures put forward in the EES do not overlap the activities

9 COM (2003) 6.



included within the SME Funding Scheme, but rather represent a reinforcement and support for these from the Central Administration of Member States.

The Directorate General of Education and Culture of the European Commission

runs a series of programmes in the field of Education, among which the Leonardo da Vinci Programme is of special relevance thanks to its potential connection with Safety and Health in the Workplace. The aim of this programme is to provide support for transnational co-operation projects in the field of vocational training and to act as a link between the world of education and the labour environment. Efforts to promote adaptability and business spirit represent one of the priority issues of this programme. In this respect, the aim of the programme is to promote investment in human resources as business strategies in order to develop the capacity of companies in adapting to the changes brought on by technological progress.

According to this programme, training activities can be financed in aspects relating to Safety and Health in the Workplace. Nevertheless, its aim of bringing education closer to the labour environment places this programme in a different context to that of the SME Funding Scheme. Thus, the Leonardo da Vinci Programme complements the previous programme, since it contributes fundamentally to improving professional training in aspects concerning Safety and Health in the Workplace. In this respect, it should be mentioned that the new Strategy of the Commission "Adapting to Change in Work and Society: a New Community Strategy on Health and Safety at Work”10

places special emphasis on the importance of the Education sector as a tool for preventing accidents at work, in both schools and vocational training centres.

The objective of the Directorate General Enterprise (DG Enterprise) of the European

Commission is to favour company competitiveness by encouraging innovation and business spirit and facilitating their access to markets. In its activities, the DG Enterprise pursues a model of sustainable development maintaining a balance between economic, environmental and social aspects. Specifically, one of the aims of the Pluri-annual programme in favour of business and the business spirit, particularly regarding small and medium-sized enterprises (2001-2005)11, is to promote an increase in company competitiveness using steps that take sustainability concepts into consideration.

The important work performed by the Euro Info Centres in providing counselling and assistance to companies on Community matters should also be highlighted.

DG Enterprise pursues its activities in accordance with an economic development model that takes into consideration aspects such as Safety and Health in the Workplace. Thus, the activities included in the SME Funding Scheme are

10 COM(2002) 118. 11 Council Decision dated the 20th December 2000 concerning the Pluri-annual Programme in favour of business and the business spirit, particularly for small and medium-sized enterprises (SMEs) (2001-2005).



complemented and supported by the respect and importance that the DG Enterprise attaches to aspects of a more social nature.

The Regional Policy Directorate General is responsible for planning and executing

the European Regional Development Fund (ERDF). The general aim of this Fund is to promote economic and social cohesion by reducing major regional imbalances and participating in regional development and restructuring. In accordance with this objective, the ERDF proposes a series of measures to support the activities developed by SMEs including, among others, financial assistance for investments and subsidising those services provided to the latter.

Nevertheless, though measures taken to support SMEs are implemented according to an overall approach including both environmental and social aspects, those activities co-financed by the SME Funding Scheme do not overlap the activities of the ERDF, but rather complement these, allowing for a much broader approach in the measures taken by the EU to support SMEs.

The Directorate General for Research (DG Research), responsible for planning and

executing the Framework Programmes for Research, Technological Development and Demonstration. FP5 (1998-2002), was conceived to help solve problems and respond to the major socio-economic challenges facing the EU Currently, the VI Framework Programme (FP6) covers Community activities in the fields of Research, Technological Development and Demonstration (RTD) for the period 2002 to 2006. These programmes have two specific objectives: reinforcing scientific and technological foundations of the industrial sector and favouring international competitiveness by means of supporting research activities. This programme allows for the direct participation of SMEs through specific actions for SMEs. Likewise, from the point-of-view of Safety and Health in the Workplace, the importance of this programme is that it allows for a greater level of knowledge with regard to those hazards associated with handling substances and materials, production processes, research, etc., all of which are factors of significant relevance in the field of prevention. In this way, the DG Research acts in a different manner to that of the SMEs Funding Scheme in the field of Safety and Health in the Workplace. Likewise, in order to avoid any possible overlapping of schemes oriented towards SMEs and the subject of Safety and Health in the Workplace, the Agency has adopted a most appropriate decision regarding its planning of the Funding Scheme by excluding research projects.

With regard to the policy of public contracts, it should be pointed out that the

execution of public contracts awarded by means of tender must respect all regulations regarding Safety and Health in the Workplace. In this sense, offers presented by bidders that do not respect the obligations in matters of Safety and Health in the Workplace indicated by the contracting body in the Sets of Conditions may be considered to be non-conforming and, therefore, be rejected.



The “Communication of the Commission on Community Legislation Corresponding to Public Contracts and the Possibility of Including Social Aspects in These Contracts”12 intends to clarify the range of possibilities offered by the legal framework of the Community by encouraging social aspects in public contracts. The authority awarding a contract must choose the best offer and respect a number of rules, such as the selection of candidates in accordance with objective requirements and awarding the contract to the most economically advantageous offer, including a series of objective criteria. In this way, Technical Specifications or selection criteria referring to aspects of Safety and Health in the Workplace such as protecting the health and safety of those working on building sites, labelling and storing hazardous or dangerous products, etc., may be included. The public contract policy contributes, therefore, to achieving those objectives established by the SME Funding Scheme to the extent that this allows better compliance with existing regulations regarding safety and health in the workplace and the possibility of requiring that bidders observe specific aspects in this respect.

There are other Community policies that implement measures to protect workers.

These include, among others, policies relating to transport, the environment, civil protection, agriculture and fisheries. The preventive approach of these policies establishes one of the starting points upon which the policy of Safety and Health in the Workplace of the SME Funding Scheme has been developed. In this sense, the SME Funding Scheme complements the activities carried out by these Community policies.

Analysis shows that there are other Community institutions and bodies which carry out other policies different to the one developed by the Agency, but which also address issues concerning Safety and Health in the Workplace. However, the SME Funding Scheme has its own characteristics and objectives that clearly differentiate it from other instruments developed by other Community policies. Thus, it is fair to point out that the SME Funding Scheme does not overlap other Community programmes and activities, but rather complements these. In order to draw this conclusion, the Evaluation Team carried out documentary research and analysis of those Community programmes focusing on SMEs and on Health and Safety in the Workplace. This analysis was complemented by several telephone calls to the different Directorate Generals, whose activities have been described above. The Evaluation Team contacted the key person responsible for programmes dealing with these two key issues in order to confirm that no overlapping occurs between the SME Accident Prevention Funding Scheme and other Community policies. The result of this analysis is summarised in the following diagram.

12 COM(2001) 566.



Likewise it would be advisable to establish a co-ordination mechanism to facilitate the complementarity between different Community institutions in matters dealing with OSH, thereby avoiding overlaps and activities with similar contents.



Organism Main Objectives Activities Relating to Accident Prevention and Safety and

Health at Workplace

Possible Overlap with the Funding Scheme

The European Foundation for Improvement of Living and

Working Conditions ➨ To contribute to the planning and establishment of better living and working conditions ➨ To obtain information to assist in the

formulation of Community Policy in this particular area ➨

NO OVERLAP Research and development projects providing data and analysis for informing and

supporting the formulation of EU policy on working and living

conditions

Health and Consumer Protection Directorate General

European Commission ➨ To promote a better quality of life by ensuring a high level of protection of consumers' health, safety and

economic interests as well as public health. It covers the following policy areas: Consumer Policy, Public

Health and Food Safety, Animal Health, Animal Welfare and Plant Health

➨ They do not deal with this issue.

They only have data related to Home and Leisure accidents and they carry out very few prevention actions. Their

main task is data collection.

➨

NO OVERLAP

Employment and Social Affairs Directorate General

European Commission ➨ To contributing to the development of a modern, innovative and sustainable European social model with more and better jobs in an inclusive society based on equal opportunities

➨

They deal with the modernisation of labour relations by making EU workers

more mobile. The EES pays special attention to the

issue of Safety and Health in the Workplace, but is governed at national

(or regional, according to Member States) level through National Action Plans. The EU is empowered only to

perform tasks of co-ordination, monitoring and evaluation.

➨

NO OVERLAP The measures put forward in the EES do not overlap the activities

included within the SME Funding Scheme, but propose reinforcement and support for

these from the central administration of Member States

Education and Culture Directorate General

European Commission ➨ To bring people together, fostering respect and

understanding To create a European knowledge area To support the development of cultural and audio-

visual sectors

➨ Improvement of Vocational Training. Action Programmes are focused on

training for enterprises and they do not concern Health or Accident Prevention.

➨NO OVERLAP

Enterprise Directorate General European Commission ➨

To address the entire business environment in order to enable enterprises to strengthen their competitiveness, grow and develop in a way that is compatible with the

overall EU goal of sustainable development. ➨ They deal with the new approach of EU

Directives concerning product safety. ➨NO OVERLAP

Regional Policy Directorate General

European Commission ➨ Economic and Social Cohesion ➨ They do not deal with issues concerning health and safety ➨

NO OVERLAP

Research Directorate General European Commission ➨ Scientific Research and Technological Development in

the European Union ➨

Genetic Research and Health Development.

They deal with research projects and such projects are excluded from the

SME Funding Scheme.

➨NO OVERLAP



4- Results and Impacts of the Funding Scheme 4.1- SELECTED PROJECTS: LEVEL OF ORGANISATION AND CATEGORIES A national project refers to one, or several, applicants belonging to a single Member State whereas a transnational project concerns at least two applicants located in different Member States. National projects were entitled to opt for a maximum of 60% of co-funding from the Agency, as compared to 80% in the case of transnational projects. The SME 2001-2002 Funding Scheme co-funded 51 projects, most of which were national as shown in the following table: Table 10: Total Co-funded Projects Total Projects 2001-2002

Funding Scheme % total

National Projects 35 68.6 Transnational Projects

16 31.3

TOTAL 51 100 The co-funding of national projects was less than €100,000 with sums varying from €25,000 to €90,000. The 35 national projects received a total sum of €2,211,674. The co-funding for transnational projects was greater. The smallest project received €64,000 compared to the €190,000 received by the largest. The Agency allocated €2,294,000 to 16 transnational projects. The Agency allocated 4.5 Million EUR to co-funding projects within the framework of the 2001-2002 Funding Scheme and distributed its budget for national and transnational projects into two practically equal parts. Transnational projects required larger budgets in order to carry out their activities (greater co-ordination efforts, higher travelling costs, higher investments in languages, greater effort involved in dissemination) and more time, especially when disseminating the results. Most of the co-funded projects were in the Information and Communication Category (49%), followed by those for Provision of Good Practices (31.3%), and, finally, Training (19.7%). The figures for co-funded projects at both national and transnational levels are detailed in the following table: Table 11: Co-funded Project Categories National

Projects

% total

Transnational Projects

% total

TOTAL Projects 2001-2002 Funding

Scheme

% total

Training 8 22.8 2 12.5 10 19.6 Information and Communication

15 42.9 10 62.5 25 49

Provision of Good Practices

12 34.3 4 25 16 31.3

TOTAL 35 100 16 100 51 100



Chart 6 shows the projects co-funded in each Member State. Nationally, the typology of projects was more geographically balanced than at transnational level. With the exception of Luxembourg, every country received at least one national project. The highest number of projects came from Spain (5 projects, or 14.3% of the total), followed by Italy (4 projects, or 11.3% of the total). At a transnational level, the country that received the highest number of projects by far was Belgium (25% of the total), followed by Germany, Spain and Finland (13% of the total). The remaining countries received one project each (6.3% of the total). The European dimension of the institutions presenting projects, as well as their experience in participating in European programmes given their proximity to Headquarters of European institutions, could be among the reasons that justify the high participation of Belgium in transnational projects.

Chart 6: Category of Projects

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

AT BE DE DK ES FI FR GR IE IT LU NL PT SE UK

National Trans-national

4.2- THE OUTCOME OF THE PROJECTS An outcome may be considered as the result obtained at the end of an intervention. Outcomes can be observed when an action has been completed and accounts for the way in which allocated funds were spent and managed. This paragraph deals with project outcomes, which were assessed taking the following into account:

The relevance in relation to OSH priorities The performance of project holders: Effectiveness and Efficiency The Call for Project Proposals priorities



4.2.1- Relevance of the Funding Scheme in relation to OSH needs and Priorities In order to assess the relevance of the Scheme in relation to OSH needs and priorities, the Evaluation Team took the qualitative data resulting from the fieldwork (responses from questionnaires and interviews) into consideration, in addition to the data available in the Application Forms and Final Activity Reports. In the questionnaire, project holders were asked to score or rate the degree of achievement of the objectives of the Funding Scheme using a points scale from 1 to 5. As the following chart shows, the results of the projects achieved the objectives of the Funding Scheme, resulted in heightened awareness of accident hazards and promoted the development and identification of Effective Good Practices. Project holders appraised the objectives of the Funding Scheme very favourably and, in the majority of cases, the score given was 4 (good).

Chart 7: Project Holders Assessment on the Funding Scheme in Relation to OSH Priorities (in %)

Likewise, some project holders highlighted the need to foster the creation of networks with SMEs from different countries in order to facilitate exchanges in relation to OSH needs and

39

4 613

03

364 1

210

2

1656

270

2

1851

227

2

124 4

339

2

943

373

9

530

600

5

0

10

20

30

40

50

60

Hei

ghte

naw

aren

ess

ofac

cide

nt ri

sks

Dev

elop

men

t and

the

iden

tific

atio

n of

Goo

d Pr

actic

es

Prom

ote

OSH

A at

wor

k as

par

t of

busi

ness

thin

king

Prom

ote

inno

vativ

epr

actic

es to

pre

vent

risks

Prom

ote

wor

kfor

cein

volv

emen

t in

OSH

A

Prom

ote

soci

alag

ents

invo

lvem

ent

in O

SHA

Oth

er n

eeds

of

SME




priorities. Dissemination of project results by providing the name of someone who might be contacted was considered essential. National Focal Points were consulted to obtain their assessment of the Funding Scheme with regard to the needs of SMEs and their priorities in relation to OSH within different countries. In their opinion, the Scheme contributed very positively to: The development and identification of Good Practices The development of risk evaluation and prevention practices The development of sustainable and European added value on OSH activities.

Chart 8: National Focal Points Assessment on the Funding Scheme in Relation to SME Needs (in %)

The objectives of the Funding Scheme responded to the needs and priorities of SMEs with regard to OSH activities and the results of the projects made a positive contribution to tackling work-related accidents in EU SMEs.

2740

277

07

6720

07

1353

330 0

747

3313

0

1333

477

0

747

2713

7

0

10

20

30

40

50

60

70

Hei

ghte

naw

aren

ess

ofac

cide

nt ri

sks

Dev

elop

men

t and

the

iden

tific

atio

n of

Goo

d Pr

actic

es

Dev

elop

men

t of

risk

eval

uatio

n an

dpr

even

tion

prac

tice

Sust

aina

ble

and

Euro

pean

add

edva

lue

on O

SHA

Red

uctio

n of

wor

k-re

late

d ac

cide

nts

inSM

E

Prom

ote

OSH

A at

wor

k as

par

t of

busi

ness

thin

king




4.2.2- Performance of project holders. Effectiveness and efficiency of projects

4.2.2.1- Performance of project holders from the point-of-view of objectives Projects were implemented according to the agreements signed to that purpose. The objectives of the projects were largely achieved in broad terms by most of the project holders. These results are shown in the following chart:

Chart 9: Achievement of Project Objectives

The performance of project holders was very good. Likewise, the analysis carried out by the Evaluation Team permits us to conclude that: Since the majority of projects were oriented towards SMEs or micro-enterprises,

project holders highlighted the great difficulties they encountered in getting them to take part in their projects. SMEs were extremely reluctant to show the results of their work and make them public.

Analysis of the Application Forms and Final Activity Reports permit us to conclude that some project holders were very ambitious when establishing objectives for their projects, which, in certain sectors, were very difficult to meet. This may be due to a lack of experience of project holders in projects of this nature.

Lack of time was one of the biggest difficulties project holders had to face, something that reduced project efficiency. A broader time span could result in better opportunities for developing higher quality projects. The Funding Scheme would have to be extended over several years in order for activities to benefit from this support. This would ensure sustainability of the activities contemplated by it.

Many project holders were faced with similar difficulties, such as projects involving construction work, due to the fact that this sector is inactive at certain periods of the

62.20%

28.90%

8.90%

0

0,00%

10,00%

20,00%

30,00%

40,00%

50,00%

60,00%

70,00%

4-largely reached 3-reached 2-almost reached 1-not reached

% Achivement of Project Objectives



year (Christmas and the summer months of July and August, depending on the country). This reduced the implementation period of their projects by an average of two months, making training courses and the dissemination of activities more difficult to organise.

Broadly speaking, project holders expressed great satisfaction at having taken part in the 2001-2002 Funding Scheme and many of them attempted to repeat the experience on the 2002-2003 Second Call for Funding Scheme. Bearing in mind that it was the first time an initiative of this kind had been launched by the Agency, these results are extremely positive. Neither project holders nor the Agency had had prior experience in the organisation or had participated in previous Schemes.

4.2.2.2- Effectiveness of the Projects The aim of this section is to analyse the effectiveness with which the Funding Scheme was carried out, based on the results obtained in real terms. The concept of effectiveness refers to the consistency between the anticipated or expected objectives and the real results obtained from carrying the Scheme out. Nevertheless, it was impossible to make this analysis using the usual approach due to the fact that, during the planning and programming stage of the Funding Scheme, no definition or quantification was made of a number of indicators that would have allowed the effectiveness of the Scheme to be determined once the co-funded projects had been completed. In spite of this, the effectiveness of the Scheme has been assessed based on the results obtained according to a number of indicators defined by the Evaluation Team in agreement with the Agency. In order to draw up the list of indicators, the Evaluation Team took the following documents into consideration: Application Forms. Final Activity Reports. Questionnaires.

The quantification of the indicators, established on the basis of the aforementioned documents, was as follows:



Table 12 : Indicators Quantitative Indicators Concept National Transnational TOTAL Common Indicators Total Publications carried out on Accident Prevention at Work in SMEs

158 105 263

Information actions (seminars, meetings, etc.) held on Accident Prevention in SMEs

315 160 475

Project holders who disseminated the project 35 16 50 SMEs benefiting from the SME 2001-2002 Accident Prevention Funding Scheme

37,055 502,349 539,404

Sustainable Projects on Accident Prevention in SMEs 35 16 51 Projects in which the workforce participated effectively 25 14 39 Projects designed for SME owners/managers focusing on motivation for preventing accident risks

22 13 35

Total number of partners 78 79 157 Training and Education SMEs for which the training courses on accident prevention were intended

1.265 114 1.379

SMEs Managers/owners for which the training courses on accident prevention were intended

18 0 18

SMEs safety officers for whom the training courses on accident prevention were intended

13 0 13

SMEs workers for whom the training courses on accident prevention were intended

354 1 355

Total number of trainees 3,545 184 3,729 Total duration of training courses (hours) 14,625 736 15,361 Information and Communication Implemented projects published on Websites apart from the Agency Website

7 8 15

Total information activities 108 75 183 Number of copies 1,918,535 20,754 1,939,289 Development of Good Practices Good Practices resulting in an effective reduction of accident risks in SMEs

204 131 335

Projects aiming at encouraging the exchange of Good Practices 13 5 18



Qualitative Indicators 13 Concept Degree

Degree of accomplishment of the Funding Scheme objectives by the project holders 62 % of project

holders (majority)

Clarity of the Call for Project Proposals 93.5 % of project holders

Clarity of the Application Form 84.8 % of project holders

Clarity of the Technical Specifications 84.8 % of project holders

Efficiency of the Agency promoting the Funding Scheme 100%

Project holders who heard about the Scheme from the Agency (directly/ Website, OJEC) 50 % of project holders

Suitability of the Application Form submission deadline 95.7 % of

satisfied project holders

Quality of the service provided by the Agency during preparation of the Application Form

52 % of project holders

considered it excellent

Analysis of the indicators shows that the results obtained by means of co-funded projects under the Funding Scheme were very positive. The main results of this Scheme included more than 250 publications, almost 500 actions taken to disseminate information (seminars, meetings, etc.) and more than 500,00 SMEs who benefited indirectly as a result of the application of the Funding Scheme. Project holders were also questioned about whether they would have carried out their projects, had they not have received funding from the Agency. Most of the project holders (58%) declared that they would have carried out their projects without any funding whatsoever from the Agency (see Chart 10).

13 From an Interpretation of the questionnaires



Chart 10: Project Holders who Implemented Projects with/without Funding from the Agency (in %)

This illustrates the enormous interest of project holders in their projects. They did not design their projects solely because of the Funding Scheme, but rather because they truly had a great deal of motivation and interest in completing their projects. In the opinion of the Evaluation Team, motivation was a key factor in the positive results achieved by projects and of the Funding Scheme in general. Though most project holders stated that they would have carried out the project without having received funding from the Agency, they also declared that, without help from the Agency, the results would have been neither so efficient, nor effective. It would have proven impossible, for instance, for them to have disseminated the results so widely. Projects would not have been so sustainable and they would not have been able to transfer the results at the same level. Co-funding from the Agency, therefore, proved a fundamental aspect of project effectiveness. The great interest of project holders in carrying out their projects, requirements of the Call for granting projects and the co-funding of the Agency endowed the projects with a number of characteristics, such as: partnership, sustainability of projects, transferability, dissemination of results, quality of publications, etc. All these would have been impossible to achieve without Agency funding. Project holders displayed a high level of professionalism and were even successful in exceeding anticipated objectives in such exacting sectors as those in which a very high number of accidents unfortunately still continue to occur.

57.8

42.2

Yes No



4.2.2.3. Efficiency of projects The aim of this section is to analyse the efficiency with which the Funding Scheme was carried out. The efficiency concept refers to the consistency between the financial resources programmed in order to achieve the established objectives and those actually used. From analysis of the financial resources used by the projects and the degree to which the established objectives were fulfilled, it might be concluded that projects exhibited a high level of efficiency. This statement is based on the following points: Most of the objectives established in each project were achieved. There were no significant deviations from the budget allocated for projects.

The excellent results reached in terms of efficiency were obtained, at least in part, thanks to the severe criteria applied by most of the project holders when budgeting their projects. Although additional resources were used in some cases, these were normally employed to reinforce activities contemplated by the project and even to carry out activities not initially included in the Application Forms.

4.2.3- Outcome of the projects in relation to Call Priorities With regard to the outcome of the projects in relation to Call priorities, the Evaluation Team has analysed the following aspects: Partnership Participation of workforce and social partners Approach of co-funded projects: SMEs or micro-businesses Sectors in which projects were focused

4.2.3.1- Partnership projects (involving several organisations) Projects submitted within the framework of the Funding Scheme were presented in partnership at national and transnational levels. As specified in the Call, priority was given to those projects involving several organisations. The results of the evaluation show that the average number of partners per project was three. Transnational projects had an average of five partners and national projects an average of two. The appraisal of work performed in partnership was excellent, as reflected in Chart 11. Project holders stated that transnational partnerships allowed them to learn about OSH situations in other countries and organisations. This was highly valued by interviewed project holders.



Chart 11: Project Holders Assessment of the Partnership

4.2.3.2- The participation of the workforce and social partners The Funding Scheme tried to enhance the level of collaboration/co-operation in all those projects which received funding. Depending upon each particular project, the workforce and social partners intervened directly in the planning of the same. Some of the projects arose as a result of the initiatives of the workforce or social partners. Evaluation has shown that the workforce and social partners were essential for choosing activities to enhance safety and to identify priority areas for training activities. The workforce was the target for training courses, the main characters in Good Practices, the individualisation of high-risk work and in the performance of accident tests in the workplace. Project holders drew special attention to the fact that the workforce and social partners provided continuous and constant support at each and every phase of the project, contributing with their practical knowledge of the sector and putting forward their own ideas. Their contribution was fundamental in highlighting the following needs: Taking part in the management of accident prevention Having all necessary information available Collaborating in putting the system into practice.

Effective participation of the workforce and social partners was achieved in the Funding Scheme during the different stages of the projects:

50

40

46

32

47

38

18

1316

0 0 0 0 0 00

5

10

15

20

25

30

35

40

45

50


National Projects Transnational Projects National and transnational



The workforce and SMEs were the direct beneficiaries of the Funding Scheme. They were the recipients of training activities, participation in seminars and the receivers of information.

Social partners mainly took part in planning projects.

Chart 12: Involvement of Workforce and Social Partners (in %)

Those projects holders consulted considered the participation of the workforce and social partners extremely highly. In the case of national projects, the workforce obtained higher participation, as shown in Chart 13.

2 3

5 2

2 42 7

5 4

4

2 0

1 3

2 7

1 3

0

1 0

2 0

3 0

4 0

5 0

6 0

Plan

ning

of t

he

proj

ect

Impl

emen

tatio

n

of th

e pr

ojec

t

Rec

eivi

ng

train

ing

Rec

eivi

ng

info

rmat

ion

Parti

cipa

ting

to

Sem

inar

s

W o rk fo rc e S o c ia l p a rtn e rs



Chart 13: Project Holders Assessment of the Participation of Workforce and Social Partners (in %)

With regards to the involvement of social partners, all National Focal Points declared that social partners participated in project conception and implementation. Concerning the participation of the workforce, most National Focal Points considered that they also took part, either directly or through unions and other associations or organisations.

38.541

15.4

2.6 2.6

30

38

27.5

50

0

5

10

15

20

25

30

35

40

45

Workforce Social Partners




4.2.3.3- Focus of the projects: SME or micro-businesses Projects focusing particularly upon SMEs and micro-businesses were given priority during the selection process. Based on the evaluation of Application Forms and Final Activity Reports, it can be concluded that most of the projects focused upon SMEs, rather than micro-businesses. This is particularly clear in the case of transnational projects, 88% of which focused on SMEs. These results are shown in the following chart:

Chart 14: Focus of Projects (SMEs, Micro-businesses)

4.2.3.4- Sectors on which projects focused As stated in the Call for Project Proposals, priority was given to those projects focusing upon priority risk, high-risk or risk-oriented projects. Evaluation confirms that most of the projects addressed these priorities, particularly projects focusing on priority and high risks (building and metal sectors, harbour areas, fisheries, etc.). Most of the transnational projects focused upon the building and construction sector. With regard to national projects, industrial projects formed the largest group, followed by the building and construction sector.

29%

13%

71%

88%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Micro-business SMEs

National Transnational



4.3 SUSTAINABILITY OF PROJECTS In accordance with the MEANS14, the sustainability concept refers to “the ability of effects to last in the middle or long-term”: Effects are sustainable when they last after the funding granted by the intervention

has ceased. Effects are non sustainable if an activity is unable to generate its own resources, or it

is accompanied by negative effects, particularly on the environment, and if that leads to blockages or rejection”.

According to this definition, analysis has been made of the sustainability of the effects of the projects co-funded by the Funding Scheme. In this respect, all the projects are sustainable - i.e., projects will continue to produce positive results in OSH-related matters in the future. In general, the reasons given by project holders to explain project sustainability are as follows: With regard to training projects, the general criteria was that projects based their

sustainability on the fact that an improvement of the skills of employees and employers in OSH matters would contribute to reducing the risk of accidents in SMEs.

With regard to the information and communication projects, the reasons used to explain the sustainability of their effects was that, by disseminating activities (publications, manuals, etc.), they would continue to produce positive effects in OSH matters either through the use of the material produced for these projects or in the form of higher levels of know-how of those persons for whom the information and communication activities were designed.

With reference to the Provision of Good Practices projects, the beneficiaries based the sustainability of the effects of their projects on the fact that the dissemination of Good Practices among different associations, organisations and SMEs would be subsequently incorporated into the production and working processes of SMEs, thereby reducing the risk of accidents in this type of company.

During this analysis, it was observed that some project holders confused the terms “sustainability” and “dissemination”, tending to consider projects as being sustainable when they were disseminated in a satisfactory manner. Nevertheless, the evaluation team has used the concept of "sustainability" according to a more restrictive criterion based not so much on the sustainability of the projects, but on the sustainability of the project results. In accordance with this interpretation, it is understood that a project is sustainable when, in spite of the fact that the project has ended and the Agency has made its financial contribution, this continues to be implemented by means of recourse to other funds.

14 MEANS Collection, Volume 6. Evaluating Socio-economic Programmes: Glossary of 300 Concepts and Technical Terms. European Commission.



According to this viewpoint, almost two-thirds of the projects co-funded by the Agency were sustainable (64.6%). The remaining projects were considered as being non-sustainable (35.4%), since there is no evidence that the project will continue to be implemented once financial aid from the Agency has been depleted. The main arguments used by project holders to justify the sustainability of the projects are as follows: The regular updating of the contents of Websites regarding those projects consisting of

the creation of Websites dealing with OSH matters. The updating of training materials in accordance with current legislation so that the

material may be used in the future with full guarantees. The regular organisation of courses using training material drawn up through the

project. In this sense, attention should be drawn to the use in some projects of New Information and Communication Technologies (NICTs) using the e-learning system.

The creation of networks and work-groups dealing with OSH matters which will continue to meet on a regular basis.

The regular updating of risk prevention methodologies. The updating of databases with new information concerning Good Practices. Holding information and communication sessions on a regular basis. Maintenance of management tools linked to OSH.

Using a more restrictive criterion, the high degree of sustainability observed in the projects co-funded through the Funding Scheme should be highlighted. In this sense, most of the projects did not end with the financial contribution being received from the Agency, but continued to be "active", producing some very positive results in OSH-related matters. Another conclusion is that, in general terms, larger organisations offer higher levels of financial continuity in the project. In view of the fact that most of the projects have concluded recently, it is probably too early to make an appraisal of the real project sustainability. It will be necessary to wait another 12 or 18 months before being able to analyse project sustainability.

A stricter definition of the term “sustainability” is necessary in order to guarantee the future sustainability of projects. 4.4- Impact of the Funding Scheme The impact of the Funding Scheme was analysed according to the following aspects: In raising awareness of occupational accident risks at EU level Transferability of project results European Week (EW) and the topic on accident prevention

Given the brief period of time elapsed from the end of the projects and this evaluation being carried out, the true impact of the 2001-2002 Scheme has proven very difficult to measure (implementation of projects finished in October 2002 and evaluation started in November 2002).



4.4.1- Impact of the Funding Scheme in increasing awareness of occupational accidents risks at EU level The Funding Scheme had a positive impact on increasing or heightening awareness of occupational accident risks at EU level. Most of the project holders interviewed pointed out that projects financed within the framework of the Funding Scheme met important needs in relation to OSH (see Chart 15).

Chart 15: Project Holders Assessment of SME Needs in Relation to OSH (in %)

The 2001-2002 Funding Scheme permitted: Activities to be initiated in a field in which there was a great deal of need, such as

accident prevention in SMEs. The direct participation of SMEs and micro-businesses, thereby providing an insight

into the real situation faced by many companies. The dissemination of a great deal of information on health and safety that was not

available in certain sectors. The projects were the subject of extensive dissemination activities, conveying the prevention message to a broad cross-section of the public, as shown by the indicators.

To increase awareness regarding accident prevention, to promote training and to produce material in order to be able to continue working in the future.

The Funding Scheme could therefore be considered as being successful in improving working conditions and preventing accidents in the workplace. The following chart shows the project holders’ appraisal of the contribution of the Funding Scheme to the prevention of accidents at work.

0

33.3

55.6

6.74.4




Chart 16: Project Holders’ Assessment on the Contribution of the Funding Scheme Towards Preventing Accidents at Work (in %)

Evaluation data is sufficiently clear and consistent for us to state that the Funding Scheme has contributed in a positive manner to both heightening and increasing awareness of accident prevention at work and has, therefore, contributed to reducing work-related accidents.

4.4.2-Transferability of Projects Transferability of project results is closely related to their sustainability. Both the transferability and sustainability represented two of the evaluation selection criteria. This reasoning lead automatically to the conclusion that all co-funded projects were transferable. The interpretation of evaluation data, together with the evaluation of Final Activity Reports, permitted us to conclude that the great majority of projects were transferable. This statement can be justified by the results of the projects themselves, the majority of which were training materials, publications and websites. Nearly all project holders interviewed (99%) stated that their project outcome was easily transferable to other SMEs. Also, other organisations, which did not initially take part in their project, contacted them to request information about their results. Among the several factors that might make transferring the results of projects more complicated, the following are of special relevance: Those results focusing upon very specific sectors. The language in which the results were disseminated might represent an obstacle for

its dissemination.

52.3

29.3

2.32.313.6




The type of training which, on occasion, might be very specific to the characteristics of the sector and country.

The OSH regulations in different countries that might also hinder dissemination. In the opinion of the National Focal Points, the results of the projects promoted in this Scheme provided an excellent opportunity for exchanging experiences and practical solutions. The National Focal Points insisted on the need to disseminate all the co-funded projects at national level on an annual basis. The projects were transferable, at least at national level. Likewise, the need for greater monitoring and supervision of projects is a basic step towards ensuring transferability.

4.4.3- European Week 2001 and the Topic on Accident Prevention The European Week (EW) is an information campaign aimed at making Europe a safe and healthy place in which to work by promoting activities reducing the number and gravity of work-related accidents. With the backing of all Member States, the European Commission and Parliament, trade unions and employer federations, it provides a unique opportunity to focus attention on the importance of safety and health in the workplace15. The first EW, aiming at preventing Muscular-skeletal Disorders (MSDs) in Practice was launched in 2000. Accordingly, EW 2001, which was oriented towards the subject of Accident Prevention, was the second campaign organised by the Agency. EW 2001 was co-ordinated by the Agency and ran in all 15 EU Member States and beyond. The Week was held in October 2001, but the campaign ran throughout 2001. Each Member State decided upon the exact week in which they wanted to organise the event within the month of October. The Agency provided Member States with budgets with which to organise these campaigns. Since the Agency is the European Union organisation responsible for Occupational Safety and Health, it launched the SME 2001-2002 Accident Prevention Funding Scheme in 2001. Bearing in mind the previous experience of the Agency in organising the EW, the Evaluation Team considers that focusing both events under the same heading of “Accident Prevention” was a very important factor as far as the launch of the first Funding Scheme was concerned. The aim of EW 2001 was to heighten the awareness of risks and enhance prevention. The EW focused on the following specific objectives: Heightening the awareness of risks and promoting actions to prevent work-related risks

throughout Europe. Promoting information and projects at workplace level. Supporting, encouraging and stimulating activities, particularly regarding the exchange

of information on Good Safety and Health Practices concerned with preventing work-related accidents.

15 The European Agency for Safety and Health



The overall objective of the Scheme is to “identify, communicate and support activities and projects with effective added-value that will motivate, develop, support and sustain effective organisation of OSH/OSH management in SMEs”. The ultimate objectives of both initiatives coincide and are in-line with the role of the Agency as a catalyst to develop, gather, analyse and disseminate information contributing to improving Safety and Health in the Workplace in Europe. In spite of this, however, both initiatives contain basic differences due to the fact that the EW is an information campaign and the Funding Scheme is a much broader initiative, which, in our opinion, provides support for the EW. Focusing both initiatives was a positive move as it helped combine forces in preventing accidents in the workplace and increased the impact of the preventive message.



4.5- OVERALL ASSESSMENT The aim of this Chapter is to evaluate the different categories of projects separately and to try to assess, as far as possible, which type of projects were more efficient. As mentioned above, most of the projects presented fell into the categories of Information and Communication and Provision of Good Practices. In spite of these classifications and irrespective of the category to which they belong, projects included Training, Dissemination and Good Practices activities. Likewise, in each project category, the Evaluation Team has only analysed those results relating to each specific category.

4.5.1- Training activities Training was the category that received the least amount of funding at both international (8 projects, representing 23% of the total) and transnational levels (2 projects, representing 12.5% of the total). This category was oriented towards a very small target public. As highlighted in the Call for Project Proposals, training activities had to focus specifically on owners and managers of SMEs, workers and safety representatives. In the light of the evaluation results, training courses carried out were oriented towards the workforce. Other collectives, such as managers and those responsible for Safety and Health, were targeted in training projects (see Chart 17).

Chart 17: Training Activities (in %)

21%23%

26%

30%

0%

5%

10%

15%

20%

25%

30%

SME Managers SME ow ners Responsible forSafety and Heath

All w orkforce

Training Activities



Training schemes were the projects that best conveyed the message of accident prevention in the workplace. Those persons taking part in training courses acquired knowledge, which they generally remembered and put into practice. As far as dissemination was concerned, training projects were much more limited than the other two categories. This could be due to the fact that: Courses were oriented towards a limited public (mass training courses are not capable

of conveying the message) and so the beneficiaries were only those who took part in this training programme directly or indirectly.

Courses were organised with reference to very specific sectors. This limited their transferability as they should have necessarily been adapted for use in other sectors.

Courses must be given in the language of the country in which they are organised. Translation would be needed to disseminate them in other countries. Far better results were achieved at national levels than transnational levels.

The specific regulations of each country had to be taken into consideration when designing the courses.

At a national level, the Evaluation Team identified training activities co-funded by European Social Fund (ESF) as being similar to ones being imparted by the Funding Scheme in some countries. Therefore, the training category is not, in this respect, as effective as the other two categories. With regard to the sustainability of the projects, a training course may be considered sustainable when the materials used are kept up-to-date or when new technologies are used for this purpose, as is the case of e-learning. Improving the training of employees and employers in OSH matters will contribute to reducing the risk of accidents in SMEs. The Training projects of the Funding Scheme were highly sustainable, evidence of which is that: They created new requirements in the organisations (specific line of training in work

accidents) They reached collaboration agreements with training institutions to carry out further

training The course materials were translated into other languages (thereby increasing their

transferability and, consequently, their sustainability) The people who received training were monitored in several instances

To resume, training projects were the ones that offered the least opportunities for dissemination and transferability, at least if dissemination across the EU is considered a priority. Sustainability would be guaranteed provided these courses are updated. 4.5.2-Information and Communication The Information and Communication projects were the most numerous of the three categories, at both national (14 projects - 40% of the total) and transnational level (9 projects - 56% of the total).



The target public towards which the projects in this category were oriented was the largest of the three. As indicated in the Call for Project Proposals, Information and Communication activities should improve access, encourage participation and provide information on OSH activities. Bearing in mind the general objective of the Funding Scheme, it is considered that this category matches its approach perfectly. Most of the Information and Communication activities focused on improving access to OSH information and encouraging the participation of workers in OSH practices, as shown in the following chart.

Chart 18: Information and Communication Activities

This category of projects was the largest of the three, especially at transnational level. This was due to the fact that the projects in this category were the simplest to plan and implement, particularly bearing in mind that the majority of project holders were public bodies or social partners. From a dissemination point-of-view, Information and Communication Projects achieved the best results as they were funded precisely in order to achieve this end. Moreover, most of the results of these projects were obtained in several languages, something that automatically facilitates their dissemination. In short, these were the projects that had the greatest impact and were the most efficient at an EU level. The scale of the projects oriented towards Information and Communication varied a great deal, ranging from those reaching the general public and those in which information was almost personalised.

80% 78%

69%

39%

0%

10%

20%

30%

40%

50%

60%

70%

80%

To improve accessto OSHA

Information

To encourage theparticipation of

w orkers in OSHApractices

Communicating theOSHA message

Others

Information and Communication activities



The level of dissemination achieved by transnational projects was higher than national ones. European bodies carried out many transnational projects with partners in almost all members of the EU This ensured their dissemination at both a European and national level by each member of the association. Information and Communication Projects were the subject of a high level of dissemination. The website, CD-Rom, publications and events were the main instruments used to disseminate these projects. These tools can be transferred easily at both national and transnational levels, the main impediment being the different languages involved. From the point of view of sustainability, materials should be checked to ensure they have indeed been updated. This project category was the one that permitted the largest number of beneficiaries to be reached due to the fact that, apart from occasional events, the website is an instrument permitting unlimited dissemination at very low cost. These were the most efficient projects for the Agency, especially considering those projects carried out at transnational levels. Furthermore, all projects, irrespective of their orientation towards any specific category, had to disseminate their results. Thus, Information and Communication was an implicit feature of all funded projects. 4.5.3- Provision of Good Practices The Provision of Good Practices category was the second largest category regarding the number of projects presented at both national (13 projects, representing 37% of the total) and transnational levels (5 projects, representing 31% of the total). Analysis of the available information lead us to conclude that project holders had difficulty in identifying what was understood as Good Practices. As defined in the Call for Project Proposals, Good practices should provide "added value to the prevention of work-related accidents”. But Good Practices depend upon a specific sector and on those SMEs in which they are analysed. In this sense, Good Practices require external assessment that would permit a decision to be made regarding whether something was a Good Practice, or not. In accordance with the definition of a Good Practice, all the projects carried out under the Funding Scheme could be considered as being Good Practices since their aim was to provide useful results that would contribute towards reducing accidents in workplaces. Projects oriented towards Good Practices encountered a fairly widespread problem in their implementation, namely the difficulty of identifying what a Good Practice was in SMEs, due mainly to the fact that companies were very reluctant to reveal how they work. Also, what may be a good practice in one company may certainly not be a good practice in another, a fact that could result in difficulties being experienced regarding the transferability of project results.



To the extent that Good Practices are continuously being applied in SMEs, Good Practice projects were automatically sustainable. In addition to which, the identification of a Good Practice per se is not sustainable if it is not accompanied by training or subsequent communication activity. Project holders were confused by this project category. Many project holders presented their projects in the Good Practices category when the project was really focused on training, or when its final destination was a publication or website. Though only 35% of the total number of project fell into this category, 75% of the project holders drafted Good Practice guides. Most of these included Practices considered as being innovative. Dissemination of Good Practices may contribute to creating an awareness among many SMEs.



5- RECOMMENDATIONS FOR FUTURE SME FUNDING SCHEMES This list of recommendations related to the SME Accident Prevention Funding Scheme 2001-2002 has been drawn up on the basis of the evaluation aspects identified in the Restricted Call for Tender (OSHA/TF/2002/T2). The formulation of recommendations hinges upon the analysis drawn up in the report. This list of recommendations should help to improve the number and quality of the applications submitted, the selection process, the services rendered by the Agency and the impact of similar Funding Schemes in the near future. This is the evaluation of the First SME Funding Scheme and represents an assessment of the more administrative aspects (planning, management and monitoring of projects) rather than an evaluation of its results.

The Call for Project Proposals was published in April 2001 and most of the projects did not begin until the end of 2001 or beginning of 2002. They were implemented in 2002 and completed in September of the same year. The evaluation of the Funding Scheme began in November 2002 and finished in March 2003. Therefore, a mere six months have elapsed since the end of the projects. For this reason it is very difficult for us to accurately assess the results and impact of the projects. In addition to which, the Funding Scheme under evaluation is currently engaged in actions to disseminate and publicise the results of the co-funded projects. This prevents the evaluators from assessing the impact of the results of the Scheme from this particular point-of-view. Recommendations have been structured into two different lists:

Priority Recommendations: are recommendations that apply to the entire Funding Scheme procedure, and

Specific Recommendations: are more practical recommendations designed to improve the quality of project results in order to guarantee the quality of applications submitted and improve the promotion of the Funding Scheme.



5.1- PRIORITY RECOMMENDATIONS FOR THE ENTIRE FUNDING SCHEME

Index of Priority Recommendations 1. Turning the SME Accident Prevention Funding Scheme into a Pluri-annual Programme 2. Increasing the visibility of the SME Accident Prevention Funding Scheme 3. Establishing a co-ordination mechanism to facilitate complementarity between different Community

institutions in OSH matters 4. Guaranteeing transparency and impartiality in the evaluation process 5. The need to reduce the entire Funding Scheme procedure 6. The Funding Scheme should facilitate the participation of Eastern European countries 7. Additional staff should be dedicated to handling the SME Accident Prevention Funding Scheme 8. Documents should be published in the highest number of languages possible 9. Projects from the SME 2001-2002 Accident Prevention Funding Scheme should be monitored 1- Turning the SME Accident Prevention Funding Scheme into a Pluri-annual

Programme Lack of time was the main problem of the First Funding Scheme. Attention was drawn to this problem not only by almost all project holders (lack of time to implement their projects), but also by those who took part in the project evaluation process.

The most straightforward recommendation would be to turn the SME Accident Prevention Funding Scheme into a Pluri-annual Programme. In this way, those time restrictions highlighted in the Report that reduced the efficiency of the Funding Scheme could be avoided.

2- Increasing the visibility of the SME Accident Prevention Funding Scheme

The SME 2001-2002 Funding Scheme allocated a budget of 4.5 Million EUR. This allocation enabled 51 projects to be funded, clearly demonstrating the impact on accident prevention in the workplace. Nevertheless, in comparison with other allocations of Community Programmes, this budget is very small and the impact is not so easily perceptible in the SME sector.

Visibility of the SME Accident Prevention Funding Scheme should be enhanced. This could be achieved in different ways:

By financing a fewer number of larger-sized projects (broad partnerships) to

ensure that messages are conveyed and used effectively by others. By increasing the budget for projects and promoting the creation of integrated

projects (projects that include more than one category: Training, Information and Communication and Good Practices).

By making the Funding Scheme an "Integrated Programme". This would allow the creation of a specific Scheme/ Programme for SMEs to be developed according to issues related to accident prevention in the workplace and would



prevent the Funding Scheme from being isolated to a greater or lesser degree from other activities being carried out at Community level.

3- Establishing a co-ordination mechanism to facilitate complementarity between

different Community institutions in OSH matters

The SME Funding Scheme has a high degree of internal coherence in relation to the Agency’s objectives and policies. The Scheme is also coherent with regard to community policies and other similar initiatives under development.

The Funding Scheme has its own characteristics and objectives. It does not overlap other Community Programmes and Activities. In accordance with the Decision of the Council of the 3rd June 2002 concerning New Community Strategy on Safety and Health in the Workplace (2002-2006)16, there exists a need to promote greater integration of Safety and Health in the Workplace within other Community policies. We suggest that a co-ordination mechanism be established in order to assure the complementarity between different Community institutions in matters dealing with OSH, thereby avoiding overlaps and activities having similar contents.

4- Guaranteeing transparency and impartiality in the evaluation process Priority was given to transparency and impartiality of the evaluation process at both national and transnational levels. To achieve this, the Agency provided the national and transnational evaluation teams with common rules and precise guidelines for assessing applications and avoiding “judge and jury problems” and “conflicts of interest” that might have resulted from the fact that evaluators could have been directly, or indirectly, related to the applications being assessed. In this respect, evaluators having any connection whatsoever with a project proposal were not allowed to take part in the evaluation of that particular application. Evaluation permits us to conclude that: The selection procedure was acceptable, transparent and impartial The composition of the national and international evaluation teams was suitable The information provided by the Agency was clear and useful The collaboration and availability of the Agency, as well as its preliminary work,

were considered to be excellent. Nevertheless, the most recommendable option to prevent any problem arising in the application assessment procedure would be to subcontract this work to external

16 2002/C 161/01.



specialists. This external evaluation would have to be carried out under the control and monitorisation of the Agency.

5- The need to reduce the entire Funding Scheme procedure

Were the Funding Scheme to maintain the current, yearly-based system, it would be advisable to slim down the entire procedure. More specifically:

The Application Form should be simplified as much as possible and even the

space in which projects are described should be limited. This would reduce the work involved for potential project applicants as well as for those taking part in the evaluation process.

Much stricter eligibility criteria should be used to ensure the technical and administrative quality of the projects received. This would also avoid the submission of applications having little likelihood of being accepted by the Agency.

6- The Funding Scheme should facilitate the participation of Eastern European

countries

With regard to the future extension of the EU to include Eastern European countries, it should be mentioned that, when this occurs, a major gulf in OSH matters will exist between current Member States and the new candidates. This wide division between both blocks of countries will set up clear differences between the objectives of one or the other in questions of OSH. An efficient manner of reducing the differences in OSH matters between both groups of countries would be to give greater preference to those projects including partners from the new Member States in order to facilitate their participation in the Funding Scheme from the very first year they join the EU.

7- Additional staff should be dedicated to handling the SME Accident Prevention

Funding Scheme

Finally, the good work carried out by the Agency in its planning and management of the 2001-2002 Funding Scheme should be emphasised. In particular, the Agency provided excellent service to potential applicants in its response to their questions. This represented a considerable amount of extra work for the Task Force, who dedicated a great deal of time contacting potential applicants. The Task Force of Campaigns and Programmes, the Unit responsible for the Funding Scheme, managed to put this Scheme into operation with a team made up, basically, of only four people. The team worked in a high work-load environment under a great deal of pressure in order to comply with the objectives and tight deadlines. In view of the timetable of the Funding Scheme, we recommend that more staff should be dedicated to administering this Scheme in the future.



8- Documents should be published in the highest number of languages possible

The Publication of the Call, Application Form and Technical Specifications in eleven languages permitting many project applicants to submit projects that would otherwise have been impossible were appraised very favourably. Nevertheless, it is recommended that all those documents disseminated throughout Europe should contain information on all co-funded projects and be published in the highest number of languages possible. In order to achieve this aim, the possibility of collaborating with National Focal Points in the work of translating these documents should be considered.

9- Projects from the SME 2001-2002 Accident Prevention Funding Scheme should

be monitored We recommend monitoring 2001-2002 Funding Scheme projects in a simple manner, perhaps by means of sending a questionnaire to the project holders or even by including a chapter in the next evaluation of the 2002-2003 Funding Scheme dedicated to following-up projects from the first Funding Scheme. A period of more than 12 months from the completion of the projects would be needed in order to carry out an in-depth analysis of results and their impact.

In general, the results of the Funding Scheme were very satisfactory but, in order to

improve future Funding Schemes, a series of very specific recommendations could be applied.



5.2- SPECIFIC RECOMMENDATIONS FOR THE FUNDING SCHEME

Index of Specific Recommendations 1. The need for more time to complete projects 2. Stressing project sustainability 3. Improving the transferability of project results 4. The Agency should organise a kick-off meeting with project holders 5. Encouraging the participation of SMEs, associations and consortiums 6. The need to support potential applicants in creating partnerships 7. Improving the definition of each project category 8. The need for more restrictive eligibility criteria 9. Improving the clarity of the Call for Project Proposals and Application Form 10. Extending the Call for Project Proposals information campaigns 11. Establishing a minimum number of promotion activities by all National Focal Points in order to disseminate

and publicise the Call for Project Proposals 12. Using the Agency’s website as means as for channelling all the information produced as a result of the

Funding Scheme 1- The need for more time to complete projects

The 51 co-funded projects participating in the 2001-2002 Funding Scheme produced an impact at national and EU levels. The majority of these projects were national and corresponded to the Information and Communication Project category. Project holders complied with the envisaged objectives and, in some cases, even exceeded them, thereby demonstrating the success of the first Funding Scheme. Nevertheless, we would recommend series of recommendations being incorporated that would increase the effectiveness and efficiency of the Scheme.

With reference to the deadline set for completing the projects, a large number of project holders commented that they needed more time in which to carry out the project. The need to submit all financial reports within the project deadline meant that some of the tasks were carried out in a shorter period of time than would have been necessary, thereby reducing the effectiveness of the project. It should be remembered that any delay occurring during the project approval stage reduced the amount of real time available to develop the projects to approximately nine months. This aspect has had a more damaging effect upon smaller organisations that have little experience in developing projects dealing with characteristics and programmes of this nature.

2- Stressing project sustainability

Project sustainability was a criterion used in the project selection procedure. All projects developed under the Funding Scheme were sustainable by definition. However, it has been observed that some project holders confused the terms “sustainability” and “dissemination”, tending to consider that their projects were sustainable when these were disseminated in a proper fashion.



It would therefore be necessary to provide a stricter definition of this term. This might be a way of guaranteeing that future potential applicants make an effort to present projects that are really sustainable. This would also contribute to guaranteeing the presentation of high-quality projects.

3- Improving the transferability of project results

The transferability of project results is a key issue in order to guarantee the success of the Funding Scheme. The following aspects should be insisted upon in order to improve transferability:

An initial, “kick-off” meeting would allow all the project holders to exchange ideas

and experiences with others. This would be a first step towards strengthening the transfer of results between different project holders. The Evaluation Team is adamant in recommending activities of this kind since interviews conducted with project holders have identified a great deal of interest in knowing what was being done in other projects and this might be a good way of allowing exactly this to happen.

The 2001-2002 Funding Scheme has just edited a publication containing the results of all the funded projects. This publication should be circulated widely as an instrument to promote the Funding Scheme at all those events in which the Agency takes part.

The languages in which this publication is written are fundamental factors guaranteeing its usefulness. The publication should be written in at least the following six languages: English, French, German, Spanish, Italian and Portuguese.

4- The Agency should organise an initial “kick-off” meeting with project holders

The Agency should organise a “kick-off” meeting with all those project holders whose projects have been selected. The aim of this meeting would be to explain the procedures to be followed (how Interim and Final Activity Reports must be submitted, how to justify costs, etc.) in addition to stressing the need to comply with established deadlines to each project holder. This would also allow project holders to get to know the Agency personnel they are going to deal with during the implementation of their projects and who they might have to contact in order to resolve any doubts. The Scheme would therefore gain in efficacy and efficiency.

This meeting could also be used as an instrument to disseminate projects completed under the Funding Scheme (a project holder who has already participated in the Scheme should be invited to give an account of his/her experience).

The organisation of a “kick-off” Seminar with project holders should be used as a means for exchanging experiences and identifying areas for improvement. An annual meeting upon completion of projects selection is highly recommended. There is a need to create a forum in which all parties involved in the Funding Scheme can share and



exchange their opinions and experiences. This is also recommended in order to guarantee transfer of project results (see Recommendation Nº 3).

5- Encouraging the participation of SMEs, associations and consortiums

The potential beneficiaries of the Funding Scheme might be extremely varied. This could be considered as a very positive aspect in the sense that the Scheme is open to suggestions from any potential project applicant. The Call for Project Proposals specified that projects could be submitted by SMEs, but the fact that there were virtually no SME project holders is highly significant.

The projects presented involved a number of SMEs, although most applicants were organisations representing employers’ or employees’ interests. Therefore, it would be recommendable to define the objectives of the Funding Scheme with greater precision and encourage the participation of companies forming associations and consortiums in order to respond to the Scheme.

6- The need to support potential applicants in creating partnerships

Another important observation concerns the difficulties faced by project holders in creating partnerships, especially at transnational level. For this reason, it would be appropriate for the Agency to create and disseminate a database including information about those applicants who submitted Applications to the Funding Scheme. The evaluation team is aware of the fact that the Agency is launching the third SME 2003-2004 Funding Scheme. In this case, the database should include contact details of those project applicants belonging to the two previous Schemes. Applicants who submitted applications in previous Schemes could be listed as potential partners for future Schemes. This database would facilitate the presentation of projects, especially transnational projects having larger partnerships.

7- Improving the definition of each project category

The Funding Scheme co-funded three different types of activities: Training, Information and Communication Activities and the Provision of Good Practices. These categories caused a certain amount of confusion among project holders, some of whom did not really know in which category they should participate (while preparing their Application Form), nor did they seem to be completely sure as to exactly which category they were participating (while implementing their projects). It is therefore recommended that each category be defined with greater precision. This is particularly necessary in the "Provision of Good Practices" category, which is not well defined. A number of projects appeared in the Provision of Good Practices category that should really have been included in one of the other two categories. A clear definition of this category would result in increased efficiency for project holders who, on occasions, were unable to determine to which category they should submit their project.



8- The need for more restrictive eligibility criteria

Eligibility criteria should be more restrictive to ensure the quality of the applications submitted and to prevent applications with little likelihood of being accepted from being submitted for evaluation. The use of some of the Priorities as eligibility criteria might be considered. Factors that might be taken into account as eligibility criteria are as follows: Projects should involve a significant number of partners. Projects should involve social partners. Projects with special focus on SMEs or micro-businesses. Projects focused on priority risks, high-risk sectors and new risk-oriented projects. Applicants should be asked to demonstrate the situation of the association and the

capacity of their partners to carry out the actions jointly in order to achieve the anticipated results.

Applicants should be asked to demonstrate that the project does not have any other Community funding for the activities envisaged.

Applicants should propose a realistic and balanced plan for financing. 9- Improving the clarity of the Call for Project Proposals and the Application Form

The Call, Application Form and Technical Specifications were set out in a clear and comprehensible manner. The evaluation also provides specific recommendations that could improve the clarity of the documents: The Call should provide potential applicants with an indication of how their

applications are to be assessed; introducing scores measuring the quality of the technical and financial proposal of the project holder could do this.

The need for a more detailed Budget Sheet allowing each of the expense items to be specified.

Acceptance of a Letter of Intent signed by each participant that would be attached to the Application Form.

Including reference indicators in the Application Form would permit the monitoring and evaluation of projects. Potential applicants would have to quantify these indicators when presenting their Application Form. Indicators which might be considered are as follows:

Common Indicators SMEs benefited Total number of partners Training and Education Total number of trainees Total duration of training courses (hours) Total number of training courses Information/Communication Number of Information and Communication Activities



Total duration (hours) Development of Good Practice Number of Good Practices resulting in an effective reduction of accident risks in SMEs

The information requested from potential applicants in the Application Form is deemed necessary in order to satisfactorily define the application projects submitted, but a number of aspects have been identified as being repetitive. Analysis of project applications shows that some questions were repeated several times, albeit in different fashions. In order to simplify the completion of the Application Form, it is recommended that the contents of each of the questions formulated be specified with greater precision in order to avoid situations in which the same response is given to different questions.

10- Extending the Call for Project Proposals information campaigns

The Funding Scheme was promoted at different levels: Regarding the dissemination of the Call for Project Proposals, it has been observed that a significant percentage of project holders found out about the Scheme not directly through the means used by the Agency, nor through the National Focal Points disseminating this information, but by other, indirect mechanisms such as contacts with other associations or during events that were totally unrelated with OSH matters. We recommend the Agency reinforcing the action taken in order to disseminate the Call for Project Proposals by extending its publicity campaigns.

11- Establishing a minimum number of promotion activities by all the National Focal

Points in order to disseminate and publicise the Call for Project Proposals

At a national level, National Focal Points have collaborated with the Agency in terms of promoting the Funding Scheme. Promotion of the Scheme varied from one National Focal Point to another. This had a direct effect on the number of applications submitted in each country. In order to achieve a greater balance regarding the participation of each country in the Scheme, we would recommend a minimum number of activities being established by all the National Focal Points in order to disseminate and publicise the Call for Project Proposals. In this way, a certain balance would be struck in the conditions under which potential applicants participate in the Scheme, which would favour the participation of all countries under the same conditions. These minimum dissemination activities might take the form, for example, of establishing a minimum number of events to publicise the launching of the Scheme, typology of the minimum number of organisations to be included in the mail shot, etc.



12- Using the Agency’s Website as a means for channelling all the information

produced as a result of the Funding Scheme

Promotion of the Funding Scheme should be improved, giving the SME Accident Prevention Funding Scheme a more visible presence on the Agency's website, including providing more information about the projects receiving funding and a list of partners who have already participated in the Scheme. This would allow potential applicants to search for and contact partners. A database containing information about participants, nationality, size, sector of activity, etc., should be set up.

The Agency’s website should be used as a means for channelling all the information produced as a result of the Funding Scheme.

5.5- RECOMMENDATIONS FOR FUTURE EVALUATIONS As far as monitoring the Funding Scheme is concerned, a set of indicators should be defined for each project category in order to measure the progress of future Funding Schemes and facilitate analysis of their efficiency in future evaluations. Using a set of uniform indicators for each Scheme, the results of all the projects in the Funding Scheme could be grouped together for information purposes. In this way, each project applicant could select a series of indicators for his project from those previously established by the Agency. At the same time, indicators should be defined for each project that could be duly quantified in the project request. A proposal of those indicators that might be used as models is as follows:

Common Indicators SMEs benefiting from the SME 2001-2002 Accident Prevention Funding Scheme Total number of partners Training and Education Total number of trainees Total duration of training courses (hours) Total number of training courses Information and Communication Implemented projects published on Websites, apart from the Agency Website Total information activities Number of copies Development of Good Practice Good Practices resulting in the effective reduction of accident risks in SMEs Projects aiming at stimulating the exchange of Good Practices



The SME Funding Scheme can be considered as a useful and efficient tool for accident prevention in SMEs throughout the EU

The more than 50 co-funded projects have allowed the implementation of a series of training, information and publicity activities, in addition to providing Good Practices

that contribute efficiently to achieving the Scheme’s objectives.

In the opinion of the evaluators, the results of the Scheme are sufficiently sound to maintain this Funding Scheme without major changes over coming years.

The Funding Scheme has its own characteristics and objectives. It does not overlap other Community programmes and activities; on the contrary, the Funding Scheme

complements other Community initiatives and activities.

The Call, Application Form and Technical Specifications have been produced in a much clearer way than in other Community Programmes, such as ESF. This has

been highly appreciated by project holders.

However, we would recommend a number of minor changes being incorporated in order to not only make the Scheme work better, but also improve its results.


EVALUATION OF THE SME 2001-2002 ACCIDENT PREVENTION FUNDING SCHEME 97

ANNEX1: METHODOLOGY USED IN COLLECTION AND ANALYSIS OF INFORMATION

Collection of information from the

Agency

Kick-off meeting

Documents provided by the

Agency

51 Projects: Application Forms and Final Reports

Identification and preliminary analysis of

essential aspects of the Funding Scheme

Agency

Project holders

National Focal Points

Key interested

parties

Personal interviews with: *Agency *Project holders* *Interested parties

Check list for Interviews: *Agency *National Focal Points *Project holders

Interviews

Elaboration of Questionnaires

for project holdersand FOP

Survey

Information generated in the

Evaluation:

Summary of Findings

Reinforcement activities



Annex 2: Questionnaires for Project Holders

QUESTIONNAIRE FOR PROJECT HOLDERS


PLEASE SEND THIS QUESTIONNAIRE TO THE FOLLOWING E-MAIL ADDRESS: [email protected]

Should you encounter any problem whatsoever, please call (+34) 944 797 686 and ask for Joseba Gojenola, or send a fax to (+34) 944 761 804

GENERAL INFORMATION Project Title

Project Holder

Name of Contact

Project Category 1-Training 2-Information and Communication 3-Provision of Good Practices

1- EU Project 2- National Project

IDOM INGENIERÍA Y CONSULTORÍA

THANK YOU VERY MUCH FOR YOUR COLLABORATION



1- How did you hear about the Funding Scheme? (Press, Agency network, etc.) Please specify. 2- Were the Call for Project Proposals, the Application Form and the Technical Specifications drawn up in such a clear and understandable way as to provide sufficient information regarding potential applications? Yes( ).............................................................. No ( )................................................................... If not, please explain how you feel they could be improved Call ( )

Application Form ( )

Technical Specifications ( )

3-Were the established deadlines satisfactory? ( ) Yes No If not, please explain why. Presenting the Application

Submitting the Final Report

Achieving the project objectives, products and targets

Running the projects

4-Please assess the SME Funding Scheme according to the following priorities in Occupational Safety and Health Score from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) 1 to 5 Heightening or improving awareness of accident risks Promoting the development and the identification of Effective Good Practice Promoting Occupational Safety and Health at Work as part of business thinking Promoting innovative practices to prevent accidents in SMEs Promoting workforce participation in Occupational Safety and Health Promoting the participation of social agents in Occupational Safety and Health Other specific needs of SMEs

5-How have the workforce and the social partners been involved in the project? ( ) Workforce Social partners 1- Participation in the planning of the project 2- Direct participation in the implementation of the project 3- Providing/receiving training 4- Providing/receiving information 5- Participation in Seminars 6- They have not participated Other – please specify



6- In the case of responding affirmatively to Points 1 (Participating in the planning of the project) or 2 (Direct participation in the implementation of the project), please explain how the workforce was involved. 7-Please evaluate the participation of the workforce and the social partners in the project using a scale from 1 to 5. (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) Workforce Social partners

8-Through your project, how many SMEs have benefited?. Please specify the number.................................................................................................................. 9-Did the project meet the important needs in your organisation relating to Occupational Safety and Health? Please rate your answer using a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) and explain why. 10-In your opinion, how has the SME Accident Prevention Funding Scheme contributed to preventing accidents at work? Please rate your answer using a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) and explain why. 11-Please describe the added value provided by the project to your organisation.

12-To what extent have the objectives of your project been achieved? Please rate your answer using a scale from 1 to 4 (1: not achieved, 2: almost achieved, 3: achieved, 4: mainly achieved). Should objectives not have been achieved, please explain why.

13-Would your project have been carried out without the Agency funding?

( ) Yes No Yes, and in a similar way Yes, but with fewer resources and less efficiency Yes, but without sustainability Yes, but less promoted No. Others – please specify



14-Might your project outcome be easily transferable to other SME or sectors?

Yes No If Yes, please explain which effects and to what extent ( )

15-Do you anticipate the project having a lasting positive effect in relation to the prevention of work related accidents? 16-If your project was carried out with partner organisations, how would you evaluate the partner(s) performance and added value? Please rate your answer using a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) 17-Please express your level of satisfaction regarding the service provided by the Agency or the National Focal Point regarding their answering questions

On a scale of 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent)

Agency National Focal Points

18-How was the Agency and the National Focal Point performance during the project implementation period in relation to the following aspects?

On a scale of 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent)

Agency National Focal Point

Answering your questions ( ) Providing additional information ( ) Supporting you project ( ) Monitoring your project ( ) Other, please specify ( )

19-Have you implemented any training activities? Yes ( ) ............................................................... No ( )............................................................. If so, please complete the following box

( ) With whom? Number of participants

How many hours long?

SME Managers SME Owners People responsible for Safety and Heath All workforce Others



20-Have you implemented any Information and Communication Activities? Yes ( ) ............................................................... No ( )........................................................ If so, with what specific objective?

( ) Kind of Activities To improve access to Occupational Safety and Health Information

To encourage the participation of workers in Occupational Safety and Health Practices

To communicate the Occupational Safety and Health message

Other 21-Have you provided Effective Good Practices to reduce accident risks in SMEs as a result of the implementation of your project? Yes ( ) ............................................................... No ( )........................................................

If so, with what specific objective? ( ) Objective

Number of Good Practices

Have you transferred your Good Practice?

If so, to how many Organisations?

Innovative Good Practices Focusing on high risks Other

22-Has your project been sufficiently promoted?

( ) Yes No If not, please explain why Internally to: ( ) Your management Your technicians All workforce Externally to: ( ) Other bodies (trade unions, entrepreneurial

organisations, etc)

Other SMEs Press and media

By the Agency ( )



Annex 3: Questionnaires for National Focal Points

QUESTIONNAIRE FOR NATIONAL FOCAL POINTS


PLEASE SEND THIS QUESTIONNAIRE TO THE FOLLOWING E-MAIL ADDRESS: [email protected]

Should you encounter any problem whatsoever, please call (+34) 944 797 686 and ask for Joseba Gojenola, or send a fax to (+34) 944 761 804

GENERAL INFORMATION Country

Name of Contact

IDOM INGENIERÍA Y CONSULTORÍA

THANK YOU VERY MUCH FOR YOUR COLLABORATION



1-Please assess the SME Accident Prevention Funding Scheme in relation to the SME needs and priorities in Occupational Safety and Health.

Scale: 1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent Score from 1 to 5 Heightened awareness of accident risks Promoting the development and identification of Effective Good Practices Promoting the development of risk evaluation and prevention practice Encouraging the development of sustainable and European added value regarding Occupational Safety and Health Activities

Contributing to reducing work-related accidents in SMEs Promoting Occupational Safety and Health at Work as part of business thinking

2-Have you implemented any Information and Communication Activity at national level thanks to the SME Funding Scheme? Yes ( ) ............................................................... No ( )........................................................ If so, with what specific objective?

( ) Which? Number of activities

Duration (in hours)

To improve access to Occupational Safety and Health Information

To encourage the participation of workers in Occupational Safety and Health Practices

To communicate the Occupational Safety and Health message

Other 3-How would you rate the participation of the Tripartite side Focal Point Network in the SME Funding Scheme? Using a scale of 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) 4-In your opinion, are there any specific tasks undertaken by the Tripartite side Focal Point Network? Which are not? Please specify 5-In your opinion, were the social partners and the workforce effectively involved in the conception and implementation of the projects? Social partners Workforce Yes ( ) No ( ) 6-Were the Call for Project Proposals and the Technical Specifications drawn up in such a clear and understandable way as to provide sufficient information for potential applications?

Yes( ).............................................................. No ( ).................................................................. If not, please explain how they could be improved

Call ( )

Application Form ( )

Technical Specifications ( )

7-Was the submission deadline satisfactory?

Yes No If not, please explain why ( )



8-Were you properly informed about the processing of applications?

Yes No If not, please explain why ( )

9-How would you evaluate the performance of the SME-EW GROUP? 10-What is your overall opinion of the quality of the project applications submitted? Please specify using a score from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) 11-What is your assessment of the Agency regarding the following aspects?

1 to 5 If in need of improvement, please explain how this might be achieved

Organising the selection process

Processing applications

Organising the Selection Committee Meeting

12-Was the selection procedure satisfactory, transparent and fair?

Using a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent)

Reasonable Transparent Fair

Please score from 1 to 5

13-In your opinion, did the SME Accident Prevention Funding Scheme facilitate the Exchange of Good Practices? Please specify in what way. 14-Please indicate the potential main-lasting, positive effects of the Funding Scheme.



Annex 4: Data from Survey (Project Holders)

Question 1 - How did you hear about the Funding Scheme? (Press, Agency network, etc.) Please specify.

Agency National Focal Point Network Internet O.J.E.C. Others

Total 15.6% 26.7% 31.1% 4.4% 22.2%

Number of Replies 7 12 14 2 10

Question 2 - Were the Call for Project Proposals, the Application Form and the Technical Specifications formulated in such a clear and understandable way so as to provide sufficient information for potential applications?

Call Application Form Technical Specifications

Yes 93.5% 84.8% 84.8%

No 6.5% 15.2% 15.2%

Total 100% 100% 100%

Number of Replies 46 46 46

Question 3 - Were deadlines satisfactory?

When Submitting the Application

When Submitting the Final Report

Regarding Achieving the Project Objectives,

Products and Targets Regarding Running

the Projects

Yes 95.7% 52.2% 58.7% 52.2%

No 4.3% 47.8% 41.3% 47.8%

Total 100% 100% 100% 100%

Number of Replies 46 46 46 46



Question 4 - Please rate the SME Funding Scheme in relation to the following priorities in Occupational Safety and Health

Heightening the

Awareness of Accident

Risks

Promoting the Development

and Identification of Effective Good

Practice

Promoting Occupational

Safety and Health at Work

as Part of Business Thinking

Promoting Innovative

Practices to Prevent

Accidents in SMEs

Promoting Workforce

Participation in Occupational

Safety and Health

Promoting the Participation of Social Agents in

Occupational Safety and

Health

Other Than Specific Needs of

SMEs

5 38.5% 36.4% 15.6% 17.8% 11.6% 8.6% 5.0%

4 46.2% 40.9% 55.6% 51.1% 44.2% 42.9% 30.0%

3 12.8% 20.5% 26.7% 22.2% 32.6% 37.1% 60.0%

2 0.0% 0.0% 0.0% 6.7% 9.3% 2.9% 0.0%

1 2.6% 2.3% 2.2% 2.2% 2.3% 8.6% 5.0%

Total 100% 100% 100% 100% 100% 100% 100%

Number of Replies 39 44 45 45 43 35 20

Question 5 - How have the workforce and social partners been involved in the project?

Participating in the

Planning of the Project

Directly Participating in the Implementation of the

Project Receiving Training

Receiving Information

Participating in Seminars

Workforce 38.46% 48.98% 67.50% 52.24% 54.17%

Social Partners 61.54% 51.02% 32.50% 47.76% 45.83%

Total 100% 100% 100% 100% 100%

Number of Replies 39 49 40 67 48



Question 7 – Please rate the participation of the workforce and social partners in the project using a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent

Workforce Social Partners

5 38.5% 30.0%

4 41.0% 37.5%

3 15.4% 27.5%

2 2.6% 5.0%

1 2.6% 0.0%

Total 100% 100%

Number of Replies 39 40

Question 9 - Did the project meet the important needs in your organisation regarding Occupational Safety and

Health? Please rate using a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) and explain why.

5 33.3%

4 55.6%

3 4.4%

2 0.0%

1 6.7%

Total 100%

Number of Replies 45

Question 10 - In your opinion, how has the SME Accident Prevention Funding Scheme contributed to

preventing accidents at work? Please rate using a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) and explain why.

5 29.5%

4 52.3%

3 13.6%

2 2.3%

1 2.3%

Total 100%




Question 12. - To what extent have the objectives of your project been achieved? Please rate on a scale from 1 to 4 (1: not achieved, 2: almost achieved, 3: achieved, 4: mostly achieved). In the case of objectives not

having been achieved, please explain why not.

5 2.2%

4 60.0%

3 28.9%

2 8.9%

1 0.0%

Total 100%


Question 13 - Would your project have been carried out without Agency funding?

Yes, and in a Similar Fashion

Yes, but with Fewer Resources

and Less Efficiency

Yes, but Without Sustainability

Yes, but Less Promoted No Others – Please

Specify

Yes 5.9% 56.7% 26.3% 57.7% 12.5% 18.8%

No 94.1% 43.3% 73.7% 42.3% 87.5% 81.3%

Total 100% 100% 100% 100% 100% 100%

Number of Replies 17 30 19 26 24 16

Question 14 – Is your project outcome easily transferable to other SMEs or sectors?

Yes 90.9%

No 9.1%

Total 100%




Question 16 - If your project was carried out with partner organisations, how would you evaluate the partner(s) performance and added value? Please rate using a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good,

5: Excellent)

5 45.9%

4 37.8%

3 16.2%

2 0.0%

1 0.0%

Total 100%


Question 17 – Please rate your level of satisfaction regarding the service provided by the Agency or National

Focal Point answering questions on a scale from 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) Agency National Focal Points

5 54.8% 35.0%

4 38.1% 45.0%

3 2.4% 10.0%

2 4.8% 5.0%

1 0.0% 5.0%

Total 100% 100%




Question 18 – Please rate the performances of the Agency and the National Focal Point during the project implementation period regarding the following aspects

Agency National Focal Points Answering

Questions Providing Additional

Information Supporting

Your ProjectMonitoring

Your ProjectAnswering Questions

Providing Additional

Information Supporting

Your ProjectMonitoring

Your Project

5 61.0% 41.0% 28.3% 36.4% 16.7% 15.4% 25.0% 0.0%

4 31.7% 41.0% 30.0% 45.5% 75.0% 61.5% 66.7% 60.0%

3 7.3% 17.9% 15.0% 12.1% 0.0% 15.4% 0.0% 30.0%

2 0.0% 0.0% 13.3% 0.0% 8.3% 0.0% 0.0% 0.0%

1 0.0% 0.0% 13.3% 6.1% 0.0% 7.7% 8.3% 10.0%

TOTAL 100% 100% 100% 100% 100% 100% 100% 100%

Nº OF ANSWERS 41 39 60 33 12 13 12 10

Question 19 - Have you implemented any training activities?

Yes 55.6%

No 44.4%

Total 100%


SME Managers SME Owners Responsible for

Safety and Health All Workforce

Number of Participants 270 240 50 744

Number of Hours 10 10 15 518




Question 20 - Have you implemented any Information and Communication activity?

To Improve Access to

Occupational Safety and Health Information

To Encourage the Participation of Workers in Occupational Safety and

Health Practices

To Communicate the Occupational Safety and

Health Message Other

Yes 79.5% 77.8% 69.2% 38.5%

No 20.5% 22.2% 30.8% 61.5%

Total 100% 100% 100% 100%


Question 21 - Have you provided Effective Good Practices to reduce accident risks in SMEs as a result of the

implementation of your project?

Yes 75.0%

No 25.0%

Total 100%


Innovative Good Practices Focusing on High Risks

Number of Good Practices 1,550 89

Yes 53.5% 18.4%

No 46.5% 81.6%

Total 100% 100%

Have You Transferred Your Good Practice?


If Transferred, to How Many Organisations? 3,930 1,848



Question 22 - Has your project been sufficiently promoted?

Internally to: Externally to:

Your Management

Your Technicians All Workforce

Other Bodies (Trade Unions, Entrepreneurial

Organisations, Etc.) Other SMEs

Press and Media

By the Agency

Yes 82.6% 77.8% 78.9% 82.6% 81.4% 80.5% 75.8%

No 17.4% 22.2% 21.1% 17.4% 18.6% 19.5% 24.2%

Total 100% 100% 100% 100% 100% 100% 100%

Number of Replies 46 36 38 46 43 41 33



Annex 5: Data From Survey (NATIONAL FOCAL POINTS)

Question 1 - Please rate the SME Accident Prevention Funding Scheme in relation to SME needs and priorities in Occupational Safety and Health using a scale of 1 to 5. (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5:

Excellent)

Heightening

Awareness of Accident Risks

Promoting the Development and Identification of Effective Good

Practices

Promoting the Development of Risk Evaluation and Prevention

Practice

Encouraging the Development of Sustainable and European Added

Value on Occupational Safety

and Health Activities

Contributing to the Reduction

of Work-related Accidents in

SMEs

Promoting Occupational

Safety and Health at Work as Part of Business Thinking

5 26.7% 6.7% 13.3% 6.7% 13.3% 6.7%

4 40.0% 66.7% 53.3% 46.7% 33.3% 46.7%

3 26.7% 20.0% 33.3% 33.3% 46.7% 26.7%

2 6.7% 0.0% 0.0% 13.3% 6.7% 13.3%

1 0.0% 6.7% 0.0% 0.0% 0.0% 6.7%

Total 100% 100% 100% 100% 100% 100%

Number of Replies 15 15 15 15 15 15



Question 2 - Have you implemented any Information and Communication activity at national level related to the SME Funding Scheme?

Yes 66.7%

No 33.3%

Total 100%


To Improve Access to Occupational Safety

and Health Information

To Encourage the Participation of Workers in Occupational Safety and Health Practices

To Communicate the Occupational Safety and

Health Message Other

Yes 26.7% 6.7% 20.0% 6.7%

No 73.3% 93.3% 80.0% 93.3%

Total 100% 100% 100% 100%


Number of Activities 26 20 34 10

Duration (in Hours) 325 300 325

Question 3 - How would you rate the participation of the Tripartite side Focal Point Network in the SME

Funding Scheme on a scale of to 5? (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent) 5 35.7% 4 42.9% 3 14.3% 2 7.1% 1 0.0%

Total 100.0%




Question 5 - In your opinion, were the social partners and workforce effectively involved in the conception and implementation of the projects?

Social Partners Workforce

Yes 100.0% 73.3%

No 0.0% 26.7%

Total 100% 100%


Question 6 - Were the Call for Project Proposals, the Application Form and the Technical Specifications

formulated in such a clear and understandable way as to provide sufficient information for potential applications?

Call Application Form Technical Specifications

Yes 100% 67% 93%

No 0.0% 33% 7%

Total 100% 100% 100%


Question 7 – Was the project submission deadline sufficient?

Yes 46.7%

No 53.3%

Total 100%


Question 8 – Were you properly informed about the processing of applications?

Yes 93.3%

No 6.7%

Total 100%




Question 10 - What is your overall opinion of the quality of the project applications submitted? Please rate using a scale of 1 to 5 (1:Very bad, 2: Bad, 3: Medium, 4: Good, 5: Excellent)

5 0%

4 54%

3 38%

2 0%

1 8%

Total 100%


Question 11 - How would you rate the Agency regarding the following aspects?

Organisation of the Selection Process

Processing Applications

Organisation of the Selection Committee

Meeting

5 27% 20.0% 14.3%

4 40% 60.0% 50.0%

3 33% 20.0% 35.7%

2 0% 0.0% 0.0%

1 0% 0.0% 0.0%

Total 100% 100% 100%




Question 12 - Was the selection procedure reasonable, transparent and fair?

Reasonable Transparent Fair

5 13.3% 13.3% 13.3%

4 60.0% 40.0% 40.0%

3 26.7% 40.0% 40.0%

2 0.0% 6.7% 6.7%

1 0.0% 0.0% 0.0%

Total 100% 100% 100%


SME-EU GROUP

5 29%

4 36%

3 21%

2 14%

1 0%

Total 100%
